Simple async example with tornado python

Question

I want find simple async server example. I have got some function with lot of wait, database transactions ... etc:

def blocking_task(n):
    for i in xrange(n):
        print i
        sleep(1)
    return i

I need run it function in separated process without blocking. Is it possible?

Answer 1

import tornado.web
from tornado.ioloop import IOLoop
from tornado import gen

from tornado.concurrent import run_on_executor
from concurrent.futures import ThreadPoolExecutor   # `pip install futures` for python2

MAX_WORKERS = 16

class TestHandler(tornado.web.RequestHandler):
    executor = ThreadPoolExecutor(max_workers=MAX_WORKERS)

    """
    In below function goes your time consuming task
    """

    @run_on_executor
    def background_task(self):
        sm = 0
        for i in range(10 ** 8):
            sm = sm + 1

        return sm

    @tornado.gen.coroutine
    def get(self):
        """ Request that asynchronously calls background task. """
        res = yield self.background_task()
        self.write(str(res))

class TestHandler2(tornado.web.RequestHandler):
    @gen.coroutine
    def get(self):
        self.write('Response from server')
        self.finish()


application = tornado.web.Application([
    (r"/A", TestHandler),
    (r"/B", TestHandler2),
    ])

application.listen(5000)
IOLoop.instance().start()

When you run above code, you can run a computationally expensive operation at http://127.0.0.1:5000/A , which does not block execution, see by visiting http://127.0.0.1:5000/B immediately after you visit http://127.0.0.1:5000/A.

Answer 2

Python 3.5 introduced the async and await keywords (functions using these keywords are also called “native coroutines”). For compatibility with older versions of Python, you can use “decorated” or “yield-based” coroutines using the tornado.gen.coroutine decorator.

Native coroutines are the recommended form whenever possible. Only use decorated coroutines when compatibility with older versions of Python is required. Examples in the Tornado documentation will generally use the native form.

Translation between the two forms is generally straightforward:

# Decorated:                    # Native:

# Normal function declaration
# with decorator                # "async def" keywords
@gen.coroutine
def a():                        async def a():
    # "yield" all async funcs       # "await" all async funcs
    b = yield c()                   b = await c()
    # "return" and "yield"
    # cannot be mixed in
    # Python 2, so raise a
    # special exception.            # Return normally
    raise gen.Return(b)             return b

Other differences between the two forms of coroutine are outlined below.

• Native coroutines:

  • are generally faster.
  • can use async for and async with statements which make some patterns much simpler.
  • do not run at all unless you await or yield them. Decorated coroutines can start running “in the background” as soon as they are called. Note that for both kinds of coroutines it is important to use await or yield so that any exceptions have somewhere to go.

• Decorated coroutines:

  • have additional integration with the concurrent.futures package, allowing the result of executor.submit to be yielded directly. For native coroutines, use IOLoop.run_in_executor instead.
  • support some shorthand for waiting on multiple objects by yielding a list or dict. Use tornado.gen.multi to do this in native coroutines.
  • can support integration with other packages including Twisted via a registry of conversion functions. To access this functionality in native coroutines, use tornado.gen.convert_yielded.
  • always return a Future object. Native coroutines return an awaitable object that is not a Future. In Tornado the two are mostly interchangeable.

---

Worth to see:

• Simplest async/await example