Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using tornado with aiohttp (or other asyncio-based libraries)

I want to use tornado with asyncio libraries like aiohttp and native python 3.5 coroutines, and it seems to be supported in the latest tornado release (4.3). However when using it in tornado event loop, the request handler hangs indefinitely. When not using aiohttp (i.e. without the lines r = await aiohttp.get('http://google.com/') and text = await r.text() below), the request handler proceeds as normal.

My test code is as follows:

from tornado.ioloop import IOLoop
import tornado.web
import tornado.httpserver
import aiohttp

IOLoop.configure('tornado.platform.asyncio.AsyncIOLoop')


class MainHandler(tornado.web.RequestHandler):
    async def get(self):
        r = await aiohttp.get('http://google.com/')
        text = await r.text()
        self.write("Hello, world, text is: {}".format(text))

if __name__ == "__main__":
    app = tornado.web.Application([
        (r"/", MainHandler),
    ])
    server = tornado.httpserver.HTTPServer(app)
    server.bind(8888, '127.0.0.1')
    server.start()
    IOLoop.current().start()
like image 455
Leonth Avatar asked Jan 13 '16 06:01

Leonth


People also ask

Does tornado use Asyncio?

Tornado is a web framework that tries to solve Python's synchronous nature. Since it uses asyncio, with Tornado you can create a node. js-like web server that runs asynchronously, thus able to handle many more requests than if it were running synchronously.

Is tornado asynchronous?

Tornado is a Python web framework and asynchronous networking library, originally developed at FriendFeed.

How does the Asyncio event loop work?

The event loop is the core of every asyncio application. Event loops run asynchronous tasks and callbacks, perform network IO operations, and run subprocesses. Application developers should typically use the high-level asyncio functions, such as asyncio.

What is an Asyncio event?

An asyncio event can be used to notify multiple asyncio tasks that some event has happened. An Event object manages an internal flag that can be set to true with the set() method and reset to false with the clear() method. The wait() method blocks until the flag is set to true. The flag is set to false initially.


2 Answers

Starting from Tornado 5, most of its async functions, classes and decorators, including IOLoop, are not only compatible with Python's standard asyncio, but are aliases to its counterparts when running on Python 3.5+.

That means when you use Tornado's things like IOLoop(), @gen.coroutine and such, behind the scenes Tornado uses the equivalent functions and classes in asyncio.

This way you can use IOLoop.current().start() and you'll get asyncio's ioloop.

like image 118
MestreLion Avatar answered Sep 20 '22 06:09

MestreLion


According to docs, you are doing it almost right. You have to create/init Tornado's ioloop with corresponding asyncio, since aiohttp is running on asyncio.

from tornado.ioloop import IOLoop
import tornado.web
import tornado.httpserver
import aiohttp
from tornado.platform.asyncio import AsyncIOMainLoop
import asyncio

class MainHandler(tornado.web.RequestHandler):
    async def get(self):
        r = await aiohttp.get('http://google.com/')
        text = await r.text()
        self.write("Hello, world, text is: {}".format(text))

if __name__ == "__main__":
    AsyncIOMainLoop().install()
    app = tornado.web.Application([
        (r"/", MainHandler),
    ])
    server = tornado.httpserver.HTTPServer(app)
    server.bind(1234, '127.0.0.1')
    server.start()
    asyncio.get_event_loop().run_forever()

The reason why your code get stuck, is that asyncio's ioloop actually is not running, only the Tornado's one, so await is waiting indefinitely.

like image 39
kwarunek Avatar answered Sep 22 '22 06:09

kwarunek