How can I use Tornado and Redis asynchronously?

Question

I'm trying to find how can I use Redis and Tornado asynchronously. I found the tornado-redis but I need more than just add a yield in the code.

I have the following code:

import redis
import tornado.web

class WaiterHandler(tornado.web.RequestHandler):

    @tornado.web.asynchronous
    def get(self):
        client = redis.StrictRedis(port=6279)
        pubsub = client.pubsub()
        pubsub.subscribe('test_channel')

        for item in pubsub.listen():
            if item['type'] == 'message':
                print item['channel']
                print item['data']

        self.write(item['data'])
        self.finish()


class GetHandler(tornado.web.RequestHandler):

    def get(self):
        self.write("Hello world")


application = tornado.web.Application([
    (r"/", GetHandler),
    (r"/wait", WaiterHandler),
])

if __name__ == '__main__':
    application.listen(8888)
    print 'running'
    tornado.ioloop.IOLoop.instance().start()

I need getting access the / url and get the "Hello World" while there's a request pending in the /wait. How can I do it?

Answer 1

You should not use Redis pub/sub in the main Tornado thread, as it will block the IO loop. You can handle the long polling from web clients in the main thread, but you should create a separate thread for listening to Redis. You can then use ioloop.add_callback() and/or a threading.Queue to communicate with the main thread when you receive messages.

Answer 2

You need to use Tornado IOLoop compatible redis client.

There are few of them available, toredis, brukva, etc.

Here's pubsub example in toredis: https://github.com/mrjoes/toredis/blob/master/tests/test_handler.py