How does long-polling work in Tornado?

Question

In Tornado's chat demo, it has a method like this:

@tornado.web.asynchronous
def post(self):
    cursor = self.get_argument("cursor", None)
    global_message_buffer.wait_for_messages(self.on_new_messages,
                                            cursor=cursor)

I'm fairly new to this long polling thing, and I don't really understand exactly how the threading stuff works, though it states:

By using non-blocking network I/O, Tornado can scale to tens of thousands of open connections...

My theory was that by making a simple app:

import tornado.ioloop
import tornado.web
import time

class MainHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    def get(self):
        print("Start request")
        time.sleep(4)
        print("Okay done now")
        self.write("Howdy howdy howdy")
        self.finish()

application =  tornado.web.Application([
    (r'/', MainHandler),
])

That if I made two requests in a row (i.e. I opened two browser windows and quickly refreshed both) I would see this:

Start request
Start request
Okay done now
Okay done now

Instead, I see

Start request
Okay done now
Start request
Okay done now

Which leads me to believe that it is, in fact, blocking in this case. Why is it that my code is blocking, and how do I get some code to do what I expect? I get the same output on Windows 7 with a core i7, and a linux Mint 13 box with I think two cores.

Edit:

I found one method - if someone can provide a method that works cross-platform (I'm not too worried about performance, only that it's non-blocking), I'll accept that answer.

Answer 1

The right way to convert your test app into a form that won't block the IOLoop is like this:

from tornado.ioloop import IOLoop
import tornado.web
from tornado import gen
import time

@gen.coroutine
def async_sleep(timeout):
    """ Sleep without blocking the IOLoop. """
    yield gen.Task(IOLoop.instance().add_timeout, time.time() + timeout)

class MainHandler(tornado.web.RequestHandler):
    @gen.coroutine
    def get(self):
        print("Start request")
        yield async_sleep(4)
        print("Okay done now")
        self.write("Howdy howdy howdy")
        self.finish()

if __name__ == "__main__":
    application =  tornado.web.Application([
        (r'/', MainHandler),
    ])
    application.listen(8888)
    IOLoop.instance().start()

The difference is replacing the call to time.sleep with one which won't block the IOLoop. Tornado is designed to handle lots of concurrent I/O without needing multiple threads/subprocesses, but it will still block if you use synchronous APIs. In order for your long-polling solution to handle concurrency the way you'd like, you have to make sure that no long-running calls block.