Memory leak using tornado's gen.engine

Question

I have code which, in a simplified form, looks like this:

from tornado import gen, httpclient, ioloop

io_loop = ioloop.IOLoop.instance()
client = httpclient.AsyncHTTPClient(io_loop=io_loop)

@gen.engine
def go_for_it():
    while True:
        r = yield gen.Task(fetch)

@gen.engine
def fetch(callback):
    response = yield gen.Task(client.fetch, 'http://localhost:8888/')
    callback(response)

io_loop.add_callback(go_for_it)
io_loop.start()

When I run it the memory footprint keeps increasing over time more or less linearly. If, however, I remove the gen.engine nesting:

@gen.engine
def go_for_it():
    while True:
        r = yield gen.Task(client.fetch, 'http://localhost:8888/')

memory usage remains constant.

I've managed to reproduce the issue with different versions of tornado 2, on both Mac OS X and Linux. Any ideas what might be the cause of this problem?

Answer 1

Digging into it with the help of objgraph package, it looks like the code leaks ExceptionStackContexts. These are created by gen.engine to handle function exceptions, they should be cleared up but clearly you found a bug.

My best guess is that the there is a reference left somewhere to one.

EDIT - The Tornado Team (Ben) has found a fix and it will be in a future release. https://github.com/facebook/tornado/commit/bff07405549a6eb173a4cfc9bbc3fc7c6da5cdd7