right way to run some code with timeout in Python

Question

I looked online and found some SO discussing and ActiveState recipes for running some code with a timeout. It looks there are some common approaches:

• Use thread that run the code, and join it with timeout. If timeout elapsed - kill the thread. This is not directly supported in Python (used private _Thread__stop function) so it is bad practice
• Use signal.SIGALRM - but this approach not working on Windows!
• Use subprocess with timeout - but this is too heavy - what if I want to start interruptible task often, I don't want fire process for each!

So, what is the right way? I'm not asking about workarounds (eg use Twisted and async IO), but actual way to solve actual problem - I have some function and I want to run it only with some timeout. If timeout elapsed, I want control back. And I want it to work on Linux and Windows.

Answer 1

A completely general solution to this really, honestly does not exist. You have to use the right solution for a given domain.

• If you want timeouts for code you fully control, you have to write it to cooperate. Such code has to be able to break up into little chunks in some way, as in an event-driven system. You can also do this by threading if you can ensure nothing will hold a lock too long, but handling locks right is actually pretty hard.

• If you want timeouts because you're afraid code is out of control (for example, if you're afraid the user will ask your calculator to compute 9**(9**9)), you need to run it in another process. This is the only easy way to sufficiently isolate it. Running it in your event system or even a different thread will not be enough. It is also possible to break things up into little chunks similar to the other solution, but requires very careful handling and usually isn't worth it; in any event, that doesn't allow you to do the same exact thing as just running the Python code.

Answer 2

What you might be looking for is the multiprocessing module. If subprocess is too heavy, then this may not suit your needs either.

import time import multiprocessing  def do_this_other_thing_that_may_take_too_long(duration):     time.sleep(duration)     return 'done after sleeping {0} seconds.'.format(duration)  pool = multiprocessing.Pool(1) print 'starting....' res = pool.apply_async(do_this_other_thing_that_may_take_too_long, [8]) for timeout in range(1, 10):     try:         print '{0}: {1}'.format(duration, res.get(timeout))     except multiprocessing.TimeoutError:         print '{0}: timed out'.format(duration)   print 'end'