Individual timeouts for concurrent.futures

Question

I see two ways to specify timeouts in concurrent.futures.

• as_completed()
• wait()

Both methods handle N running futures.

I would like to specify an individual timeout for each future.

Use Case:

• Future for getting data from DB has a timeout of 0.5 secs.
• Future for getting data from a HTTP server has a timeout of 1.2 secs.

How do I handle this with concurrent.futures? Or is this library not the right tool?

Conclusion

• AFAIK the solution by mdurant is a good work-around.
• I think I will use a different library the next time. Maybe asyncio has better support for this. See: https://docs.python.org/3/library/asyncio-task.html#asyncio.sleep

Answer 1

How about implementing your own:

wait(dbfutures + httpfutures, timeout=0.5)
[fut.cancel() for fut in bdfutures if not fut.done()]
wait(httpfutures, timeout=0.7)
[fut.cancel() for fut in httpfutures if not fut.done()]

(or a while loop with sleep/check or wait with short timeout)