I've given concurrent.futures.ThreadPoolExecutor
a bunch of tasks, and I want to wait until they're all completed before proceeding with the flow. How can I do that, without having to save all the futures and call wait
on them? (I want an action on the executor.)
ThreadPoolExecutor. ThreadPoolExecutor is an Executor subclass that uses a pool of threads to execute calls asynchronously. An Executor subclass that uses a pool of at most max_workers threads to execute calls asynchronously.
ThreadPoolExecutor Methods : submit(fn, *args, **kwargs): It runs a callable or a method and returns a Future object representing the execution state of the method. map(fn, *iterables, timeout = None, chunksize = 1) : It maps the method and iterables together immediately and will raise an exception concurrent. futures.
Just call Executor.shutdown
:
shutdown(wait=True)
Signal the executor that it should free any resources that it is using when the currently pending futures are done executing. Calls to
Executor.submit()
andExecutor.map()
made after shutdown will raiseRuntimeError
.If wait is
True
then this method will not return until all the pending futures are done executing and the resources associated with the executor have been freed.
However if you keep track of your futures in a list then you can avoid shutting the executor down for future use using the futures.wait()
function:
concurrent.futures.wait(fs, timeout=None, return_when=ALL_COMPLETED)
Wait for the
Future
instances (possibly created by differentExecutor
instances) given byfs
to complete. Returns a named 2-tuple of sets. The first set, named done, contains the futures that completed (finished or were cancelled) before the wait completed. The second set, named not_done, contains uncompleted futures.
note that if you don't provide a timeout
it waits until all futures have completed.
You can also use futures.as_completed()
instead, however you'd have to iterate over it.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With