Py3.6 :: ThreadPoolExecutor future.add_done_callback vs. concurrent.futures.as_completed

Question

I’m learning concurrent.futures.ThreadPoolExecutor in Py3.6 and a bit confused as to what’s the difference, pros-and-cons between using

1 future.add_done_callback(callback)

2 concurrent.futures.as_completed(futures)

When would you choose one over the other? If I understand correctly the purpose is same for both more or less.. #1 calls the callback(future) fn as soon as the task has finished and corresponding future has settled, and #2 returns the futures object in the order which the tasks finish and futures settle..

In both cases we we can retrieve the returned value using future.results() (or raise future.exception() if exception was raised).

Thanks for any clarification around that.

Answer 1

The definitions of the functions are at https://github.com/python/cpython/blob/f85af035c5cb9a981f5e3164425f27cf73231b5f/Lib/concurrent/futures/_base.py#L200

def as_completed(fs, timeout=None):
    """An iterator over the given futures that yields each as it completes.

add_done_callback is a method within futures class and a lower level function than as_completed. Essentially, as_completed uses add_done_callback internally. as_completed also has timeout argument for callback. In general, we may use as_completed if working with multiple futures, while add_done_callback is used with single future. Overall, both add_done_callback and as_completed may achieve the similar objectives for simpler programs.

Just a thought. We can use different callback function for each future in the list of futures with add_done_callback, while as_completed may work only accept a single single callback.