Python ThreadPoolExecutor Suppress Exceptions

Question

from concurrent.futures import ThreadPoolExecutor, wait, ALL_COMPLETED


def div_zero(x):
    print('In div_zero')
    return x / 0


with ThreadPoolExecutor(max_workers=4) as executor:
    futures = executor.submit(div_zero, 1)

    done, _ = wait([futures], return_when=ALL_COMPLETED)
    # print(done.pop().result())

    print('Done')

The program above will run to completion without any error message.

You can only get the exception if you explicitly call future.result() or future.exception(), like what I did in the line commented-out.

I wonder why this Python module chose this kind of behavior even if it hides problems. Because of this, I spent hours debugging a programming error (referencing a non-exist attribute in a class) that would otherwise be very obvious if the program just crashes with exception, like Java for instance.

Answer 1

I suspect the reason is so that the entire pool does not crash because of one thread raising an exception. This way, the pool will process all the tasks and you can get the threads that raised exceptions separately if you need to.

Answer 2

Each thread is (mostly) isolated from the other threads, including the primary thread. The primary thread does not communicate with the other threads until you ask it to do so.

This includes errors. The result is what you are seeing, the errors occurring other threads do not interfere with the primary thread. You only need to handle them when you ask for the results.