Check if a python thread threw an exception

Question

I have a set of tasks to do in parallel, but at the end of them, I need to know if any of the threads threw an exception. I don't need to handle the exception directly, I just need to know if one of the threads failed with an exception, so I can cleanly terminate the script

Here is a simple example:

#!/usr/bin/python

from time import sleep
from threading import Thread

def func(a):
    for i in range(0,5):
        print a
        sleep(1)

def func_ex():
    sleep(2)
    raise Exception("Blah")


x = [Thread(target=func, args=("T1",)), Thread(target=func, args=("T2",)), Thread(target=func_ex, args=())]

print "Starting"
for t in x:
    t.start()

print "Joining"
for t in x:
    t.join()


print "End"

Before "End", I want to iterate through the threads, see if any failed, and then decide if I can continue with the script, or if I need to exit at this point.

I don't need to intercept the exception or stop the other threads, I just need to know at the end if any failed.

Answer 1

By the time the join() call on a thread returns the thread's stack has been unwound and all information about exceptions has been lost. Thus, unfortunately, you'll need to provide your own mechanism for registering exceptions; some techniques are discussed here.

Answer 2

A simple technique for situations where you do not need to handle the exception is to use a global list and append to it pertinent information. Your code would become something like:

#!/usr/bin/python

from time import sleep
from threading import Thread, current_thread #needed to get thread name or whatever identifying info you need

threadErrors = [] #global list

def func(a):
    for i in range(0,5):
        print a
        sleep(1)

def func_ex():
    global threadErrors #if you intend to change a global variable from within a different scope it has to be declared
    try:
        sleep(2)
        raise Exception("Blah")
    except Exception, e:
        threadErrors.append([repr(e), current_thread.name]) #append a list of info
        raise #re-raise the exception or use sys.exit(1) to let the thread die and free resources 

x = [Thread(target=func, args=("T1",)), Thread(target=func, args=("T2",)), Thread(target=func_ex, args=())]

print "Starting"
for t in x:
    t.start()

print "Joining"
for t in x:
    t.join()

if len(threadErrors) > 0: #check if there are any errors 
    for e in threadErrors:
        print(threadErrors[e][0]+' occurred in thread: '+threadErrors[e][1])
        #do whatever with each error info
else: 
    #there are no errors so do normal clean-up stuff

#do clean-up that should happen in either case here

print "End"

Note: global variables are generally regarded as poor technique, but they are a simple mechanism for communicating between threads. You just have to remember that if one thread is sending info by this route the other thread has to be looking for it.

Answer 3

If you are going to do this in test, I would suggest using pytest-reraise With it you can do something like this:

def test_assert(reraise):

def run():
    with reraise:
        assert False

reraise() # This will not raise anything yet

t = Thread(target=run)
t.start()
t.join()

reraise() # This will raise the assertion error