How to stop multiprocess python-program cleanly after exception?

Question

I have a Python program that has several processes (for now only 2) and threads (2 per process). I would like to catch every exception and especially shut down my program cleanly on Ctrl+c but I can't get it to work. Everytime an Exception occurs the program stops but does not shut down correctly, leaving me with an unusable commandline.

What I have tried so far in pseudocode is:

try:
    for process in processes:
        process.join()
except:
    pass #Just to suppress error-messages, will be removed later
finally:
    for process in processes:
        process.terminate()

But as I already said with no luck. Also note, that I get the Exception error message for both Processes, so they are both halted I believe?

Maybe I should also mention that most of the threads are blocked in listening on a pipe.

EDIT So I nearly got it working. I needed to try: every thread and make sure the threads are joined correctly. There is just one flaw: Exception KeyboardInterrupt in <module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored when shutting down. This is raised in the main-thread of the main-process. This thread is already finished, meaning it has passed the last line of code.

Answer 1

The problem (I expect) is that the exceptions are raised inside the processes not in the join calls.

I suggest you try wrapping each process's main method in try-except loop. Then have a flag (e.g. an instance of multiprocessing.Value) that the except statement sets to False. Each process could check the value of the flag and stop (cleaning up after itself) if it's set to False.

Note, if you just terminate a process it won't clean up after itself, as this is the same as sending a SIG_TERM to it.