Proper method for terminating a python program regardless of call location, with cleanup

Question

How does one properly and cleanly terminate a python program if needed? sys.exit() does not reliably perform this function as it merely terminates the thread it is called from, exit() and quit() are not supposed to be used except in terminal windows, raise SystemExit has the same issues as sys.exit() and is bad practice, and os._exit() immediately kills everything and does not clean up, which can cause issues with residuals.

Is there a way to ALWAYS kill the program and all threads, regardless of where it is called from, while still cleaning up?

Answer 1

Is there a way to ALWAYS kill the program and all threads, regardless of where it is called from, while still cleaning up?

No - "regardless where it is called from" and "cleaning up" do not mix.

---

It is simply not meaningful to both reliably and safely kill a thread. Killing a thread (or process) means interrupting what it is doing - that includes clean up. Not interrupting any clean up means, well, not actually killing a thread. You cannot have both at the same time.

If you want to kill all threads, then os._exit() is precisely what you are asking for. If you want to clean up a thread, no generic feature can fulfil that.

The only reliable way to shut down threads is to implement your own, safe interrupt. To some extent, this must be customised to your use case - after all, you are the only one knowing when it is safe to shut down.

Killing a thread with Exceptions

The underlying CPython API allows you to raise an exception in another thread. See for example this answer.

This is not portable and not safe. You could be killing that thread at any arbitrary point. If your code expects an exception or your resources clean up after themselves (via __del__), you can limit harm, but not exclude it. Still, it is very close to what most people think of as a "clean kill".

Self-Cleaning daemon threads with atexit

Threads running with Thread.daemon are abruptly terminated if no other threads remain. Usually, this is half of what you want: gracefully terminate if all proper threads exit.

Now, the key is that a daemon thread does not prevent shutdown. This also means it does not prevent atexit from running! Thus, a daemon can use atexit to automatically shutdown itself and cleanup on termination.

import threading
import atexit
import time


class CleanThread(threading.Thread):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.daemon = True
        # use a signal to inform running code about shutdown
        self.shutdown = threading.Event()

    def atexit_abort(self):
        # signal to the thread to shutdown with the interpreter
        self.shutdown.set()
        # Thread.join in atexit causes interpreter shutdown
        # to be delayed until cleanup is done
        self.join()  # or just release resources and exit

    def run(self):
        atexit.register(self.atexit_abort)
        while not self.shutdown.wait(0.1):
            print('who wants to live forever?')
        print('not me!')
        atexit.unregister(self.atexit_abort)


thread = CleanThread()
thread.start()
time.sleep(0.3)
# program exits here

Note that this still requires your code to listen for a cleanup signal! Depending on what the Thread does, there are other mechanisms to achieve this. For example, the concurrent.future module empties the task queue of all worker threads on shutdown.