Stopping threads spawned by BaseHTTPServer using ThreadingMixin

Question

I have read on here on this post that using ThreadingMixin (from the SocketServer module), you are able to create a threaded server with BaseHTTPServer. I have tried it, and it does work. However, how can I stop active threads spawned by the server (for example, during a server shutdown)? Is this possible?

Answer 1

The simplest solution is to just use daemon_threads. The short version is: just set this to True, and don't worry about it; when you quit, any threads still working will stop automatically.

As the ThreadingMixIn docs say:

When inheriting from ThreadingMixIn for threaded connection behavior, you should explicitly declare how you want your threads to behave on an abrupt shutdown. The ThreadingMixIn class defines an attribute daemon_threads, which indicates whether or not the server should wait for thread termination. You should set the flag explicitly if you would like threads to behave autonomously; the default is False, meaning that Python will not exit until all threads created by ThreadingMixIn have exited.

Further details are available in the threading docs:

A thread can be flagged as a “daemon thread”. The significance of this flag is that the entire Python program exits when only daemon threads are left. The initial value is inherited from the creating thread. The flag can be set through the daemon property.

Sometimes this isn't appropriate, because you want to shut down without quitting, or because your handlers may have cleanup that needs to be done. But when it is appropriate, you can't get any simpler.

If all you need is a way to shutdown without quitting, and don't need guaranteed cleanup, you may be able to use platform-specific thread-cancellation APIs via ctypes or win32api. This is generally a bad idea, but occasionally it's what you want.

If you need clean shutdown, you need to build your own machinery for that, where the threads cooperate. For example, you could create a global "quit flag" variable protected by a threading.Condition, and have your handle function check this periodically.

This is great if the threads are just doing slow, non-blocking work that you can break up into smaller pieces. For example, if the handle function always checks the quit flag at least once every 5 seconds, you can guarantee being able to shutdown the threads within 5 seconds. But what if the threads are doing blocking work—as they probably are, because the whole reason you used ThreadingMixIn was to let you make blocking calls instead of writing select loops or using asyncore or the like?

Well, there is no good answer. Obviously if you just need the shutdown to happen "eventually" rather than "within 5 seconds" (or if you're willing to abandon clean shutdown after 5 seconds, and revert to either using platform-specific APIs or daemonizing the threads), you can just put the checks before and after each blocking call, and it will "often" work. But if that's not good enough, there's really nothing you can do.

If you need this, the best answer is to change your architecture to use a framework that has ways to do this. The most popular choices are Twisted, Tornado, and gevent. In the future, PEP 3156 will bring similar functionality into the standard library, and there's a partly-complete reference implementation tulip that's worth playing with if you're not trying to build something for the real world that has to be ready soon.