Detect when threads are running in a python application?

Question

How can we detect when threads are running in a python application?

Motivation: We were recently debugging a large Python application that had several customer supplied modules that were supplied without source code. Unbeknowst to us (and our customer!), one of these modules would launch threads under very specific (and undocumented) conditions. These threads caused unexpected side effects which changed our environment in unexpected ways. The entire debugging scenario would be funny if it wasn't my team stuck with the task of figuring out where the side effects were coming from :)

We finally discovered what was happening when one of the threads raised an un-trapped error. Had this not happened, we would still be pulling our hair out.

So, this is why we're looking for a way to detect the presence of threads in our application.

Here are some possible techniques we came up with and the corresponding challenges:

1. Grep source code for thread/threading. Challenge: We don't always have access to source code. And even if we can find instances of thread creation in the source code, we still want a real-time technique for detecting when threads are currently active.

2. Look for thread or threading in sys.modules. Challenge: Only insures these modules were loaded - doesn't prove that threads are currently running.

3. Monkey-patch the Python standard thread/threading library to leave us clues as to when threads have been created. Challenge: Only tells us whether threads have been created - does not provide any information about whether threads are currently running.

Thanks! Malcolm

Answer 1

Debugger

Usually a debugger can show you all the threads and what each one is executing. If you tell us which debugger you are using, someone can tell you how to see the thread info. If you aren't using a debugger, I highly suggest you start. Debugging multi-threaded programs without a real debugger is quite error-prone and inefficient. Personally, I use WingIDE.

Runtime

If you're just poking around in the interpreter or trying to detect this situation in your code, perhaps sys._current_frames() might help. It returns the stack frame for each thread.