How can I tell where my python script is hanging?

Question

So I'm debugging my python program and have encountered a bug that makes the program hang, as if in an infinite loop. Now, I had a problem with an infinite loop before, but when it hung up I could kill the program and python spat out a helpful exception that told me where the program terminated when I sent it the kill command. Now, however, when the program hangs up and I ctrl-c it, it does not abort but continues running. Is there any tool I can use to locate the hang up? I'm new to profiling but from what I know a profiler can only provide you with information about a program that has successfully completed. Or can you use a profiler to debug such hang ups?

Answer 1

Let's assume that you are running your program as:

python YOURSCRIPT.py 

Try running your program as:

python -m trace --trace YOURSCRIPT.py 

And have some patience while lots of stuff is printed on the screen. If you have an infinite loop, it will go on for-ever (halting problem). If it gets stuck somewhere, then mostly you are stuck on I/O or it is a deadlock.

Answer 2

I wrote a module that prints out threads that hang longer that 10 seconds at one place. hanging_threads.py

Run:

python -m pip install hanging_threads 

Add this to your code:

from hanging_threads import start_monitoring start_monitoring(seconds_frozen=10, test_interval=100) 

Here is an example output:

--------------------    Thread 5588     --------------------   File "C:\python33\lib\threading.py", line 844, in _exitfunc         t.join()   File "C:\python33\lib\threading.py", line 743, in join         self._block.wait()   File "C:\python33\lib\threading.py", line 184, in wait         waiter.acquire() 

This occurs at the exit of the main thread when you forget to set another thread as daemon.