Check what a running process is doing: print stack trace of an uninstrumented Python program

Question

Is there a way on Linux to check what a running Python daemon process is doing? That is, without instrumenting the code and without terminating it? Preferably I'd like to get the name of the module and the line number in it that is currently running.

Conventional debugging tools such as strace, pstack and gdb are not very useful for Python code. Most stack frames just contain functions from the interpreter code like PyEval_EvalFrameEx and and PyEval_EvalCodeEx, it doesn't give you any hint on were in the .py-file the execution is.

Answer 1

Some of the answers in Showing the stack trace from a running Python application are applicable in this situation:

• pyrasite (this was the one that worked for me):

  $ sudo pip install pyrasite $ echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope $ sudo pyrasite 16262 dump_stacks.py # dumps stacks to stdout/stderr of the python program 

• pyringe

• pydbattach - couldn't get this to work, but the repository https://github.com/albertz/pydbattach contains pointer to other tools
• pstack reportedly prints the python stack on Solaris

Answer 2

winpdb allows you to attach to a running python process, but to do this, you must start the python process this way:

 rpdb2 -d -r script.py 

Then, after setting a password:

A password should be set to secure debugger client-server communication. Please type a password:mypassword 

you could launch winpdb to File>Attach to (or File>Detach from) the process.

Answer 3

py-spy (https://github.com/benfred/py-spy) has a few useful tools for inspecting running Python processes. In particular, py-spy dump will print a stack trace (including function, file, and line) for every thread.

Answer 4

on POSIX systems like Linux, you can use good old GDB, see

• https://t37.net/debug-a-running-python-process-without-printf.html and
• https://wiki.python.org/moin/DebuggingWithGdb

There's also the excellent PyCharm IDE (free community version available) that can attach to a running Python process right from within the IDE, using Pdb 4 under the hood, see this blog entry:

• http://blog.jetbrains.com/pycharm/2015/02/feature-spotlight-python-debugger-and-attach-to-process/