How to debug a Python segmentation fault?

Question

How can I debug a Python segmentation fault?

We are trying to run our python code on SuSE 12.3. We get reproducible segmentation faults. The python code has been working on other platforms without segmentation faults, for years.

We only code Python, no C extension ....

What is the best way to debug this? I know a bit ansi c, but that was ten years ago ....

Python 2.7.5

Update

The segmentation fault happens on interpreter shutdown.

I can run the script several times:

python -m pdb myscript.py arg1 arg1 continue run continue run 

But the segmentation faults happen, if I leave the pdb with ctrl-d.

Update 2

I now try to debug it with gdb:

gdb  > file python > run myscript.py arg1 arg2 Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffefbe2700 (LWP 15483)] 0x00007ffff7aef93c in PyEval_EvalFrameEx () from /usr/lib64/libpython2.7.so.1.0 (gdb) bt #0  0x00007ffff7aef93c in PyEval_EvalFrameEx () from /usr/lib64/libpython2.7.so.1.0 #1  0x00007ffff7af5303 in PyEval_EvalCodeEx () from /usr/lib64/libpython2.7.so.1.0 #2  0x00007ffff7adc858 in ?? () from /usr/lib64/libpython2.7.so.1.0 #3  0x00007ffff7ad840d in PyObject_Call () from /usr/lib64/libpython2.7.so.1.0 #4  0x00007ffff7af1082 in PyEval_EvalFrameEx () from /usr/lib64/libpython2.7.so.1.0 #5  0x00007ffff7af233d in PyEval_EvalFrameEx () from /usr/lib64/libpython2.7.so.1.0 #6  0x00007ffff7af233d in PyEval_EvalFrameEx () from /usr/lib64/libpython2.7.so.1.0 #7  0x00007ffff7af5303 in PyEval_EvalCodeEx () from /usr/lib64/libpython2.7.so.1.0 #8  0x00007ffff7adc5b6 in ?? () from /usr/lib64/libpython2.7.so.1.0 #9  0x00007ffff7ad840d in PyObject_Call () from /usr/lib64/libpython2.7.so.1.0 #10 0x00007ffff7ad9171 in ?? () from /usr/lib64/libpython2.7.so.1.0 #11 0x00007ffff7ad840d in PyObject_Call () from /usr/lib64/libpython2.7.so.1.0 #12 0x00007ffff7aeeb62 in PyEval_CallObjectWithKeywords () from /usr/lib64/libpython2.7.so.1.0 #13 0x00007ffff7acc757 in ?? () from /usr/lib64/libpython2.7.so.1.0 #14 0x00007ffff7828e0f in start_thread () from /lib64/libpthread.so.0 #15 0x00007ffff755c7dd in clone () from /lib64/libc.so.6 

Update 3

I installed gdbinit from http://hg.python.org/cpython/file/default/Misc/gdbinit and the debugging symbols from http://download.opensuse.org/debug/distribution/12.3/repo/oss/suse/x86_64/

(gdb) pystack No symbol "_PyUnicode_AsString" in current context. 

What now?

Update 4 We installed the a new RPM (python-2.7.5-3.1.x86_64). We get less segfaults, but they still happen. Here is the link to repository:

http://download.opensuse.org/repositories/devel:/languages:/python:/Factory/openSUSE_12.3/x86_64/

Update 5 Solved my initial problem:

It was http://bugs.python.org/issue1856 (shutdown (exit) can hang or segfault with daemon threads running)

Related: Detect Interpreter shut down in daemon thread

Answer 1

I got to this question because of the Segmentation fault, but not on exit, just in general, and I found that nothing else helped as effectively as faulthandler. It's part of Python 3.3, and you can install in 2.7 using pip.