Debug python dump file in windbg

Question

one of my python.exe (v3.5.2) process hang in remote Windows Server. I cannot attach it from remote site so I create a full memory dump and download it back to analyze.

however, windbg doesn't have the tool like py-bt in gdb. what I can do is downloading cpython symbol files and loading into windbg.

the question is: how can I print the python script call stack just like "py-bt" do?

the call stack is as follow, looks like hang on HTTPS connection

0:000> k
ChildEBP RetAddr  
0050dd24 74592242 ntdll!ZwWaitForSingleObject+0xc
0050dd6c 7459bdee mswsock!SockWaitForSingleObject+0x143
0050ddf8 777561e0 mswsock!WSPRecv+0x391
0050de48 614d8e76 ws2_32!recv+0x103
0050de68 614138bb _ssl!sock_read+0x26 [c:\build\cpython\externals\openssl-1.0.2h\crypto\bio\bss_sock.c @ 141]
0050de90 614bc229 _ssl!BIO_read+0x5b [c:\build\cpython\externals\openssl-1.0.2h\crypto\bio\bio_lib.c @ 210]
0050debc 614bc36e _ssl!ssl3_read_n+0x1c9 [c:\build\cpython\externals\openssl-1.0.2h\ssl\s3_pkt.c @ 255]
0050df78 614bd136 _ssl!ssl3_get_record+0x9e [c:\build\cpython\externals\openssl-1.0.2h\ssl\s3_pkt.c @ 339]
0050dfb8 614c0bb5 _ssl!ssl3_read_bytes+0x1a6 [c:\build\cpython\externals\openssl-1.0.2h\ssl\s3_pkt.c @ 1228]
0050dfdc 614c0c13 _ssl!ssl3_read_internal+0x45 [c:\build\cpython\externals\openssl-1.0.2h\ssl\s3_lib.c @ 4459]
0050dfec 61402880 _ssl!ssl3_read+0x13 [c:\build\cpython\externals\openssl-1.0.2h\ssl\s3_lib.c @ 4483]
0050e034 613ff8f7 _ssl!_ssl__SSLSocket_read_impl+0x1f0 [c:\build\cpython\modules\_ssl.c @ 1944]
0050e080 61b12ae3 _ssl!_ssl__SSLSocket_read+0x67 [c:\build\cpython\modules\clinic\_ssl.c.h @ 267]
0050e09c 61b7409f python35!PyCFunction_Call+0x113 [c:\build\cpython\objects\methodobject.c @ 109]
0050e0d0 61b712a5 python35!call_function+0x2ff [c:\build\cpython\python\ceval.c @ 4705]
0050e148 61b7301f python35!PyEval_EvalFrameEx+0x20a5 [c:\build\cpython\python\ceval.c @ 3239]
0050e194 61b74259 python35!_PyEval_EvalCodeWithName+0x75f [c:\build\cpython\python\ceval.c @ 4018]
0050e1dc 61b740f3 python35!fast_function+0x109 [c:\build\cpython\python\ceval.c @ 4813]
0050e210 61b712a5 python35!call_function+0x353 [c:\build\cpython\python\ceval.c @ 4730]
0050e288 61b7301f python35!PyEval_EvalFrameEx+0x20a5 [c:\build\cpython\python\ceval.c @ 3239]

Answer 1

I was surprised that nothing like CPython's gdbinit existed for WinDbg so I decided to write a debugger extension. I've uploaded the code to the PyExt repository on GitHub.

---

To install it:

• Grab the latest zip from the releases page
• Unzip pyext.dll into WinDbg's winext directory. (Make sure to use the x86 or x64 version that corresponds to the build of WinDbg you're using.)
• Open a dump file in WinDbg and load the extension with: .load pyext
• Add the Python symbol server to the symbol path with: .sympath+ srv*c:\symbols*http://pythonsymbols.sdcline.com/symbols/
• Use !pystack to see the Python backtrace at the time the dump was captured.

---

Example:

0:000> .load pyext
0:000> .sympath+ srv*c:\symbols*http://pythonsymbols.sdcline.com/symbols/
Symbol search path is: srv*c:\symbols*http://msdl.microsoft.com/download/symbols;srv*c:\symbols*http://pythonsymbols.sdcline.com/symbols/

************* Symbol Path validation summary **************
Response  Time (ms)     Location
Deferred                srv*c:\symbols*http://msdl.microsoft.com/download/symbols
Deferred                srv*c:\symbols*http://pythonsymbols.sdcline.com/symbols/

0:000> ~*e!pystack
Thread 0:
    File "C:\python33\lib\threading.py", line 284, in wait
        [Globals] 
    File "C:\python33\lib\threading.py", line 1028, in join
        [Globals] 
    File "scripts\win32debug.py", line 153, in _launch_and_wait
        [Globals] 
    File "scripts\win32debug.py", line 180, in dump_process
        [Globals] 
    File "object_types.py", line 33, in <module>
        [Locals] [Globals] 

---

If you run into any problems or would find a new feature to be helpful, feel free to open an GitHub issue and let me know.