Pdb go to a frame in exception within exception

Question

I am debugging a program named a.py using pdb

def f(x) :
    x / x

def g(x) :
    try :
        f(x)
    except Exception as e :
        assert 0

g(0)

When I run the program using python3 -m pdb a.py, the program stops at assert 0 line, and I get the following error information:

Traceback (most recent call last):
  File "/tmp/a.py", line 6, in g
    f(x)
  File "/tmp/a.py", line 2, in f
    x / x
ZeroDivisionError: division by zero

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.6/pdb.py", line 1667, in main
    pdb._runscript(mainpyfile)
  File "/usr/lib64/python3.6/pdb.py", line 1548, in _runscript
    self.run(statement)
  File "/usr/lib64/python3.6/bdb.py", line 434, in run
    exec(cmd, globals, locals)
  File "<string>", line 1, in <module>
  File "/tmp/a.py", line 11, in <module>
    g(0)
  File "/tmp/a.py", line 9, in g
    assert 0
AssertionError

and the stack is (shown using bt command):

(Pdb) bt
  /usr/lib64/python3.6/pdb.py(1667)main()
-> pdb._runscript(mainpyfile)
  /usr/lib64/python3.6/pdb.py(1548)_runscript()
-> self.run(statement)
  /usr/lib64/python3.6/bdb.py(434)run()
-> exec(cmd, globals, locals)
  <string>(1)<module>()->None
  /tmp/a.py(11)<module>()->None
-> g(0)
> /tmp/a.py(9)g()
-> assert 0
(Pdb) 

The problem is, I cannot go to function f to debug x / x simply using up and down, because my stack ends at the g function.

How should I debug such exceptions within exceptions? What about exceptions within exceptions within exceptions ...?

Answer 1

tl;dr: You can debug the inner exception even after you've already entered post-mortem debugging for the outer exception. Here's how to do it:

1. Enter interactive mode from pdb (type interact into the pdb prompt).
2. Run:

   import pdb, sys; pdb.post_mortem(sys.last_value.__context__.__traceback__)
   

   Note:
   • Replace __context__ with __cause__ if your exception is explicitly chained; also append more __context__s or __cause__s if it's nested deeper.
   • If you're inspecting a handled exception (one that was caught in a try-catch), replace sys.last_value with sys.exc_info()[1]. If you're unsure, inspect the exception values before proceeding. (Thanks @The Doctor for pointing this out in comments)
3. This starts a new pdb session that allows you to debug the inner exception.

The following is a long and detailed explanation of why this work. Before diving into the solution, I'll first explain some related concepts:

---

Chained Exceptions

The "exception within exception" here is known as chained exceptions. Exceptions can be chained explicitly or implicitly:

>>>: try:
...:     raise ZeroDivisionError
...: except Exception as inner_exc:
...:     raise ValueError  # implicit chaining
...:
---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-6-1ae22e81c853> in <module>
      1 try:
----> 2     raise ZeroDivisionError
      3 except Exception as inner_exc:

ZeroDivisionError:

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-6-1ae22e81c853> in <module>
      2     raise ZeroDivisionError
      3 except Exception as inner_exc:
----> 4     raise ValueError  # implicit chaining

ValueError:


>>>: try:
...:     raise ZeroDivisionError
...: except Exception as inner_exc:
...:     raise ValueError from inner_exc  # explicit chaining
...:
---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-5-63c49fcb10a2> in <module>
      1 try:
----> 2     raise ZeroDivisionError
      3 except Exception as inner_exc:

ZeroDivisionError:

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
<ipython-input-5-63c49fcb10a2> in <module>
      2     raise ZeroDivisionError
      3 except Exception as inner_exc:
----> 4     raise ValueError from inner_exc  # explicit chaining

ValueError:

If we catch the outer exception as outer_exc, then we can inspect the inner exception through outer_exc.__cause__ (if explicitly chained) or outer_exc.__context__ (if implicitly chained).

---

Post-mortem Debugging

Running a script with python -m pdb allows the Python debugger to enter post-mortem debugging mode on exception. "Post-mortem" here means "after the exception has occurred". You can do the same from an IPython console or within Jupyter notebooks by running the %debug magic.

You can also manually enter post-mortem debugging mode if you have access to the traceback object. Luckily, traceback objects are stored on the exception object itself as the __traceback__ attribute:

>>> try:
...     raise ZeroDivisionError:
... except Exception as e:
...     # Variable `e` is local to this block, so we store it in another variable
...     # to extend its lifetime.
...     exc = e

>>> import pdb
>>> pdb.post_mortem(exc.__traceback__)
> <ipython-input-8-e5b5ed89e466>(2)<module>()
-> raise ZeroDivisionError
(Pdb)

---

Debugging Chained Exceptions

Now we can try debugging the chained exceptions! Assume we're already in post-mortem debugging mode for the outer exception. What we need to do is:

1. Get the outer exception object;
2. Access the inner exception object, and get its traceback;
3. Call pdb.post_mortem() on that traceback object. Here's what we do:

# First, enter interactive mode to execute commands.
(Pdb) interact
*interactive*
# The current exception is stored in `sys.exc_info()`.  This gives back a tuple
# of (exception type, exception value, traceback).
>>> import sys
>>> sys.exc_info()
(<class 'AssertionError'>, AssertionError(), <traceback object at 0x10c683e00>)
>>> sys.exc_info()[1]
AssertionError()
# In our case, the inner exception is implicitly chained.  Access it through
# the `__context__` attribute.
>>> sys.exc_info()[1].__context__
ZeroDivisionError('division by zero')
# Get its traceback, and enter post-mortem debugging.
>>> sys.exc_info()[1].__context__.__traceback__
<traceback object at 0x10c683c80>
>>> import pdb
>>> pdb.post_mortem(sys.exc_info()[1].__context__.__traceback__)
> /Volumes/hrt/trunk-3/a.py(2)f()
-> x / x
(Pdb)

There you have it! You can now debug the inner exception using normal pdb commands, such as walking through the stack or inspecting local variables.

Answer 2

Actually re-raising the exception (option 3 in wholevinski's answer) solves my problem, since it does not require me to modify function f. Here is the code:

'''Option 3'''
def f(x) :
    x -= 1
    x / x

def g(x) :
    try :
        for i in range(x, 0, -1) :
            print(f(i))
    except Exception as e :
        raise e

g(10)

Pdb output of the stack:

(Pdb) bt
  /usr/lib64/python3.6/pdb.py(1667)main()
-> pdb._runscript(mainpyfile)
  /usr/lib64/python3.6/pdb.py(1548)_runscript()
-> self.run(statement)
  /usr/lib64/python3.6/bdb.py(434)run()
-> exec(cmd, globals, locals)
  <string>(1)<module>()->None
  /tmp/a.py(13)<module>()->None
-> g(10)
  /tmp/a.py(11)g()
-> raise e
  /tmp/a.py(9)g()
-> print(f(i))
> /tmp/a.py(4)f()
-> x / x
(Pdb)