breakpoint in except clause doesn't have access to the bound exception

Question

Consider the following example:

try:
    raise ValueError('test')
except ValueError as err:
    breakpoint()  # at this point in the debugger, name 'err' is not defined

Here, after the breakpoint is entered, the debugger doesn't have access to the exception instance bound to err:

$ python test.py 
--Return--
> test.py(4)<module>()->None
-> breakpoint()
(Pdb) p err
*** NameError: name 'err' is not defined

Why is this the case? How can I access the exception instance? Currently I'm using the following workaround but it feels awkward:

try:
    raise ValueError('test')
except ValueError as err:
    def _tmp():
        breakpoint()
    _tmp()
    # (lambda: breakpoint())()  # or this one alternatively

Interestingly, using this version, I can also access the bound exception err when moving one frame up in the debugger:

$ python test.py 
--Return--
> test.py(5)_tmp()->None
-> breakpoint()
(Pdb) up
> test.py(6)<module>()
-> _tmp()
(Pdb) p err
ValueError('test')

Disassembly via dis

In the following I compared two versions, one using breakpoint directly and the other wrapping it in a custom function _breakpoint:

def _breakpoint():
    breakpoint()

try:
    raise ValueError('test')
except ValueError as err:
    breakpoint()   # version (a), cannot refer to 'err'
    # _breakpoint()  # version (b), can refer to 'err'

The output of dis is similar except for some memory locations and the name of the function of course:

So it must be the additional stack frame that allows pdb to refer to the bound exception instance. However it is not clear why this is the case, since within the except block anything can refer to the bound exception instance.

Answer 1

breakpoint() is not a breakpoint in the sense that it halts execution at the exact location of this function call. Instead it's a shorthand for import pdb; pdb.set_trace() which will halt execution at the next line of code (it calls sys.settrace under the covers). Since there is no more code inside the except block, execution will halt after that block has been exited and hence the name err is already deleted. This can be seen more clearly by putting an additional line of code after the except block:

try:
    raise ValueError('test')
except ValueError as err:
    breakpoint()
print()

which gives the following:

$ python test.py 
> test.py(5)<module>()
-> print()

This means the interpreter is about to execute the print() statement in line 5 and it has already executed everything prior to it (including deletion of the name err).

When using another function to wrap the breakpoint() then the interpreter will halt execution at the return event of that function and hence the except block is not yet exited (and err is still available):

$ python test.py 
--Return--
> test.py(5)<lambda>()->None
-> (lambda: breakpoint())()

Exiting of the except block can also be delayed by putting an additional pass statement after the breakpoint():

try:
    raise ValueError('test')
except ValueError as err:
    breakpoint()
    pass

which results in:

$ python test.py 
> test.py(5)<module>()
-> pass
(Pdb) p err
ValueError('test')

Note that the pass has to be put on a separate line, otherwise it will be skipped:

$ python test.py 
--Return--
> test.py(4)<module>()->None
-> breakpoint(); pass
(Pdb) p err
*** NameError: name 'err' is not defined

Note the --Return-- which means the interpreter has already reached the end of the module.

Answer 2

This is an excellent question!

When something strange is going on, I always dis-assemble the Python code and have a look a the byte code.

This can be done with the dis module from the standard library.

Here, there is the problem, that I cannot dis-assemble the code when there is a breakpoint in it :-)

So, I modified the code a bit, and set a marker variable abc = 10 to make visible what happens after the except statement.

Here is my modified code, which I saved as main.py.

try:
    raise ValueError('test')
except ValueError as err:
    abc = 10

When you then dis-assemble the code...

❯ python -m dis main.py 
  1           0 SETUP_FINALLY           12 (to 14)

  2           2 LOAD_NAME                0 (ValueError)
              4 LOAD_CONST               0 ('test')
              6 CALL_FUNCTION            1
              8 RAISE_VARARGS            1
             10 POP_BLOCK
             12 JUMP_FORWARD            38 (to 52)

  3     >>   14 DUP_TOP
             16 LOAD_NAME                0 (ValueError)
             18 COMPARE_OP              10 (exception match)
             20 POP_JUMP_IF_FALSE       50
             22 POP_TOP
             24 STORE_NAME               1 (err)
             26 POP_TOP
             28 SETUP_FINALLY            8 (to 38)

  4          30 LOAD_CONST               1 (10)
             32 STORE_NAME               2 (abc)
             34 POP_BLOCK
             36 BEGIN_FINALLY
        >>   38 LOAD_CONST               2 (None)
             40 STORE_NAME               1 (err)
             42 DELETE_NAME              1 (err)
             44 END_FINALLY
             46 POP_EXCEPT
             48 JUMP_FORWARD             2 (to 52)
        >>   50 END_FINALLY
        >>   52 LOAD_CONST               2 (None)
             54 RETURN_VALUE

You get a feeling what is going on.

You can read more about the dis module both in the excellent documentation or on the Python module of the week site:

https://docs.python.org/3/library/dis.html https://docs.python.org/3/library/dis.html

Certainly, this is not a perfect answer. Actually, I have to sit down and read documentation myself. I am surprised that SETUP_FINALLY was called before the variable abc in the except block was handled. Also, I am not sure what's the effect of POP_TOP - immediately executed after storing the err name.

P.S.: Excellent question! I am super excited how this turns out.