List comprehensions: different behaviour with respect to scope in debug mode and in normal runtime

Question

Consider the following:

def f():
    a = 2
    b = [a + i for i in range(3)]
f()

This runs without problems. As I understand it (please correct me if I'm wrong, though), the list comprehension expression introduces a new scope, but since it is created within a function (as opposed to, say, a class), it has access to the surrounding scope, including the variable a.

In contrast, if I were to enter debug mode, stop at line 3 above, and then just manually write the following in the interpreter

>>> b = [a + i for i in range(3)]

I get an error:

Traceback (most recent call last):
  File "<string>", line 293, in runcode
  File "<interactive input>", line 1, in <module>
  File "<interactive input>", line 1, in <listcomp>
NameError: global name 'a' is not defined

Why is this? When I'm stopped at a given line in debug mode, isn't the scope that I have access to the same as what it would be at runtime?

(I'm using PyScripter, by the way)

Answer 1

No, you don't quite get the same scope.

Python determines at compile time what variables are looked up in what scope. List comprehensions get their own scope, so names used in the list comprehension are either local to the list comprehension, closures (nonlocal), or globals.

In the function scope, a is a closure to the list comprehension; the compiler knows a is located in the parent scope of f. But if you enter the same expression in an interactive prompt, there is no nested scope because there is no surrounding function being compiled at the same time. As a result, a is assumed by the compiler to be a global instead:

>>> import dis
>>> dis.dis(compile("b = [a + i for i in range(3)]", '<stdin>', 'single').co_consts[0])
  1           0 BUILD_LIST               0
              3 LOAD_FAST                0 (.0)
        >>    6 FOR_ITER                16 (to 25)
              9 STORE_FAST               1 (i)
             12 LOAD_GLOBAL              0 (a)
             15 LOAD_FAST                1 (i)
             18 BINARY_ADD
             19 LIST_APPEND              2
             22 JUMP_ABSOLUTE            6
        >>   25 RETURN_VALUE

A LOAD_GLOBAL bytecode is used for a here (the .0 is the range(3) iterable for the comprehension).

In a function scope however:

>>> def f():
...     a = 2
...     b = [a + i for i in range(3)]
... 
>>> dis.dis(f.__code__.co_consts[2])
  3           0 BUILD_LIST               0
              3 LOAD_FAST                0 (.0)
        >>    6 FOR_ITER                16 (to 25)
              9 STORE_FAST               1 (i)
             12 LOAD_DEREF               0 (a)
             15 LOAD_FAST                1 (i)
             18 BINARY_ADD
             19 LIST_APPEND              2
             22 JUMP_ABSOLUTE            6
        >>   25 RETURN_VALUE
>>> f.__code__.co_cellvars
('a',)

a is loaded with LOAD_DEREF, loading the first closure (the cell variable named 'a').

When testing a list comprehension like that in an interactive prompt, you'll have to provide your own nested scope; wrap the expression in a function:

>>> def f(a):
...     return [a + i for i in range(3)]
...
>>> f(a)
[2, 3, 4]