Python class scoping rules

Question

EDIT: Looks like this is a very old "bug" or, actually, feature. See, e.g., this mail

I am trying to understand the Python scoping rules. More precisely, I thought that I understand them but then I found this code here:

x = "xtop"
y = "ytop"
def func():
    x = "xlocal"
    y = "ylocal"
    class C:
        print(x)
        print(y)
        y = 1
func()

In Python 3.4 the output is:

xlocal
ytop

If I replace the inner class by a function then it reasonably gives UnboundLocalError. Could you explain me why it behaves this strange way with classes and what is the reason for such choice of scoping rules?

Answer 1

TL;DR: This behaviour has existed since Python 2.1 PEP 227: Nested Scopes, and was known back then. If a name is assigned to within a class body (like y), then it is assumed to be a local/global variable; if it is not assigned to (x), then it also can potentially point to a closure cell. The lexical variables do not show up as local/global names to the class body.

---

On Python 3.4, dis.dis(func) shows the following:

>>> dis.dis(func)   4           0 LOAD_CONST               1 ('xlocal')               3 STORE_DEREF              0 (x)    5           6 LOAD_CONST               2 ('ylocal')               9 STORE_FAST               0 (y)    6          12 LOAD_BUILD_CLASS              13 LOAD_CLOSURE             0 (x)              16 BUILD_TUPLE              1              19 LOAD_CONST               3 (<code object C at 0x7f083c9bbf60, file "test.py", line 6>)              22 LOAD_CONST               4 ('C')              25 MAKE_CLOSURE             0              28 LOAD_CONST               4 ('C')              31 CALL_FUNCTION            2 (2 positional, 0 keyword pair)              34 STORE_FAST               1 (C)              37 LOAD_CONST               0 (None)              40 RETURN_VALUE 

The LOAD_BUILD_CLASS loads the builtins.__build_class__ on the stack; this is called with arguments __build_class__(func, name); where func is the class body, and name is 'C'. The class body is the constant #3 for the function func:

>>> dis.dis(func.__code__.co_consts[3])   6           0 LOAD_NAME                0 (__name__)               3 STORE_NAME               1 (__module__)               6 LOAD_CONST               0 ('func.<locals>.C')               9 STORE_NAME               2 (__qualname__)    7          12 LOAD_NAME                3 (print)              15 LOAD_CLASSDEREF          0 (x)              18 CALL_FUNCTION            1 (1 positional, 0 keyword pair)              21 POP_TOP    8          22 LOAD_NAME                3 (print)              25 LOAD_NAME                4 (y)              28 CALL_FUNCTION            1 (1 positional, 0 keyword pair)              31 POP_TOP    9          32 LOAD_CONST               1 (1)              35 STORE_NAME               4 (y)              38 LOAD_CONST               2 (None)              41 RETURN_VALUE 

Within the class body, x is accessed with LOAD_CLASSDEREF (15) while y is load with LOAD_NAME (25). The LOAD_CLASSDEREF is a Python 3.4+ opcode for loading values from closure cells specifically within class bodies (in previous versions, the generic LOAD_DEREF was used); the LOAD_NAME is for loading values from locals and then globals. However closure cells show up neither as locals nor globals.

Now, because the name y is stored to within the class body (35), it is consistently being used as not a closure cell but a local/global name. The closure cells do not show up as local variables to the class body.

This behaviour has been true ever since implementing PEP 227 - nested scopes. And back then BDFL stated that this should not be fixed - and thus it has been for these 13+ years.

---

The only change since PEP 227 is the addition of nonlocal in Python 3; if one uses it within the class body, the class body can set the values of the cells within the containing scope:

x = "xtop" y = "ytop" def func():     x = "xlocal"     y = "ylocal"     class C:         nonlocal y  # y here now refers to the outer variable         print(x)         print(y)         y = 1      print(y)     print(C.y)  func() 

The output now is

xlocal ylocal 1 Traceback (most recent call last):   File "test.py", line 15, in <module>     func()   File "test.py", line 13, in func     print(C.y) AttributeError: type object 'C' has no attribute 'y' 

That is, print(y) read the value of the cell y of the containing scope, and y = 1 set the value in that cell; in this case, no attribute was created for the class C.

Answer 2

First focus on the case of a closure -- a function within a function:

x = "xtop" y = "ytop" def func():     x = "xlocal"     y = "ylocal"     def inner():  #       global y         print(x)         print(y)         y='inner y'         print(y)     inner()   

Note the commented out global in inner If you run this, it replicates the UnboundLocalError you got. Why?

Run dis.dis on it:

>>> import dis >>> dis.dis(func)   6           0 LOAD_CONST               1 ('xlocal')               3 STORE_DEREF              0 (x)    7           6 LOAD_CONST               2 ('ylocal')               9 STORE_FAST               0 (y)    8          12 LOAD_CLOSURE             0 (x)              15 BUILD_TUPLE              1              18 LOAD_CONST               3 (<code object inner at 0x101500270, file "Untitled 3.py", line 8>)              21 LOAD_CONST               4 ('func.<locals>.inner')              24 MAKE_CLOSURE             0              27 STORE_FAST               1 (inner)   14          30 LOAD_FAST                1 (inner)              33 CALL_FUNCTION            0 (0 positional, 0 keyword pair)              36 POP_TOP              37 LOAD_CONST               0 (None)              40 RETURN_VALUE 

Note the different access mode of x vs y inside of func. The use of y='inner y' inside of inner has created the UnboundLocalError

Now uncomment global y inside of inner. Now you have unambiguously create y to be the top global version until resigned as y='inner y'

With global uncommented, prints:

xlocal ytop inner y 

You can get a more sensible result with:

x = "xtop" y = "ytop" def func():     global y, x     print(x,y)     x = "xlocal"     y = "ylocal"     def inner():         global y         print(x,y)         y = 'inner y'         print(x,y)     inner()     

Prints:

xtop ytop xlocal ylocal xlocal inner y 

---

The analysis of the closure class is complicated by instance vs class variables and what / when a naked class (with no instance) is being executed.

The bottom line is the same: If you reference a name outside the local namespace and then assign to the same name locally you get a surprising result.

The 'fix' is the same: use the global keyword:

x = "xtop" y = "ytop" def func():     global x, y     x = "xlocal"     y = "ylocal"     class Inner:         print(x, y)         y = 'Inner_y'         print(x, y)  

Prints:

xlocal ylocal xlocal Inner_y 

You can read more about Python 3 scope rules in PEP 3104