Variable scope and name resolution in Python

Question

I think that I fundamentally don't understand how Python does things like variable scope and name resolution. In particular, the fact that the function broken() below doesn't work really surprises me. And, though I've fished around the web for awhile looking for a helpful explanation, but I still don't get it. Can anyone explain or link to a good description of how this stuff works in Python, with enough details that it will seem obvious why broken() doesn't work after reading the relevant materials?

# Why does this code work fine
def okay0():
    def foo():
        L = []
        def bar():
            L.append(5)
        bar()
        return L
    foo()

# and so does this
def okay1():
    def foo():
        def bar():
            L.append(5)
        L = []
        bar()
        return L
    foo()

# but the following code raises an exception?
def broken():
    def foo():
        L = []
        bar()
        return L
    def bar():
        L.append(5)
    foo()

# Example
test_list = [okay0, okay1, broken]
for test_function in test_list:
    try:
        test_function()
    except:
        print("broken")
    else:
        print("okay")

Answer 1

A function defined within another function can access its parent's scope.

In your specific case, L is always defined within foo(). On the first two examples, bar() is defined within foo() as well, so it can access L by the rule above (ie, foo() is bar()'s parent).

However, on broken(), bar() and foo() are siblings. They know nothing of each others' scopes, so bar() cannot see L.

From the documentation:

Although scopes are determined statically, they are used dynamically. At any time during execution, there are at least three nested scopes whose namespaces are directly accessible:

• the innermost scope, which is searched first, contains the local names
• the scopes of any enclosing functions, which are searched starting with the nearest enclosing scope, contains non-local, but also non-global names
• the next-to-last scope contains the current module’s global names
• the outermost scope (searched last) is the namespace containing built-in names

Now, why does okay1 work, if L is defined textually after bar()?

Python does not try to resolve the identifiers until it has to actually run the code (dynamic binding, as explained in @Giusti's answer).

When Python gets to execute the function, it sees an identifier L and looks for it on the local namespace. On the cpython implementation, it is an actual dictionary, so it looks on a dictionary for a key named L.

If it does not find it, it checks on the scopes of any enclosing functions, ie the other dictionaries representing the local namespaces of the enclosing functions.

Note that, even if L is defined after bar(), when bar() is called, L has already been defined. So, when bar() is executed, L already exists on the local namespace of foo(), which is searched when Python does not see L within bar().

Supporting piece of the documentation:

A namespace is a mapping from names to objects. Most namespaces are currently implemented as Python dictionaries, but that’s normally not noticeable in any way (except for performance), and it may change in the future.

(...)

The local namespace for a function is created when the function is called, and deleted when the function returns or raises an exception that is not handled within the function. (Actually, forgetting would be a better way to describe what actually happens.) Of course, recursive invocations each have their own local namespace.

A scope is a textual region of a Python program where a namespace is directly accessible. “Directly accessible” here means that an unqualified reference to a name attempts to find the name in the namespace.

Answer 2

It is simpler than it looks.

The first case is probably the most obvious:

 def okay0():
    def foo():
        L = []
        def bar():
            L.append(5)
        bar()
        return L
    foo()

Here all you have are the regular scope rules. L and bar belong to the same scope, and L is declared first. So bar() can access L.

The second sample is also similar:

def okay1():
    def foo():
        def bar():
            L.append(5)
        L = []
        bar()
        return L
    foo()

Here both L and bar() belong to the same scope. They are local to foo(). It may look different because Python uses dynamic binding. That is, the resolution of the name L in foo() is only resolved when the function is called. By that time, Python already knows that L is a local variable to the same function that contains foo(), so the acess is valid.

However, while Python has dynamic binding, it does not have dynamic scope, so this will fail:

def broken():
    def foo():
        L = []
        bar()
        return L
    def bar():
        L.append(5)
    foo()

Here, there are two variables named L. One is local to foo() and another is local to bar(). Since these functions are not nested and Python does not have dynamic scope, they are two different variables. Because bar() does not use L in an assignment, you get an exception.