python noobie scoping question

Question

I wrote this code:

x = 0
def counter():
 x = 1
 def temp(self):
  print x
  x += 1
 return temp

Trying to test if python is lexical or dynamic scope. My thinking was that

y = counter()
y()

Should either print 0 or 1, and that would tell me how python is scoped. However, calling y throws an exception saying that x is undefined. There seems to be something fundamentally flawed in my understanding of how Python works.

Can someone explain how this works? Yes, I am aware this can easily be done using objects. I'm trying to explore the idea of giving functions state without the use of objects. I wrote the code this way because the above translated to a lexically scoped language like Scheme would definitely have worked.

Answer 1

From the docs:

A special quirk of Python is that – if no global statement is in effect – assignments to names always go into the innermost scope. Assignments do not copy data — they just bind names to objects.

So, when Python parses

 def temp(self):
  print x
  x += 1

it sees the assignment x += 1 and therefore decides that x must be in the innermost scope. When later you call temp(...) via y() -- (where, by the way, either the self should be omitted from the definition of temp or else y() should supply one argument) -- Python encounters the print x statement and finds that x has not been defined in the local (innermost) scope. Hence the error,

UnboundLocalError: local variable 'x' referenced before assignment

If you declare

 def temp(self):
     global x

then Python will look for x in the global scope (where x=0). In Python2 there is no way to tell Python to look for x in the extended scope (where x=1). But in Python3 that can be achieved with the declaration

 def temp(self):
     nonlocal x

Answer 2

Python has two scopes (which is really three) plus special rules for nested local scopes. The two scopes are global, for the module-level names, and local, for anything in a function. Anything you assign to in a function is automatically local, unless you declare it otherwise with the global statement in that function. If you use a name that is not assigned to anywhere in the function, it's not a local name; Python will (lexically) search nesting functions to see if they have that name as a local name. If there are no nesting functions or the name isn't local in any of them, the name is assumed to be global.

(The global namespace is special as well, because it's actually both the module global namespace and the builtin namespace, which is tucked away in the builtins or __builtins__ module.)

In your case, you have three x variables: one in the module (global) scope, one in the counter function and one in the temp function -- because += is also an assignment statement. Because the mere fact of assigning to the name makes it local to the function, your += statement will try to use a local variable that hasn't been assigned to yet, and that will raise an UnboundLocalError.

If you intended for all three of these x references to refer to the global variable, you'd need to do global x in both the counter and the temp function. In Python 3.x (but not 2.x), there is a nonlocal declaration quite like global that you can use to make temp assign to the variable in counter, but leave the global x alone.

Answer 3

The Python documentation answers the question in detail: http://docs.python.org/reference/executionmodel.html#naming-and-binding

In summary, Python has static scoping rules. If function f defines or deletes a variable name, then the variable name refers to a variable in the closure of function f. If function f only uses a variable name (no definition or deletion), then the name refers to whatever the name means in f's parent scope. Keep going up to parent scopes until a definition is found or you reach global scope. For example:

def f1(x):
  def g(y):
    z.foo = y # assigns global z
  g(1)

def f2(x):
  def g(y):
    x.foo = y # assigns f2's variable, because f2 defines x 
  g(1)

def f3(x):
  def g(y):
    x = C()
    x.foo = y # assigns g's variable, because g defines x
  g(1)

The global keyword and (in Python 3) the nonlocal keywords override the default scoping rules.

While variable names are resolved statically, variable values are resolved dynamically. The value of the variable comes from the most recent definition or deletion of that variable at the time the variable is accessed. Values are looked up in the function closure where the variable resides.