globals and locals in python exec()

Question

I'm trying to run a piece of python code using exec.

my_code = """ class A(object):   pass  print 'locals: %s' % locals() print 'A: %s' % A  class B(object):   a_ref = A """  global_env = {} local_env = {} my_code_AST = compile(my_code, "My Code", "exec") exec(my_code_AST, global_env, local_env)  print local_env 

which results in the following output

locals: {'A': <class 'A'>} A: <class 'A'> Traceback (most recent call last):   File "python_test.py", line 16, in <module>     exec(my_code_AST, global_env, local_env)   File "My Code", line 8, in <module>   File "My Code", line 9, in B NameError: name 'A' is not defined 

However, if I change the code to this -

my_code = """ class A(object):   pass  print 'locals: %s' % locals() print 'A: %s' % A  class B(A):   pass """  global_env = {} local_env = {} my_code_AST = compile(my_code, "My Code", "exec") exec(my_code_AST, global_env, local_env)  print local_env 

then it works fine - giving the following output -

locals: {'A': <class 'A'>} A: <class 'A'> {'A': <class 'A'>, 'B': <class 'B'>} 

Clearly A is present and accessible - what's going wrong in the first piece of code? I'm using 2.6.5, cheers,

Colin

* UPDATE 1 *

If I check the locals() inside the class -

my_code = """ class A(object):   pass  print 'locals: %s' % locals() print 'A: %s' % A  class B(object):   print locals()   a_ref = A """  global_env = {} local_env = {} my_code_AST = compile(my_code, "My Code", "exec") exec(my_code_AST, global_env, local_env)  print local_env 

Then it becomes clear that locals() is not the same in both places -

locals: {'A': <class 'A'>} A: <class 'A'> {'__module__': '__builtin__'} Traceback (most recent call last):   File "python_test.py", line 16, in <module>     exec(my_code_AST, global_env, local_env)   File "My Code", line 8, in <module>   File "My Code", line 10, in B NameError: name 'A' is not defined 

However, if I do this, there is no problem -

def f():   class A(object):     pass    class B(object):     a_ref = A  f()  print 'Finished OK' 

* UPDATE 2 *

ok, so the docs here - http://docs.python.org/reference/executionmodel.html

'A class definition is an executable statement that may use and define names. These references follow the normal rules for name resolution. The namespace of the class definition becomes the attribute dictionary of the class. Names defined at the class scope are not visible in methods.'

It seems to me that 'A' should be made available as a free variable within the executable statement that is the definition of B, and this happens when we call f() above, but not when we use exec(). This can be more easily shown with the following -

my_code = """ class A(object):   pass  print 'locals in body: %s' % locals() print 'A: %s' % A  def f():   print 'A in f: %s' % A  f()  class B(object):   a_ref = A """ 

which outputs

locals in body: {'A': <class 'A'>} A: <class 'A'> Traceback (most recent call last):   File "python_test.py", line 20, in <module>     exec(my_code_AST, global_env, local_env)   File "My Code", line 11, in <module>   File "My Code", line 9, in f NameError: global name 'A' is not defined 

So I guess the new question is - why aren't those locals being exposed as free variables in functions and class definitions - it seems like a pretty standard closure scenario.

Answer 1

Well, I believe it's either an implementation bug or an undocumented design decision. The crux of the issue is that a name-binding operation in the module-scope should bind to a global variable. The way it is achieved is that when in the module level, globals() IS locals() (try that one out in the interpreter), so when you do any name-binding, it assigns it, as usual, to the locals() dictionary, which is also the globals, hence a global variable is created.

When you look up a variable, you first check your current locals, and if the name is not found, you recursively check locals of containing scopes for the variable name until you find the variable or reach the module-scope. If you reach that, you check the globals, which are supposed to be the module scope's locals.

>>> exec(compile("import sys\nprint sys._getframe().f_code.co_name", "blah", "exec"), {}, {}) <module> >>> exec("a = 1\nclass A(object):\n\tprint a\n", {}, {}) Traceback (most recent call last):   File "<stdin>", line 1, in <module>   File "<string>", line 2, in <module>   File "<string>", line 3, in A NameError: name 'a' is not defined >>> d = {} >>> exec("a = 1\nclass A(object):\n\tprint a\n", d,d) 1 

This behavior is why inheritance worked (The name-lookup used code object's scope locals(), which indeed had A in it).

In the end, it's an ugly hack in the CPython implementation, special-casing globals lookup. It also causes some nonsensical artifical situations - e.g.:

>>> def f(): ...     global a ...     a = 1 ... >>> f() >>> 'a' in locals() True 

Please note that this is all my inference based on messing with the interpreter while reading section 4.1 (Naming and binding) of the python language reference. While this isn't definitive (I haven't opened CPython's sources), I'm fairly sure I'm correct about the behavior.

Answer 2

After print locals() and globals(),you will find the reason why exec throws "not defined" exception, and you can try this

d = dict(locals(), **globals()) exec (code, d, d)