Why are module-level variables in a Python exec inaccessible?

Question

I'm trying to use Python's exec in a project to execute embedded Python code.

The problem I've encountered is that variables created at the module-level in an exec statement are inaccessible from functions defined in the same module.

Let's say you have the following Python program:

x = 5
def foo():
    print x
foo()

If you put the above four lines in a file and ran it, it would work no problemo.

However, if you try running this same piece of code from within an exec statement, it won't work.

Here's our previous program, inside an exec statement:

import __builtin__

global_env = {'__builtins__': __builtin__}
local_env = dict()

exec """
x = 5
def foo():
    print x
foo()
""" in global_env, local_env

On execution, instead of working, it yields the following error:

Traceback (most recent call last):
  File "lab.py", line 94, in <module>
    """ in global_env, local_env
  File "<string>", line 5, in <module>
  File "<string>", line 4, in foo
NameError: global name 'x' is not defined

I thought module-level variables were stored globally, but it seems that, at least in exec, they're not.

For example, in the previous example, if you replace the call to foo() with:

print global_env
print local_env

You get:

{'__builtins__': <module '__builtin__' (built-in)>}
{'x': 5, 'foo': <function foo at 0x102c12938>}

So anything defined in the module-level (including x) is stored in locals().

But it's impossible to access x from anywhere except the module-level (of the exec statement). In particular, as we saw above, the local scope of x is invisible to functions defined in the same exec statement.

Workarounds

I've found two ways to workaround this issue, and make x accessible again.

The first one is using the global keyword in the function:

exec """
x = 5
def foo():
    global x
    print x
foo()
""" in global_env, local_env

The second one is using the same dictionary for globals() and locals() in exec:

exec """
x = 5
def foo():
    print x
foo()
""" in global_env

However, these are just half-fixes/workarounds that don't address the original issue.

So my questions are: Why are module-level variables in an exec stored locally, and why are inaccessible from anywhere but the module-level?

Some closely related StackOverflow posts:

• globals and locals in python exec()
• Cannot change global variables in a function through an exec() statement?

Answer 1

To understand what is going on, you need to read the docs very closely. The key part says:

If two separate objects are given as globals and locals, the code will be executed as if it were embedded in a class definition.

This means that local assignments get made into the local namespace (equivalent to class-level variables), but functions (i.e. methods) do not become closures if they try to reference a local (class) variable.

Compare your code with:

class Test(object):
    x = 1
    def foo():
        print x
    foo()

You'll get the same error, for the same reason. foo is not a closure, and so it attempts to reference x in the global namespace (unsuccessfully).