Why did Python 3 changes to exec break this code?

Question

I looked through the myriad 'Python exec' threads on SO, but couldn't find one that answered my issue. Terribly sorry if this has been asked before. Here's my problem:

# Python 2.6: prints 'it is working'
# Python 3.1.2: "NameError: global name 'a_func' is not defined"
class Testing(object):
  def __init__(self):
    exec("""def a_func():
      print('it is working')""")
    a_func()

Testing()

---

# Python 2.6: prints 'it is working'
# Python 3.1.2: prints 'it is working'
class Testing(object):
  def __init__(self):
    def a_func():
      print('it is working')
    a_func()

Testing()

As the standard function definition works in both Python versions, I'm assuming the problem must be a change to the way exec works. I read the API docs for 2.6 and 3 for exec and also read the "What's New In Python 3.0" page and couldn't see any reason why the code would break.

Answer 1

You can see the generated bytecode for each Python version with:

>>> from dis import dis

And, for each interpreter:

#Python 3.2
>>> dis(Testing.__init__)
...
  5          10 LOAD_GLOBAL              1 (a_func)
...

#Python 2.7
>>> dis(Testing.__init__)
...
  5           8 LOAD_NAME                0 (a_func)
...

As you can see, Python 3.2 searches for a global value (LOAD_GLOBAL) named a_func and 2.7 first searches the local scope (LOAD_NAME) before searching the global one.

If you do print(locals()) after the exec, you'll see that a_func is created inside the __init__ function.

I don't really know why it's done that way, but seems to be a change on how symbol tables are processed.

BTW, if want to create a a_func = None on top of your __init__ method to make the interpreter know it's a local variable, it'll not work since the bytecode now will be LOAD_FAST and that don't make a search but directly gets the value from a list.

The only solution I see is to add globals() as second argument to exec, so that will create a_func as a global function an may be accessed by the LOAD_GLOBAL opcode.

Edit

If you remove the exec statement, Python2.7 change the bytecode from LOAD_NAME to LOAD_GLOBAL. So, using exec, your code will always be slower on Python2.x because it has to search the local scope for changes.

As Python3's exec is not a keyword, the interpreter can't be sure if it's really executing new code or doing something else... So the bytecode don't change.

E.g.

>>> exec = len
>>> exec([1,2,3])
3

---

tl;dr

exec('...', globals()) may solve the problem if you don't care the result being added to global namespace

Answer 2

Completing the answer above, just in case. If the exec is in some function, I would recommend using the three-argument version as follows:

def f():
    d = {}
    exec("def myfunc(): ...", globals(), d)
    d["myfunc"]()

This is the cleanest solution, as it doesn't modify any namespace under your feet. Instead, myfunc is stored in the explicit dictionary d.