What is the difference between locals and globals when using Python's eval()?

Question

Why does it make a difference if variables are passed as globals or as locals to Python's function eval()?

As also described in the documenation, Python will copy __builtins__ to globals, if not given explicitly. But there must be also some other difference which I cannot see.

Consider the following example function. It takes a string code and returns a function object. Builtins are not allowed (e.g. abs()), but all functions from the math package.

def make_fn(code):
    import math
    ALLOWED_LOCALS = {v:getattr(math, v)
        for v in filter(lambda x: not x.startswith('_'), dir(math))
    }
    return eval('lambda x: %s' % code, {'__builtins__': None}, ALLOWED_LOCALS)

It works as expected not using any local or global objects:

   fn = make_fn('x + 3')
   fn(5) # outputs 8

But it does not work using the math functions:

   fn = make_fn('cos(x)')
   fn(5)

This outputs the following exception:

   <string> in <lambda>(x)
   NameError: global name 'cos' is not defined

But when passing the same mapping as globals it works:

def make_fn(code):
   import math
   ALLOWED = {v:getattr(math, v)
      for v in filter(lambda x: not x.startswith('_'), dir(math))
   }
   ALLOWED['__builtins__'] = None
   return eval('lambda x: %s' % code, ALLOWED, {})

Same example as above:

   fn = make_fn('cos(x)')
   fn(5) # outputs 0.28366218546322625

What happens here in detail?

Answer 1

Python looks up names as globals by default; only names assigned to in functions are looked up as locals (so any name that is a parameter to the function or was assigned to in the function).

You can see this when you use the dis.dis() function to decompile code objects or functions:

>>> import dis
>>> def func(x):
...     return cos(x)
... 
>>> dis.dis(func)
  2           0 LOAD_GLOBAL              0 (cos)
              3 LOAD_FAST                0 (x)
              6 CALL_FUNCTION            1
              9 RETURN_VALUE        

LOAD_GLOBAL loads cos as a global name, only looking in the globals namespace. The LOAD_FAST opcode uses the current namespace (function locals) to look up names by index (function local namespaces are highly optimized and stored as a C array).

There are three more opcodes to look up names; LOAD_CONST (reserved for true constants, such as None and literal definitions for immutable values), LOAD_DEREF (to reference a closure) and LOAD_NAME. The latter does look at both locals and globals and is only used when a function code object could not be optimized, as LOAD_NAME is a lot slower.

If you really wanted cos to be looked up in locals, you'd have to force the code to be unoptimised; this only works in Python 2, by adding a exec() call (or exec statement):

>>> def unoptimized(x):
...     exec('pass')
...     return cos(x)
... 
>>> dis.dis(unoptimized)
  2           0 LOAD_CONST               1 ('pass')
              3 LOAD_CONST               0 (None)
              6 DUP_TOP             
              7 EXEC_STMT           

  3           8 LOAD_NAME                0 (cos)
             11 LOAD_FAST                0 (x)
             14 CALL_FUNCTION            1
             17 RETURN_VALUE        

Now LOAD_NAME is used for cos because for all Python knows, the exec() call added that name as a local.

Even in this case, the locals LOAD_NAME looks into, will be the locals of the function itself, and not the locals passed to eval, which are for only for the parent scope.