Difference between "global" and "import __main__"

Question

I defined three functions that should change a global variable x.

def changeXto1():
    global x
    x = 1

def changeXto2():
    from __main__ import x
    x = 2

def changeXto3():
    import __main__
    __main__.x = 3

x = 0
print x
changeXto1()
print x
changeXto2()
print x
changeXto3()
print x

It gives the result:

0
1
1
3

changeXto1 uses the normal global statement. The result is as expected x == 1. changeXto2 uses from __main__ import to address x. This doesn't work. Afterwards x is still 1. changeXto3 uses import main to address x via __main__.x. The result afterwards is 3 as expected.

Why doesn't from __main__ import work in changeXto2, while import __main__ is working in changeXto3? Why do we need a global statement in Python if we can address global variables also with the __main__ module?

Answer 1

This is related to how Python translate your code to bytecode (the compilation step).

When compiling a function, Python treat all variable that are assigned as local variable and perform an optimisation to reduce the number of name lookup it would have to do. Each local variable get assigned an index, and when the function is called their value will be stored in a stack local array addressed by index. The compiler will emit LOAD_FAST and STORE_FAST opcode to access the variable.

The global syntax indicate instead to the compiler that even if the variable is assigned a value, it should not be considered a local variable, should not be assigned an index. It will instead use LOAD_GLOBAL and STORE_GLOBAL opcode to access the variable. Those opcode are slower since they use the name to do a lookup in possibly many dictionaries (locals, globals).

If a variable is only accessed for reading the value, the compiler always emit LOAD_GLOBAL since it don't know whether it is supposed to be a local or global variable, and thus assume it is a global.

So, in your first function, using global x informs the compiler that you want it to treat the write access to x as writing to a global variable instead of a local variable. The opcodes for the function make it clear:

>>> dis.dis(changeXto1)
  3           0 LOAD_CONST               1 (1)
              3 STORE_GLOBAL             0 (x)
              6 LOAD_CONST               0 (None)
              9 RETURN_VALUE        

In your third example, you import the __main__ module into a local variable named __main__ and then assign to its x field. Since module are object that store all top-level mapping as fields, you are assigning to the variable x in the __main__ module. And as you found, the __main__ module fields directly map to the values in the globals() dictionary because your code is defined in the __main__ module. The opcodes show that you don't access x directly:

>>> dis.dis(changeXto3)
  2           0 LOAD_CONST               1 (-1)
              3 LOAD_CONST               0 (None)
              6 IMPORT_NAME              0 (__main__)
              9 STORE_FAST               0 (__main__)

  3          12 LOAD_CONST               2 (3)
             15 LOAD_FAST                0 (__main__)
             18 STORE_ATTR               1 (x)
             21 LOAD_CONST               0 (None)
             24 RETURN_VALUE        

The second example is interesting. Since you assign a value to the x variable, the compiler assume it is a local variable and does the optimisation. Then, the from __main__ import x does import the module __main__ and create a new binding the value of x in the module __main__ to the local variable named x. This is always the case, from ${module} import ${name} just create a new binding the current namespace. When you assign a new value to the variable x you just change the current binding, not the binding in module __main__ that is unrelated (though if the value is mutable, and you mutate it, the change will be visible through all the bindings). Here are the opcodes:

>>> dis.dis(f2)
  2           0 LOAD_CONST               1 (-1)
              3 LOAD_CONST               2 (('x',))
              6 IMPORT_NAME              0 (__main__)
              9 IMPORT_FROM              1 (x)
             12 STORE_FAST               0 (x)
             15 POP_TOP             

  3          16 LOAD_CONST               3 (2)
             19 STORE_FAST               0 (x)
             22 LOAD_CONST               0 (None)
             25 RETURN_VALUE        

A good way to think about this is that in Python all assignment are binding a name to a value in a dictionary, and dereference is just doing a dictionary lookup (this is a rough approximation, but pretty close to the conceptual model). When doing obj.field, then you are looking up the hidden dictionary of obj (accessible via obj.__dict__) for the "field" key.

When you have a naked variable name, then it is looked up in the locals() dictionary, then the globals() dictionary if it is different (they are the same when the code is executed at a module level). For an assignment, it always put the binding in the locals() dictionary, unless you declared that you wanted a global access by doing global ${name} (this syntax also works at top-level).

So translating your function, this is almost if you had written:

# NOTE: this is valid Python code, but is less optimal than
# the original code. It is here only for demonstration.

def changeXto1():
    globals()['x'] = 1

def changeXto2():
    locals()['x'] = __import__('__main__').__dict__['x']
    locals()['x'] = 2

def changeXto3():
    locals()['__main__'] = __import__('__main__')
    locals()['__main__'].__dict__['x'] = 3