How to define free-variable in python?

Question

The local/global/free variable definitions from python doc:

If a name is bound in a block, it is a local variable of that block, unless declared as nonlocal. If a name is bound at the module level, it is a global variable. (The variables of the module code block are local and global.) If a variable is used in a code block but not defined there, it is a free variable.

---

Code 1:

>>> x = 0 >>> def foo(): ...   print(x) ...   print(locals()) ...  >>> foo() 0 {} 

---

Code 2:

>>> def bar(): ...   x = 1 ...   def foo(): ...     print(x) ...     print(locals()) ...   foo() ...  >>> bar() 1 {'x':1} 

_{Free variables are returned by locals() when it is called in function blocks, but not in class blocks.}

---

In Code 1, x is a global variable, and it's used but not defined in foo().
However it's not a free variable, because it's not returned by locals().
I think it's not what the doc said. Is there a technical definition for free variable?

Answer 1

Definition of a free variable: Used, but neither global nor bound.

For example:

1. x is not free in Code 1, because it's a global variable.
2. x is not free in bar() in Code 2, because it's a bound variable.
3. x is free in foo().

Python makes this distinction because of closures. A free variable is not defined in the current environment, i. e. collection of local variables, and is also not a global variable! Therefore it must be defined elsewhere. And this is the concept of closures. In Code 2, foo() closes on x defined in bar(). Python uses lexical scope. This means, the interpreter is able to determine the scope by just looking at the code.

For example: x is known as a variable in foo(), because foo() is enclosed by bar(), and x is bound in bar().

Global scope is treated specially by Python. It would be possible to view the global scope as an outermost scope, but this is not done because of performance (I think). Therefore it is not possible that x is both free and global.

Exemption

Life is not so simple. There exist free global variables. Python docs (Execution model) says:

The global statement has the same scope as a name binding operation in the same block. If the nearest enclosing scope for a free variable contains a global statement, the free variable is treated as a global.

>>> x = 42 >>> def foo(): ...   global x ...   def baz(): ...     print(x) ...     print(locals()) ...   baz() ...  >>> foo() 42 {} 

I didn't know that myself. We are all here to learn.

Answer 2

From what I have understood the documentation is indeed a bit ambiguous on free variables. There are free global variables which are treated as plain globals and lexically bound free variables. Eli Bendersky sum's it up nicely in a blog post on symbol tables:

Unfortunately, there's a shorthand in the core of Python that may initially confuse readers as to exactly what constitutes a "free" variable. Fortunately, it's a very slight confusion that's easy to put in order. The execution model reference says:

If a variable is used in a code block but not defined there, it is a free variable.

This is consistent with the formal definition. In the source, however, "free" is actually used as a shorthand for "lexically bound free variable" (i.e. variables for which a binding has been found in an enclosing scope), with "global" being used to refer to all remaining free variables. So when reading the CPython source code it is important to remember that the full set of free variables includes both the variables tagged specifically as "free", as well as those tagged as "global".

Thus, to avoid a confusion I say "lexically bound" when I want to refer to the variables actually treated in CPython as free.

(emphasis mine)

The reason why this shorthand was used is probably because when you have a global free variable there's really no change whatsoever in the bytecode emitted. If a global variable is 'free' or if it isn't doesn't change the fact that the look-up for that name will use LOAD_GLOBAL in both cases. So global free variables aren't all that special.

On the other hand, lexically bound variables are treated specially and are enclosed in cell objects, the objects are the storage space for lexically bound free variables and are located in the __closure__ attribute for a given function. A special LOAD_DEREF instruction is created for these that examines the cells present for the free variables. The description for the LOAD_DEREF instruction is:

LOAD_DEREF(i)

Loads the cell contained in slot i of the cell and free variable storage

So in Python free variables only make a difference as a concept in situations where a definition for an object that has state is lexically (i.e statically) nested in another definition for an object that has state.