Creating functions in a loop

Question

You're running into a problem with late binding -- each function looks up i as late as possible (thus, when called after the end of the loop, i will be set to 2).

Easily fixed by forcing early binding: change def f(): to def f(i=i): like this:

def f(i=i):
    return i

Default values (the right-hand i in i=i is a default value for argument name i, which is the left-hand i in i=i) are looked up at def time, not at call time, so essentially they're a way to specifically looking for early binding.

If you're worried about f getting an extra argument (and thus potentially being called erroneously), there's a more sophisticated way which involved using a closure as a "function factory":

def make_f(i):
    def f():
        return i
    return f

and in your loop use f = make_f(i) instead of the def statement.

The Explanation

The issue here is that the value of i is not saved when the function f is created. Rather, f looks up the value of i when it is called.

If you think about it, this behavior makes perfect sense. In fact, it's the only reasonable way functions can work. Imagine you have a function that accesses a global variable, like this:

global_var = 'foo'

def my_function():
    print(global_var)

global_var = 'bar'
my_function()

When you read this code, you would - of course - expect it to print "bar", not "foo", because the value of global_var has changed after the function was declared. The same thing is happening in your own code: By the time you call f, the value of i has changed and been set to 2.

The Solution

There are actually many ways to solve this problem. Here are a few options:

• Force early binding of i by using it as a default argument

  Unlike closure variables (like i), default arguments are evaluated immediately when the function is defined:

  for i in range(3):
      def f(i=i):  # <- right here is the important bit
          return i
  
      functions.append(f)
  

  To give a little bit of insight into how/why this works: A function's default arguments are stored as an attribute of the function; thus the current value of i is snapshotted and saved.

  >>> i = 0
  >>> def f(i=i):
  ...     pass
  >>> f.__defaults__  # this is where the current value of i is stored
  (0,)
  >>> # assigning a new value to i has no effect on the function's default arguments
  >>> i = 5
  >>> f.__defaults__
  (0,)
  

• Use a function factory to capture the current value of i in a closure

  The root of your problem is that i is a variable that can change. We can work around this problem by creating another variable that is guaranteed to never change - and the easiest way to do this is a closure:

  def f_factory(i):
      def f():
          return i  # i is now a *local* variable of f_factory and can't ever change
      return f
  
  for i in range(3):           
      f = f_factory(i)
      functions.append(f)
  

• Use functools.partial to bind the current value of i to f

  functools.partial lets you attach arguments to an existing function. In a way, it too is a kind of function factory.

  import functools
  
  def f(i):
      return i
  
  for i in range(3):    
      f_with_i = functools.partial(f, i)  # important: use a different variable than "f"
      functions.append(f_with_i)
  

Caveat: These solutions only work if you assign a new value to the variable. If you modify the object stored in the variable, you'll experience the same problem again:

>>> i = []  # instead of an int, i is now a *mutable* object
>>> def f(i=i):
...     print('i =', i)
...
>>> i.append(5)  # instead of *assigning* a new value to i, we're *mutating* it
>>> f()
i = [5]

Notice how i still changed even though we turned it into a default argument! If your code mutates i, then you must bind a copy of i to your function, like so:

• def f(i=i.copy()):
• f = f_factory(i.copy())
• f_with_i = functools.partial(f, i.copy())