Scope of lambda functions and their parameters?

Question

I need a callback function that is almost exactly the same for a series of gui events. The function will behave slightly differently depending on which event has called it. Seems like a simple case to me, but I cannot figure out this weird behavior of lambda functions.

So I have the following simplified code below:

def callback(msg):     print msg  #creating a list of function handles with an iterator funcList=[] for m in ('do', 're', 'mi'):     funcList.append(lambda: callback(m)) for f in funcList:     f()  #create one at a time funcList=[] funcList.append(lambda: callback('do')) funcList.append(lambda: callback('re')) funcList.append(lambda: callback('mi')) for f in funcList:     f() 

The output of this code is:

mi mi mi do re mi 

I expected:

do re mi do re mi 

Why has using an iterator messed things up?

I've tried using a deepcopy:

import copy funcList=[] for m in ('do', 're', 'mi'):     funcList.append(lambda: callback(copy.deepcopy(m))) for f in funcList:     f() 

But this has the same problem.

Answer 1

When a lambda is created, it doesn't make a copy of the variables in the enclosing scope that it uses. It maintains a reference to the environment so that it can look up the value of the variable later. There is just one m. It gets assigned to every time through the loop. After the loop, the variable m has value 'mi'. So when you actually run the function you created later, it will look up the value of m in the environment that created it, which will by then have value 'mi'.

One common and idiomatic solution to this problem is to capture the value of m at the time that the lambda is created by using it as the default argument of an optional parameter. You usually use a parameter of the same name so you don't have to change the body of the code:

for m in ('do', 're', 'mi'):     funcList.append(lambda m=m: callback(m)) 

Answer 2

The problem here is the m variable (a reference) being taken from the surrounding scope. Only parameters are held in the lambda scope.

To solve this you have to create another scope for lambda:

def callback(msg):     print msg  def callback_factory(m):     return lambda: callback(m)  funcList=[] for m in ('do', 're', 'mi'):     funcList.append(callback_factory(m)) for f in funcList:     f() 

In the example above, lambda also uses the surounding scope to find m, but this time it's callback_factory scope which is created once per every callback_factory call.

Or with functools.partial:

from functools import partial  def callback(msg):     print msg  funcList=[partial(callback, m) for m in ('do', 're', 'mi')] for f in funcList:     f()