Weird behavior: Lambda inside list comprehension

Question

In python 2.6:

[x() for x in [lambda: m for m in [1,2,3]]]

results in:

[3, 3, 3]

I would expect the output to be [1, 2, 3]. I get the exact same problem even with a non list comprehension approach. And even after I copy m into a different variable.

What am I missing?

Answer 1

To make the lambdas remember the value of m, you could use an argument with a default value:

[x() for x in [lambda m=m: m for m in [1,2,3]]]
# [1, 2, 3]

This works because default values are set once, at definition time. Each lambda now uses its own default value of m instead of looking for m's value in an outer scope at lambda execution time.

Answer 2

The effect you’re encountering is called closures, when you define a function that references non-local variables, the function retains a reference to the variable, rather than getting its own copy. To illustrate, I’ll expand your code into an equivalent version without comprehensions or lambdas.

inner_list = []
for m in [1, 2, 3]:
    def Lambda():
         return m
    inner_list.append(Lambda)

So, at this point, inner_list has three functions in it, and each function, when called, will return the value of m. But the salient point is that they all see the very same m, even though m is changing, they never look at it until called much later.

outer_list = []
for x in inner_list:
    outer_list.append(x())

In particular, since the inner list is constructed completely before the outer list starts getting built, m has already reached its last value of 3, and all three functions see that same value.

Answer 3

Long story short, you don't want to do this. More specifically, what you're encountering is an order of operations problem. You're creating three separate lambda's that all return m, but none of them are called immediately. Then, when you get to the outer list comprehension and they're all called the residual value of m is 3, the last value of the inner list comprehension.

-- For comments --

>>> [lambda: m for m in range(3)]
[<function <lambda> at 0x021EA230>, <function <lambda> at 0x021EA1F0>, <function <lambda> at 0x021EA270>]

Those are three separate lambdas.

And, as further evidence:

>>> [id(m) for m in [lambda: m for m in range(3)]]
[35563248, 35563184, 35563312]

Again, three separate IDs.

Answer 4

Look at the __closure__ of the functions. All 3 point to the same cell object, which keeps a reference to m from the outer scope:

>>> print(*[x.__closure__[0] for x in [lambda: m for m in [1,2,3]]], sep='\n')
<cell at 0x00D17610: int object at 0x1E2139A8>
<cell at 0x00D17610: int object at 0x1E2139A8>
<cell at 0x00D17610: int object at 0x1E2139A8>

If you don't want your functions to take m as a keyword argument, as per unubtu's answer, you could instead use an additional lambda to evaluate m at each iteration:

>>> [x() for x in [(lambda x: lambda: x)(m) for m in [1,2,3]]]
[1, 2, 3]