Why do Python yield statements form a closure?

Question

I have two functions that return a list of functions. The functions take in a number x and add i to it. i is an integer increasing from 0-9.

def test_without_closure():     return [lambda x: x+i for i in range(10)]    def test_with_yield():     for i in range(10):         yield lambda x: x+i 

I would expect test_without_closure to return a list of 10 functions that each add 9 to x since i's value is 9.

print sum(t(1) for t in test_without_closure()) # prints 100 

I expected that test_with_yield would also have the same behavior, but it correctly creates the 10 functions.

print sum(t(1) for t in test_with_yield()) # print 55 

My question is, does yielding form a closure in Python?

Answer 1

Yielding does not create a closure in Python, lambdas create a closure. The reason that you get all 9s in "test_without_closure" isn't that there's no closure. If there weren't, you wouldn't be able to access i at all. The problem is that all closures contain a reference¹ to the same i variable, which will be 9 at the end of the function.

This situation isn't much different in test_with_yield. Why, then, do you get different results? Because yield suspends the run of the function, so it's possible to use the yielded lambdas before the end of the function is reached, i.e. before i is 9. To see what this means, consider the following two examples of using test_with_yield:

[f(0) for f in test_with_yield()] # Result: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]  [f(0) for f in list(test_with_yield())] # Result: [9, 9, 9, 9, 9, 9, 9, 9, 9, 9] 

What's happening here is that the first example yields a lambda (while i is 0), calls it (i is still 0), then advances the function until another lambda is yielded (i is now 1), calls the lambda, and so on. The important thing is that each lambda is called before the control flow returns to test_with_yield (i.e. before the value of i changes).

In the second example, we first create a list. So the first lambda is yielded (i is 0) and put into the list, the second lambda is created (i is now 1) and put into the list ... until the last lambda is yielded (i is now 9) and put into the list. And then we start calling the lambdas. So since i is now 9, all lambdas return 9.

---

¹ The important bit here is that closures hold references to variables, not copies of the value they held when the closure was created. This way, if you assign to the variable inside a lambda (or inner function, which create closures the same way that lambdas do), this will also change the variable outside of the lambda and if you change the value outside, that change will be visible inside the lambda.