Generator expression uses list assigned after the generator's creation

Question

I found this example and I can't understand why it works unpredictably? I supposed it must output [1, 8, 15] or [2, 8, 22].

array = [1, 8, 15] g = (x for x in array if array.count(x) > 0) array = [2, 8, 22] print(list(g))   >>>[8] 

Answer 1

The reason is that, at creation time, the generator (a for b in c if d) only evaluates c (which sometimes makes b predictable as well). But a, b, d are evaluated at consumption time (at each iteration). Here, it uses the current binding of array from the enclosing scope when evaluating d (array.count(x) > 0).

You can for instance do:

g = (x for x in [] if a) 

Without having declared a in advance. But, you have to make sure a exists when the generator is consumed.

But you cannot do similarly:

g = (x for x in a if True) 

Upon request:

You can observe similar (however not identical) patterns with a common generator function:

def yielder():     for x in array:         if array.count(x) > 0:             yield x  array = [1, 8, 15] y = yielder() array = [2, 8, 22] list(y) # [2, 8, 22] 

The generator function does not execute any of its body ahead of consumption. Hence, even the array in the for-loop header is bound late. An even more disturbing example occurs where we "switch out" array during iteration:

array = [1, 8, 15] y = yielder() next(y) # 1 array = [3, 7] next(y)  # still iterating [1, 8, 15], but evaluating condition on [3, 7] # StopIteration raised 

Answer 2

From the docs on Generator expressions:

Variables used in the generator expression are evaluated lazily when the __next__() method is called for the generator object (in the same fashion as normal generators). However, the iterable expression in the leftmost for clause is immediately evaluated, so that an error produced by it will be emitted at the point where the generator expression is defined, rather than at the point where the first value is retrieved.

So when you run

array = [1, 8, 15] g = (x for x in array if array.count(x) > 0) 

only the first array in the generator expression is evaluated. x and array.count(x) will only be evaluated when you call next(g). Since you make array point to another list [2, 8, 22] before consuming the generator you get the 'unexpected' result.

array = [2, 8, 22] print(list(g))  # [8]