filter object becomes empty after iteration? [duplicate]

Question

I'm learning how to use the filter function.

This is the code I've written:

people = [{'name': 'Mary', 'height': 160},
          {'name': 'Isla', 'height': 80},
          {'name': 'Sam'}]

people2 = filter(lambda x: "height" in x, people)

As you can see what I'm trying to do is to remove all the dictionaries that don't contain the 'height' key.

The code works properly, in fact if I do:

print(list(people2))

I get:

[{'name': 'Mary', 'height': 160}, {'name': 'Isla', 'height': 80}]

The problem is that if I do it twice:

print(list(people2))
print(list(people2))

the second time, I get an empty list.

Can you explain me why?

Answer 1

This is a classic python3 doh!.

A filter is a special iterable object you can iterate over. However, much like a generator, you can iterate over it only once. So, by calling list(people2), you are iterating over each element of the filter object to generate the list. At this point, you've reached the end of the iterable and nothing more to return.

So, when you call list(people2) again, you get an empty list.

Demo:

>>> l = range(10)
>>> k = filter(lambda x: x > 5, l)
>>> list(k)
[6, 7, 8, 9]
>>> list(k)
[]

I should mention that with python2, filter returns a list, so you don't run into this issue. The problem arises when you bring py3's lazy evaluation into the picture.

Answer 2

It's because what filter really turns is an iterator. This iterator doesn't really do anything until you start to use it's results, in this case when you cast it to a list. people2 is this thing that's ready to filter the list of people, then when list is called on it, it iterates through the list of people and delivers the filtered result. Now that iterator is done, there's nothing left for it to iterate over, so when you call list on it a second time, there's nothing there.

Read this for some more details - Lazy evaluation python