exhausted iterators - what to do about them?

Question

(In Python 3.1) (Somewhat related to another question I asked, but this question is about iterators being exhausted.)

# trying to see the ratio of the max and min element in a container c
filtered = filter(lambda x : x is not None and x != 0, c)
ratio = max(filtered) / min(filtered)

It took me half hour to realize what the problem is (the iterator returned by filter is exhausted by the time it gets to the second function call). How do I rewrite it in the most Pythonic / canonical way?

Also, what can I do to avoid bugs of this sort, besides getting more experience? (Frankly, I don't like this language feature, since these types of bugs are easy to make and hard to catch.)

Answer 1

The itertools.tee function can help here:

import itertools

f1, f2 = itertools.tee(filtered, 2)
ratio = max(f1) / min(f2)

Answer 2

you can convert an iterator to a tuple simply by calling tuple(iterator)

however I'd rewrite that filter as a list comprehension, which would look something like this

# original
filtered = filter(lambda x : x is not None and x != 0, c)

# list comp
filtered = [x for x in c if x is not None and x != 0]

Answer 3

Actually your code raises an exception that would prevent this problem! So I guess the problem was that you masked the exception?

>>> min([])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: min() arg is an empty sequence
>>> min(x for x in ())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: min() arg is an empty sequence

Anyways, you can also write a new function to give you the min and max at the same time:

def minmax( seq ):
    " returns the `(min, max)` of sequence `seq`"
    it = iter(seq)
    try:
        min = max = next(it)
    except StopIteration:
        raise ValueError('arg is an empty sequence')
    for item in it:
        if item < min:
            min = item
        elif item > max:
            max = item
    return min, max