Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to handle an exhausted iterator?

Tags:

While searching the Python Documentation I found the equivalent python implementation of Pythons build-in zip() function.

Instead of catching a StopIteration exception which signals that there are no further items produced by the iterator the author(s) use an if statement to check if the returned default value form next() equals object() ("sentinel") and stop the generator:

def zip(*iterables):
    # zip('ABCD', 'xy') --> Ax By
    sentinel = object()
    iterators = [iter(it) for it in iterables]
    while iterators:
        result = []
        for it in iterators:
            elem = next(it, sentinel)

            if elem is sentinel:
                return

            result.append(elem)
        yield tuple(result)

I wonder now if there is any difference between the exception catching or an if statement as used by the Python Docs?

Or better, as @hiro protagonist pointed out:
What's wrong with using a try statement considering EAFP (Easier to ask for forgiveness than permission) in Python?

def zip(*iterables):
    # zip('ABCD', 'xy') --> Ax By
    iterators = [iter(it) for it in iterables]
    while iterators:
        result = []
        for it in iterators:

            try:
                elem = next(it)
            except StopIteration:
                return

            result.append(elem)
        yield tuple(result)

Also as Stoyan Dekov mentioned "A try/except block is extremely efficient if no exceptions are raised. Actually catching an exception is expensive." (see the docs for more information)
But an exception would only occur once, namely as soon as the iterator is exhausted. So exception handling would be the better solution in this case?

like image 510
elegent Avatar asked Jul 30 '15 08:07

elegent


People also ask

What does __ ITER __ do in Python?

The __iter__() function returns an iterator for the given object (array, set, tuple, etc. or custom objects). It creates an object that can be accessed one element at a time using __next__() function, which generally comes in handy when dealing with loops.

Does a generator return an iterator?

Generator functions are written using the function* syntax. When called, generator functions do not initially execute their code. Instead, they return a special type of iterator, called a Generator.

Why every iterator is not generator?

So lists, tuples, sets, dictionaries... are iterators. But those are not generators, because all of the elements they contain are defined and evaluated after the container initialization, and they can be iterated over many times. Therefore, some iterators are not generators.

Is iterator memory efficient?

Iterators will be faster and have better memory efficiency. Just think of an example of range(1000) vs xrange(1000) . (This has been changed in 3.0, range is now an iterator.) With range you pre-build your list, but xrange is an iterator and yields the next item when needed instead.


2 Answers

You mean as opposed to this?

def zip2(*iterables):
    # zip('ABCD', 'xy') --> Ax By
    iterators = [iter(it) for it in iterables]
    while iterators:
        result = []
        for it in iterators:
            try:
                elem = next(it)
            except StopIteration:
                return

            result.append(elem)
        yield tuple(result)

interesting question... i'd have preferred this alternative version - especially considering EAFP (Easier to ask for forgiveness than permission.)

even if try/except is slower than the if statement; this happens once only - as soon as the first iterator is exhausted.

it may be worth noting that this is not the actual implementaion in python; just an implementation that is equivalent to the real implementation.


UPDATE according to comments:

note that PEP 479 suggests to return from the generator and not raise StopIteration.

like image 53
hiro protagonist Avatar answered Oct 11 '22 12:10

hiro protagonist


Generally raising exceptions is always considered an expensive operation in any programming language. There are plenty of websites to read why is that and I'm not going to go into details on what it involves.

From Python Docs.

A try/except block is extremely efficient if no exceptions are raised. Actually catching an exception is expensive.

Both using if/else and try/catch has its advantages and disadvantages depending on the situation.

  • For example try/catch is used mostly for cases where an exception is a rare event (e.g. the code will succeed in almost all cases).
  • In your example you know the loop will throw an exception every time it is invoked, which makes it very inefficient to use try/catch
like image 41
SDekov Avatar answered Oct 11 '22 11:10

SDekov