How do I check if an iterator is actually an iterator container?

Question

I have a dummy example of an iterator container below (the real one reads a file too large to fit in memory):

class DummyIterator:
    def __init__(self, max_value):
        self.max_value = max_value

    def __iter__(self):
        for i in range(self.max_value):
            yield i

def regular_dummy_iterator(max_value):
    for i in range(max_value):
        yield i

This allows me to iterate over the value more than once so that I can implement something like this:

def normalise(data):
    total = sum(i for i in data)
    for val in data:
        yield val / total

# this works when I call next()
normalise(DummyIterator(100))

# this doesn't work when I call next()
normalise(regular_dummy_iterator(100))

How do I check in the normalise function that I am being passed an iterator container rather than a normal generator?

Answer 1

First of all: There is no such thing as a iterator container. You have an iterable.

An iterable produces an iterator. Any iterator is also an iterable, but produces itself as the iterator:

>>> list_iter = iter([])
>>> iter(list_iter) is list_iter
True

You don't have an iterator if the iter(ob) is ob test is false.

Answer 2

You can test whether you have an iterator (is consumed once next raises the StopIteration exception) vs just an iterable (can probably be iterated over multiple times) by using the collections.abcmodule. Here is an example:

from collections.abc import Iterable, Iterator

def my_iterator(): 
    yield 1

i = my_iterator()
a = []

isinstance(i, Iterator) # True
isinstance(a, Iterator) # False

What makes my_iterator() an Iterator is the presence of both the __next__ and __iter__ magic methods (and by the way, basically what is happening behind the scenes when you call isinstance on a collections.abc abstract base class is a test for the presence of certain magic methods).

Note that an iterator is also an Iterable, as is the empty list (i.e., both have the __iter__ magic method):

isinstance(i, Iterable) # True
isinstance(a, Iterable) # True

Also note, as was pointed out in Martijn Pieters' answer, that when you apply the generic iter() function to both, you get an iterator:

isinstance(iter(my_iterator()), Iterator) # True
isinstance(iter([])), Iterator) # True

The difference here between [] and my_iterator() is that iter(my_iterator()) returns itself as the iterator, whereas iter([]) produces a new iterator every time.

As was already mentioned in MP's same answer, your object above is not an "iterator container." It is an iterable object, i.e., "an iterable". Whether or not it "contains" something isn't really related; the concept of containing is represented by the abstract base class Container. A Container may be iterable, but it doesn't necessarily have to be.