If a python iterator returns iterable objects, how can I chain those objects into one big iterator?

Question

I'll give a simplified example here. Suppose I have an iterator in python, and each object that this iterator returns is itself iterable. I want to take all the objects returned by this iterator and chain them together into one long iterator. Is there a standard utility to make this possible?

Here is a contrived example.

x = iter([ xrange(0,5), xrange(5,10)])

x is an iterator that returns iterators, and I want to chain all the iterators returned by x into one big iterator. The result of such an operation in this example should be equivalent to xrange(0,10), and it should be lazily evaluated.

Answer 1

You can use itertools.chain.from_iterable

>>> import itertools
>>> x = iter([ xrange(0,5), xrange(5,10)])
>>> a = itertools.chain.from_iterable(x)
>>> list(a)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

If that's not available on your version (apparently, it's new in 2.6), you can just do it manually:

>>> x = iter([ xrange(0,5), xrange(5,10)])
>>> a = (i for subiter in x for i in subiter)
>>> list(a)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

Answer 2

Here's the old-fashioned way:

Python 2.2.3 (#42, May 30 2003, 18:12:08) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from __future__ import generators
>>> def mychain(iterables):
...     for it in iterables:
...         for item in it:
...             yield item
...
>>> x = iter([ xrange(0,5), xrange(5,10)])
>>> a = mychain(x)
>>> list(a)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>>

I'm not sure what the purpose or benefit of using iter() is, in this case:

>>> x = [xrange(0,5), xrange(5,10)]
>>> a = mychain(x)
>>> list(a)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>>