How to make a repeating generator in Python

Question

How do you make a repeating generator, like xrange, in Python? For instance, if I do:

>>> m = xrange(5)
>>> print list(m)
>>> print list(m)

I get the same result both times — the numbers 0..4. However, if I try the same with yield:

>>> def myxrange(n):
...   i = 0
...   while i < n:
...     yield i
...     i += 1
>>> m = myxrange(5)
>>> print list(m)
>>> print list(m)

The second time I try to iterate over m, I get nothing back — an empty list.

Is there a simple way to create a repeating generator like xrange with yield, or generator comprehensions? I found a workaround on a Python tracker issue, which uses a decorator to transform a generator into an iterator. This restarts every time you start using it, even if you didn't use all the values last time through, just like xrange. I also came up with my own decorator, based on the same idea, which actually returns a generator, but one which can restart after throwing a StopIteration exception:

@decorator.decorator
def eternal(genfunc, *args, **kwargs):
  class _iterable:
    iter = None
    def __iter__(self): return self
    def next(self, *nargs, **nkwargs):
      self.iter = self.iter or genfunc(*args, **kwargs):
      try:
        return self.iter.next(*nargs, **nkwargs)
      except StopIteration:
        self.iter = None
        raise
  return _iterable()

Is there a better way to solve the problem, using only yield and/or generator comprehensions? Or something built into Python? So I don't need to roll my own classes and decorators?

Update

The comment by u0b34a0f6ae nailed the source of my misunderstanding:

xrange(5) does not return an iterator, it creates an xrange object. xrange objects can be iterated, just like dictionaries, more than once.

My "eternal" function was barking up the wrong tree entirely, by acting like an iterator/generator (__iter__ returns self) rather than like a collection/xrange (__iter__ returns a new iterator).

Answer 1

Not directly. Part of the flexibility that allows generators to be used for implementing co-routines, resource management, etc, is that they are always one-shot. Once run, a generator cannot be re-run. You would have to create a new generator object.

However, you can create your own class which overrides __iter__(). It will act like a reusable generator:

def multigen(gen_func):
    class _multigen(object):
        def __init__(self, *args, **kwargs):
            self.__args = args
            self.__kwargs = kwargs
        def __iter__(self):
            return gen_func(*self.__args, **self.__kwargs)
    return _multigen

@multigen
def myxrange(n):
   i = 0
   while i < n:
     yield i
     i += 1
m = myxrange(5)
print list(m)
print list(m)

Answer 2

Using itertools its super easy.

import itertools

alist = [1,2,3]
repeatingGenerator = itertools.cycle(alist)

print(next(generatorInstance)) #=> yields 1
print(next(generatorInstance)) #=> yields 2
print(next(generatorInstance)) #=> yields 3
print(next(generatorInstance)) #=> yields 1 again!

Answer 3

If you write a lot of these, John Millikin's answer is the cleanest it gets.

But if you don't mind adding 3 lines and some indentation, you can do it without a custom decorator. This composes 2 tricks:

1. [Generally useful:] You can easily make a class iterable without implementing .next() - just use a generator for __iter__(self)!

2. Instead of bothering with a constructor, you can define a one-off class inside a function.

=>

def myxrange(n):
    class Iterable(object):
        def __iter__(self):
            i = 0
            while i < n:
                yield i
                i += 1
    return Iterable()

Small print: I didn't test performance, spawning classes like this might be wasteful. But awesome ;-)