Yield on recursive data structure

Question

I'm trying to use a generator with a Python class that works somewhat similarly to a linked list.

Here is a really simple example of what I mean:

class GeneratorTest():
    def __init__(self, list):
        if list:
            self.elem = list[0]
            if list[1:]:
                self.n = GeneratorTest(list[1:])
            else:
                self.n = None

    def __iter__(self):
        return self

    def next(self):
        my_next = self
        while my_next is not None:
            yield my_next
            my_next = my_next.n

Of course this is just an example, but it's enough to illustrate the point.

Now, I was expecting to be able to invoke something like:

g = GeneratorTest([1,2,3,4,5])
for x in g:
    print x

And have the cycle stop when it reached the last value, but the for loop just continues endlessly.

I'm quite new to generators, so I'm sure it's a basic premise I'm missing here.

Is the problem related to the fact that I yield the same object that creates the generator? I'm sure that if I had an object with a list of GeneratorTest objects, I could return each of these objects quite simply, but I feel as there should be a way to make this work without a "wrapper" object.

What am I missing here?

Answer 1

The problem is that next (or, in Py3, __next__) shouldn't be a generator - it should maintain its state externally, and return each value. Yours keeps returning a new generator each time, but since Python doesn't iterate over that generator, your loop never actually runs. This may mean you want __iter__ to return something other than self initially (although whatever it returns is required to have an __iter__ that returns self).

But the good news is that generators exist precisely to keep track of these rules for you. Move your current next code into __iter__ and everything works - Python does iterate over whatever __iter__ returns (as you would expect).