Why does a yield from inside __next__() return generator object?

Question

I am using yield to return the next value in the __next__() function in my class. However it does not return the next value, it returns the generator object.

I am trying to better understand iterators and yield. I might be doing it in the wrong way.

Have a look.

class MyString:
    def __init__(self,s):
        self.s=s

    def __iter__(self):
        return self

    def __next__(self):
        for i in range(len(self.s)):
            yield(self.s[i])

r=MyString("abc")
i=iter(r)
print(next(i))

This returns:

generator object __next__ at 0x032C05A0

Answer 1

next pretty much just calls __next__() in this case. Calling __next__ on your object will start the generator and return it (no magic is done at this point).

---

In this case, you might be able to get away with not defining __next__ at all:

class MyString:
    def __init__(self,s):
        self.s=s

    def __iter__(self):
        for i in range(len(self.s)):
            yield(self.s[i])
        # Or...
        # for item in self.s:
        #     yield item

If you wanted to use __iter__ and __next__ (to define an iterator rather than simply making an iterable), you'd probably want to do something like this:

class MyString:
    def __init__(self,s):
        self.s = s
        self._ix = None

    def __iter__(self):
        return self

    def __next__(self):
        if self._ix is None:
            self._ix = 0

        try:
            item = self.s[self._ix]
        except IndexError:
            # Possibly reset `self._ix`?
            raise StopIteration
        self._ix += 1
        return item

Answer 2

Let's take a look at the purpose of the __next__ method. From the docs:

iterator.__next__()

Return the next item from the container. If there are no further items, raise the StopIteration exception.

Now let's see what the yield statement does. Another excerpt from the docs:

Using a yield expression in a function’s body causes that function to be a generator

And

When a generator function is called, it returns an iterator known as a generator.

Now compare __next__ and yield: __next__ returns the next item from the container. But a function containing the yield keyword returns an iterator. Consequently, using yield in a __next__ method results in an iterator that yields iterators.

---

If you want to use yield to make your class iterable, do it in the __iter__ method:

class MyString:
    def __init__(self, s):
        self.s = s

    def __iter__(self):
        for s in self.s:
            yield s

The __iter__ method is supposed to return an iterator - and the yield keyword makes it do exactly that.

---

For completeness, here is how you would implement an iterator with a __next__ method. You have to keep track of the state of the iteration, and return the corresponding value. The easiest solution is probably to increment an index every time __next__ is called:

class MyString:
    def __init__(self,s):
        self.s = s
        self.index = -1

    def __iter__(self):
        return self

    def __next__(self):
        self.index += 1

        if self.index >= len(self.s):
            raise StopIteration

        return self.s[self.index]