python - what does yield (yield) do?

Question

Since python 2.5 there is the ability to send(), throw(), close() into a generator. Inside the defined generator one can 'catch' the sent data by doing something like:

def gen():
    while True:
        x = (yield)
        if x == 3:
            print('received 3!!')
            break
        else:
            yield x

What i am trying to play with is doing something like:

def gen2():
    while True:
        yield (yield)

Noticed that it is a legal generator which does something.. First thing i'm trying to figure out is:

Is there a good usage for such writing?

Also when doing something like:

g = gen2()
next(g)
g.send(10) # output: 10
g.send(2) # output: nothing
g.send(3) # output: 3
g.send(44) # output: nothing

Why each second 'send' does not do anything?

Answer 1

yield (yield) first yields None from the inner yield. It then receives a value from send or next. The inner yield evaluates to this received value, and the outer yield promptly yields that value.

---

Each yield conceptually has two parts:

1. Transmit a value to the caller of send or next.
2. Receive a value from the next send or next call.

Similarly, each send or next conceptually has two parts:

1. Transmit a value to the yield expression that the generator is currently paused at. (This value is None for next.)
2. Receive a value from the next yield expression.

The most confusing part of the system is probably that these parts are staggered. The two parts of a yield correspond to two different invocations of send or next, and the two parts of a send or next correspond to two different yields.

If we work through a simple example:

def gen():
    print('Not ran at first')
    yield (yield)

g = gen()  # Step 1
print(next(g))  # Step 2
print(g.send(1))  # Step 3
g.send(2)  # Step 4

Here's how things work out:

Inside the generator                      Outside the generator

Step 1

                                          g calls gen()
g returns a generator object 
without executing the print
just yet statement.
                                          >>> g
                                          <generator object gen at 0x7efe286d54f8>

Step 2

                                          next(g) sends None to g
g receives None, ignores it
  (since it is paused at the start
   of the function)

g prints ('not ran at first')

g executes the "transmit" phase
  of the inner yield, transmitting
  None
                                          next(g) receives None

Step 3

                                          g.send(1) sends 1 to g
g executes the "receive" phase
  of the inner yield, receiving 1
g executes the "transmit" phase
  of the outer yield, transmitting 1
                                          g.send(1) receives 1 from g

Step 4

                                          g.send(2) sends 2 to g
g executes the "receive" phase
  of the outer yield, receiving 2
g reaches the end of gen and raises
  a StopIteration
                                          g.send(2) raises the StopIteration
                                          from g

Answer 2

yield is an expression. The value of the expression is the value of whatever was sent using .send, or None if nothing was sent (including if next was used instead of .send). .send is a method call and thus of course also returns a value, which is the value yielded by the generator. In other words, every time you .send, a value (which may be None) is yielded, and every time you yield, a value (which may be None) is sent.

Here is a simple example:

def gen():
    sent1 = yield 1
    print(sent1, "was sent")
    sent2 = yield 2
    print(sent2, "was sent")
    print("Reached end of generator")

g = gen()
print(next(g), "was yielded")
print(g.send("A"), "was yielded")
print(g.send("B"), "was yielded")
next(g)

# output
1 was yielded
A was sent
2 was yielded
B was sent
Reached end of generator
# StopIteration is raised here

In your example, the first next yields None, since the first yield is the inner yield in yield (yield) (i.e., the one in parentheses). The first send passes 10 as the value this yield. Each subsequent value that you send becomes the value of one of the yields. The reason some of your send calls produce no output is that the inner yield specifies no value, so it yields None. As mentioned above, when you call send, a value is yielded; in your case, that value is None for the inner yield, so no output is displayed at the interactive prompt. The outer yield, on the other hand, does specify a value, namely the result of the inner yield. So when you send a value in to the inner yield, it will be yielded by the outer yield at the next iteration. (I'm assuming you're referring to output at the interactive prompt; if you run your code as a script, there will be no output at all, since you never print anything or otherwise produce explicit output.)

Here is another example that may be illuminating:

def gen():
    yield (yield (yield (yield "WHOA")))

>>> g = gen()
>>> next(g)
'WHOA'
>>> g.send(1)
1
>>> g.send(2)
2
>>> g.send(3)
3
>>> g.send(4)
Traceback (most recent call last):
  File "<pyshell#11>", line 1, in <module>
    g.send(4)
StopIteration

Notice that each time a value is sent in, it is immediately yielded back. This is because each yield yields the value of a more-deeply-nested yield. Each yield "becomes" the sent value and is immediately yielded by the next yield in the chain. This continues until all yields are exhausted and StopIteration is raised.

Similar questions about this have been asked before. My impression is that confusion tends to arise because people expect send to "just send" a value. But that is not the case. Using send advances the generator and yields the next result, just like using next. You can think of next(gen) as equivalent to gen.send(None).