Using yield from with conditional in python

Question

Let's say I have a generator like this:

def a(data):
    for val in data:
        yield val

And let's say I want to wrap this generator in another generator, b, that only yields some of the values from a, depending on their value. b should be able to forward values sent back from the caller to a. I know that the most current way of wrapping a generator in another generator is to use the yield from statement. Something like:

def b(data):
    yield from val = a(data) if val == "foo"

I know that syntax is wrong (it's just to get across the idea), so I'm wondering if there is a correct way to make yield from work with a conditional statement. Or is there some other construct I should use?

Answer 1

As you said, yield from is only for the case where you want to pass everything from the wrapped generator through. If you don't want that, you need to manually iterate over the wrapped generator and do what you want:

def b(data):
    for value in a(data):
        if some_condition(value):
            yield value
        # otherwise don't yield it.

If you want to handle send, you need to handle that yourself too. Here is a simple example that you can play around with to see what's going on:

def a():
    sent = yield "Begin"
    for x in [1, 2, 3]:
        if sent:
            sent = yield "You sent {0}".format(sent)
        else:
            sent = yield x

def b():
    gen = a()
    val = yield next(gen)
    while True:
        wrappedVal = gen.send(val)
        if wrappedVal == 2:
            continue
        val = yield wrappedVal

Basically in this example b "hides" the value 2 if it is yielded by a. (I adapted this from my answer to this question, which is similarly about wrapping generators, but in a slightly different way. You may find discussion there useful, though.)

However, you would need to be very careful when doing this. Since b may skip values from a, any caller that tries to send values into b may not get the expected results, because b may skip values in unexpected places. In cases where you're using send, it often means the caller watches the generator for certain yielded values that communicate some information, then sends values back in response. With b in the middle, this communication may get out of sync. For instance, suppose a yields a "message" which b yields through to the caller, and the caller sends a response back, which b then sends to a. But suppose b swallows a's response. This will cause problems for the caller. You may still be able to make it work, but it just means you have to be very careful in writing b to main any communication channels that are expected by the API of a.