Lazy iterators (generators) with asyncio

Question

I have a blocking, non-async code like this:

def f():
    def inner():
        while True:
            yield read()
    return inner()

With this code the caller can choose when to stop the function to generate data. How to change this to async? This solution doesn't work:

async def f():
    async def inner():
        while True:
            yield await coroutine_read()
    return inner()

... because yield can't be used in async def functions. If i remove the async from the inner() signature, I can't use await anymore.

Answer 1

Upd:

Starting with Python 3.6 we have asynchronous generators and able to use yield directly inside coroutines.

---

As noted above, you can't use yield inside async funcs. If you want to create coroutine-generator you have to do it manually, using __aiter__ and __anext__ magic methods:

import asyncio


# `coroutine_read()` generates some data:
i = 0
async def coroutine_read():
    global i
    i += 1
    await asyncio.sleep(i)
    return i


# `f()` is asynchronous iterator.
# Since we don't raise `StopAsyncIteration` 
# it works "like" `while True`, until we manually break.
class f:
    async def __aiter__(self):
        return self

    async def __anext__(self):
        return await coroutine_read()


# Use f() as asynchronous iterator with `async for`:
async def main():
    async for i in f():
        print(i)
        if i >= 3:
            break


if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

Output:

1
2
3
[Finished in 6.2s]

You may also like to see other post, where StopAsyncIteration uses.