Semantics of `async for` - can __anext__ calls overlap?

Question

If __anext__ gives control back to event loop (via an await, or call_soon/call_later), is it possible for another __anext__ to be called on the same instance, while first one had not resolved yet, or they would be queued? Are there any other cases where it would be unsafe to assume that only one __anext__ is running at the same time?

Answer 1

Short answer: Using async for does not overlap, but calling __anext__ does.

Long answer:

Here is what I made while playing with __anext__ mechanics:

import asyncio

class Foo(object):
    def __init__(self):
        self.state = 0

    def __aiter__(self):
        return self

    def __anext__(self):
        def later():
            try:
                print(f'later: called when state={self.state}')

                self.state += 1
                if self.state == 3:
                    future.set_exception(StopAsyncIteration())
                else:
                    future.set_result(self.state)
            finally:
                print(f'later: left when state={self.state}')

        print(f'__anext__: called when state={self.state}')
        try:
            future = asyncio.Future()

            loop.call_later(0.1, later)

            return future
        finally:
            print(f'__anext__: left when state={self.state}')

async def main():
    print('==== async for ====')
    foo = Foo()
    async for x in foo:
        print('>', x)

    print('==== __anext__() ====')
    foo = Foo()
    a = foo.__anext__()
    b = foo.__anext__()
    c = foo.__anext__()
    print('>', await a)
    print('>', await b)
    print('>', await c)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.run_until_complete(asyncio.gather(*asyncio.Task.all_tasks()))
loop.close()

I had implemented __anext__ to return future instead of just being async def, so I have better control of intermediate steps of resolving these futures.

Here is an output:

==== async for ====
__anext__: called when state=0
__anext__: left when state=0
later: called when state=0
later: left when state=1
> 1
__anext__: called when state=1
__anext__: left when state=1
later: called when state=1
later: left when state=2
> 2
__anext__: called when state=2
__anext__: left when state=2
later: called when state=2
later: left when state=3
==== __anext__() ====
__anext__: called when state=0
__anext__: left when state=0
__anext__: called when state=0
__anext__: left when state=0
__anext__: called when state=0
__anext__: left when state=0
later: called when state=0
later: left when state=1
later: called when state=1
later: left when state=2
later: called when state=2
later: left when state=3
> 1
> 2
~~~ dies with StopAsyncIteration ~~~

In async for case it can be seen that __anext__ completes first, then event loop kicks in and runs whatever was scheduled with a delay. If __anext__s were stacking, event loop would take this opportunity to schedule another __anext__ call until delayed later would start - instead, event loop blocks until it's later's time to run.

So, if your async iterator is only to be used in async for, it's safe to assume that there would be only one __anext__ running at the same time.

With __anext__ it's worse: you can stack them up as much as you'd like. However, this shouldn't be of much concern, if your __anext__ is coroutine - it shouldn't keep any state when invoked anyway. Or at least I think so.