awaiting on future object internals

Question

Having this simple code:

import asyncio

async def main():
    f = asyncio.Future()
    await f

asyncio.run(main())

A coroutine(here main) can await on Future object. It's basically blocked until f either have a result or an exception set, or until they are cancelled.

Out of curiosity I wanted to know "How this waiting occurs"? I checked the Python implementation of Task.py, specifically def __step().

In it's simplest form with happy cases and when the returned result is a Future, it's:

.
.
result = coro.send(None)
.
.
blocking = getattr(result, '_asyncio_future_blocking', None)
.
.
if blocking:   # So `result` is a Future with `_asyncio_future_blocking == True`
    result._asyncio_future_blocking = False
    result.add_done_callback(self.__wakeup, context=self._context)
    self._fut_waiter = result
    if self._must_cancel:
        if self._fut_waiter.cancel(msg=self._cancel_message):
            self._must_cancel = False

I got all my other answers from this section(like when the result is a bare yield or CancellationError happens and etc.) except this one!

So it sets the result._asyncio_future_blocking to False and the add e callback to the Future. But this __wakeup is only gets called when the Future is done but it's not done yet. I can't see any self._loop.call_soon(self.__step). All I can say is somebody is watching the self._fut_waiter. I don't know the rest of the story.

Apparently a comment in Task class's proves that:

    # An important invariant maintained while a Task not done:
    #
    # - Either _fut_waiter is None, and _step() is scheduled;
    # - or _fut_waiter is some Future, and _step() is *not* scheduled.
    #
    # The only transition from the latter to the former is through
    # _wakeup().  When _fut_waiter is not None, one of its callbacks
    # must be _wakeup().

Is it somehow registered to the select function? I would appreciate if someone tells me where these futures are getting checked? How these futures communicate with the event loop? where is the waiting area?

I saw David Beazley's lecture on AsyncIO and I understood how his event loop works, but in AsyncIO framework, everything is a Future not file descriptors or sockets. So what is Future's role?

Answer 1

First of all, I dont feel qualified to answer your question and I do hope someone with more knowledge will get in touch with this topic. But since there are no other answers yet, I'll try my best to serve a qualified answer to this interesting topic.

A Future object is an awaitable Object. Awaitable Objects schedule themselves mostly as Tasks i.e due asyncio.gather; wait_for or wait. The Future object however is the lower level parent of Task object.

The key differences between Task and Future are that:

Coroutines can await on Future objects until they either have a result or an exception set, or until they are cancelled.

and

Tasks are used to run coroutines in event loops.

The above should answer your question So what is Future's role?. Your next questions are a little bit harder to answer, but its important to note that coroutines await Future object.

To discover It may helps to understand first what coroutines are. Couroutines are generator based objects and the syntax yield from and yield, with the methods send and throw makes this all happen.

throw(), send() methods for coroutines are used to push values and raise errors into Future-like objects.

[Source]

But all of this doesn't answer your question How these futures communicate with the event loop?. In fact there is no Scheduler and the communication is achieved by

using yield from future and yield from task.

Sources I use/used for learning asyncio:

• Python Docs
• PEP 3156; PEP 492; PEP 342; PEP 380
• Video of Robert Smallshire
• Video series by Łukasz Langa
• Video of David Beazley
• How does asyncio actually work?
• Async await; Coroutine and send; Suspension
• Does await yield to the eventloop?