python asyncios create_task and await functions

Question

I am trying to understand pythons asynico module and came across the following piece of code at https://docs.python.org/3/library/asyncio-task.html#asyncio.create_task

import time
import asyncio

async def say_after(delay, what):
    await asyncio.sleep(delay)
    print(what)

async def main():
    task1 = asyncio.create_task(
        say_after(1, 'hello'))

    task2 = asyncio.create_task(
        say_after(2, 'world'))

    print('started at', time.strftime('%X'))

    # Wait until both tasks are completed (should take
    # around 2 seconds.)
    await task1
    await task2

    print('finished at', time.strftime('%X'))

asyncio.run(main())

It turns out that await task2, (or task1, but not both) can simply be removed and the code appears to be doing exactly the same. I find this very counterintuitive, what is happening here? Thank you for your time.

Answer 1

There are three different scenarios you've posed:

1. No await statements (comment-out both)
2. Use only await task1 (comment-out the second)
3. Use only await task2 (commment-out the first)

Here's your script; extend the sleep time on task2 a bit just for illustration's sake.

# tasktest.py
import time
import asyncio

async def say_after(delay, what):
    await asyncio.sleep(delay)
    print(what)

async def main():
    task1 = asyncio.create_task(
        say_after(1, 'hello'))

    task2 = asyncio.create_task(
        say_after(3, 'world'))

    print('started at', time.strftime('%X'))
    await task1
    # await task2
    print('finished at', time.strftime('%X'))

asyncio.run(main())

1. No await statements

Here's the meat of asyncio.run():

loop = events.new_event_loop()
try:
    events.set_event_loop(loop)
    loop.set_debug(debug)
    return loop.run_until_complete(main)   # < -----
finally:
    try:
        _cancel_all_tasks(loop)            # < -----
        loop.run_until_complete(loop.shutdown_asyncgens())
    finally:
        events.set_event_loop(None)
        loop.close()

Importantly, the loop only cares that main() is complete, and then cancels all other tasks that are associated with the running event loop. (Each task is tied to an event loop when it is specified.)

If you define main() without any await statements, create_task() schedules the tasks to be executed, but main() does not wait for either one of them to complete.

2. await task1

Setup:

await task1
# await task2

Output:

(base_py37) $ python3 tasktest.py 
started at 11:06:46
hello
finished at 11:06:47

Both tasks move from pending to running, but only task1 completes, because main() only awaited on a task that takes ~1 second, not long enough for task2 to run.^* (Notice that main() takes only 1 second.)

3. await task2

Setup:

# await task1
await task2

Output:

(base_py37) $ python3 tasktest.py 
started at 11:08:37
hello
world
finished at 11:08:40

Both tasks move from pending to running, and now both task1 and task2 complete, because main() awaited on a task that takes ~3 seconds, long enough for both tasks to run to completion.

---

_{*This applies at least to my setup (Mac OSX, ...) but as mentioned in the other answer here, the timing may play out differently on another setup and, if the task run-times are similar, both may get to run in places like case # 2.}

Answer 2

await does not start the coroutines in the tasks, it simply tells the coroutine main to wait for those two coros. The event loop is implicitly started with run_until_complete which in turn means it will wait for the coro passed to it (main) to complete. It is up to that coro to extend its own lifetime (by awaiting) long enough to ensure that the tasks created from inside it can complete.

async def main():
    task1 = asyncio.create_task(
        say_after(1, 'hello'))
    task2 = asyncio.create_task(
        say_after(2, 'world'))
    #await task1
    #await task2
    print(asyncio.all_tasks(asyncio.get_event_loop()))

# will print (added line breaks and shortened paths for readibility):
{
    <Task pending coro=<main() running at C:/Users/.../lmain.py:17> cb=[_run_until_complete_cb() at C:...\lib\asyncio\base_events.py:150]>, 
    <Task pending coro=<say_after() running at C:/Users/.../lmain.py:5>>, 
    <Task pending coro=<say_after() running at C:/Users/.../lmain.py:5>>
}

As you see, all three coros are running without awaiting anything. It's just that the two say_after coros will take longer than the main that implicitly controls how long the event loop runs.

If you made main wait for work done in the loop long enough, both tasks would complete:

async def main():
    task1 = asyncio.create_task(
        say_after(1, 'hello'))
    task2 = asyncio.create_task(
        say_after(2, 'world'))
    print('started at', time.strftime('%X'))
    #await task1
    #await task2
    await asyncio.sleep(5)
    print('finished at', time.strftime('%X'))

# output
started at 15:31:48
hello
world
finished at 15:31:53

So, which task, if any, completes when you test commenting out the awaiting of task1 and/or task2 above is basically just a matter of timing, mainly influenced by HW, OS and possibly runtime (i.e. IDE vs. shell).

P.S. Tasks only have three states: pending, cancelled and finished. Every task is in state pending right after its creation and remains in that state until the coroutine wrapped in it either terminates (in any way) or until it gets cancelled by the event loop controlling it.