Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python asyncio: Future vs Task

This creates a future

loop.run_in_executor(None, some_fn)

This creates a task

asyncio.create_task(some_fn())

What is the difference between the two?

According to my understanding:

The former spins up another thread and executes the function (which is typically IO-bound) concurrently.

The latter creates a task within the same thread and executes the function concurrently.

Is this correct? The documentation isn't really helpful in understanding the differences between the two.

like image 645
nz_21 Avatar asked Nov 16 '20 01:11

nz_21


People also ask

What is futures in Asyncio Python?

class asyncio. Future (*, loop=None) A Future represents an eventual result of an asynchronous operation. Not thread-safe. Future is an awaitable object.

What is a task in Asyncio?

Tasks within Asyncio are responsible for the execution of coroutines within an event loop. These tasks can only run in one event loop at one time and in order to achieve parallel execution you would have to run multiple event loops over multiple threads.

What are Python futures?

futures. This module was added in Python 3.2 for providing the developers a high-level interface for launching asynchronous tasks. It is an abstraction layer on the top of Python's threading and multiprocessing modules for providing the interface for running the tasks using pool of thread or processes.

What does Future pending mean in Python?

The future remains "pending" because the callback that would update is supposed to be called by the event loop, which is currently not operational - it just sits in a queue. Replacing time. sleep(0.1) with await asyncio.


2 Answers

What is the difference between [futures and tasks]?

In short, future is the more general concept of a container of an async result, akin to a JavaScript promise. Task is a subclass of future specialized for executing coroutines.

Nothing in the definition of asyncio future indicates multi-threaded execution, and asyncio is in fact strongly single-threaded. It's a specific feature of run_in_executor that it safely synchronizes functions it invokes in other threads with an asyncio future it creates and returns.

Also, be careful not to confuse asyncio futures with concurrent.futures futures, the latter being indeed multi-threaded. Despite similarities in the API coming from the fact that asyncio futures were inspired by the ones from concurrent.futures, they work in conceptually different ways and cannot be used interchangeably.

like image 60
user4815162342 Avatar answered Oct 23 '22 15:10

user4815162342


*My answers are assuming you use the default loops provided by Python based on your OS with CPython.

The asyncio.create_task method is used to schedule the execution of a coroutine (here some_fn) on the event loop. Since the event loop runs on a single thread, the coroutine will be executed asynchronously to the current context by using cooperative multitasking.

Contrary to that, loop.run_in_executor(None, some_fn), will use the ThreadPoolExecutor to create a thread pool, and perform the tasks, usually BlockingIO, with preemptive multitasking. You can see here the implementation for ThreadPoolExecutor in CPython. In the implementation, you'll see that it creates a queue and a pool of threads which will get tasks from the queue and perform them. So yes - it will create a new thread for your scheduled task.


A couple of notes:

  1. A Task is a Future
  2. Generally speaking, if your code does not have blockingIO methods, and you can leverage asyncio for all your IOs, you will be better just using the event loop, without the need to use an Executor explicitly. This will be both more efficient since your code now uses cooperative multitasking, and will simplify execution since you know when your code gives back control to the event loop, thus managing concurrency (e.g. race condition) is either very simple or even non-existent.
like image 6
Mickey Avatar answered Oct 23 '22 15:10

Mickey