This creates a future
loop.run_in_executor(None, some_fn)
This creates a task
asyncio.create_task(some_fn())
What is the difference between the two?
According to my understanding:
The former spins up another thread and executes the function (which is typically IO-bound) concurrently.
The latter creates a task within the same thread and executes the function concurrently.
Is this correct? The documentation isn't really helpful in understanding the differences between the two.
class asyncio. Future (*, loop=None) A Future represents an eventual result of an asynchronous operation. Not thread-safe. Future is an awaitable object.
Tasks within Asyncio are responsible for the execution of coroutines within an event loop. These tasks can only run in one event loop at one time and in order to achieve parallel execution you would have to run multiple event loops over multiple threads.
futures. This module was added in Python 3.2 for providing the developers a high-level interface for launching asynchronous tasks. It is an abstraction layer on the top of Python's threading and multiprocessing modules for providing the interface for running the tasks using pool of thread or processes.
The future remains "pending" because the callback that would update is supposed to be called by the event loop, which is currently not operational - it just sits in a queue. Replacing time. sleep(0.1) with await asyncio.
What is the difference between [futures and tasks]?
In short, future is the more general concept of a container of an async result, akin to a JavaScript promise. Task is a subclass of future specialized for executing coroutines.
Nothing in the definition of asyncio future indicates multi-threaded execution, and asyncio is in fact strongly single-threaded. It's a specific feature of run_in_executor
that it safely synchronizes functions it invokes in other threads with an asyncio future it creates and returns.
Also, be careful not to confuse asyncio futures with concurrent.futures futures, the latter being indeed multi-threaded. Despite similarities in the API coming from the fact that asyncio futures were inspired by the ones from concurrent.futures
, they work in conceptually different ways and cannot be used interchangeably.
*My answers are assuming you use the default loops provided by Python based on your OS with CPython.
The asyncio.create_task
method is used to schedule the execution of a coroutine (here some_fn
) on the event loop. Since the event loop runs on a single thread, the coroutine will be executed asynchronously to the current context by using cooperative multitasking.
Contrary to that, loop.run_in_executor(None, some_fn)
, will use the ThreadPoolExecutor
to create a thread pool, and perform the tasks, usually BlockingIO, with preemptive multitasking. You can see here the implementation for ThreadPoolExecutor
in CPython. In the implementation, you'll see that it creates a queue and a pool of threads which will get tasks from the queue and perform them. So yes - it will create a new thread for your scheduled task.
A couple of notes:
Task
is a Future
Executor
explicitly. This will be both more efficient since your code now uses cooperative multitasking, and will simplify execution since you know when your code gives back control to the event loop, thus managing concurrency (e.g. race condition) is either very simple or even non-existent.If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With