asyncio, wrapping a normal function as asynchronous

Question

Is a function like:

async def f(x):
    time.sleep(x)

await f(5)

properly asynchronous/non-blocking?

Is the sleep function provided by asyncio any different?

and finally, is aiorequests a viable asynchronous replacement for requests?

(to my mind it basically wraps main components as asynchronous)

https://github.com/pohmelie/aiorequests/blob/master/aiorequests.py

Answer 1

The provided function is not a correctly written async function because it invokes a blocking call, which is forbidden in asyncio. (A quick hint that there's something wrong with the "coroutine" is that it doesn't contain a single await.) The reason that it is forbidden is that a blocking call such as sleep() will pause the current thread without giving other coroutines a chance to run. In other words, instead of pausing the current coroutine, it will pause the whole event loop, i.e. all coroutines.

In asyncio (and other async frameworks) blocking primitives like time.sleep() are replaced with awaitables like asyncio.sleep(), which suspend the awaiter and resume it when the time is right. Other coroutines and the event loop are not only unaffected by suspension of a coroutine, but that's precisely when they get the chance to run. Suspension and resumption of coroutines is the core of async-await cooperative multitasking.

Asyncio supports running legacy blocking functions in a separate thread, so that they don't block the event loop. This is achieved by calling run_in_executor which will hand off the execution to a thread pool (executor in the parlance of Python's concurrent.futures module) and return an asyncio awaitable:

async def f(x):
    loop = asyncio.get_event_loop()
    # start time.sleep(x) in a separate thread, suspend
    # the current coroutine, and resume when it's done
    await loop.run_in_executor(time.sleep, x)

This is the technique used by aiorequests to wrap request's blocking functions. Native asyncio functions like asyncio.sleep() do not use this approach; they directly tell the event loop to suspend them and how to wake them up (source).

run_in_executor is useful and effective for quick wrapping of legacy blocking code, and not much else. It is always inferior to a native async implementation, for several reasons:

• It doesn't implement cancellation. Unlike threads, asyncio tasks are fully cancelable, but this doesn't extend to run_in_executor, which shares the limitations of threads.

• It doesn't provide light-weight tasks which may number in tens of thousands and run in parallel. run_in_executor uses a thread pool under the hood, so if you await more functions than the maximum number of workers, some functions will have to wait their turn to even start working. The alternative, to increase the number of workers, will swamp the OS with too many threads. Asyncio allows the number of parallel operations to match what you'd have in a hand-written state machine using poll to listen for events.

• It is likely incompatible with more complex APIs, such as those that expose user-provided callbacks, iterators, or that provide their own thread-based async functionality.

It is recommended to avoid crutches like aiorequests and dive directly into aiohttp. The API is very similar to that of requests, and it is almost as pleasant to use.