Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is asyncio's loop.run_in_executor thread-safe?

I'm trying out asyncio and have to mix it with some normal multi threaded blocking code, so I need to offload the execution using run_in_exector.

The asyncio docs warn that "most functions" aren't threadsafe, and that call_soon_threadsafe is the only thread-safe function. There are a couple others, like Future.add_done_callback, too, that are explicitly documented as thread safe. It then has a sentence afterwards saying "you can use run_in_executor to run callbacks in other threads", but doesn't comment on the thread-safety of it specifically.

run_in_executor isn't doc'd to be thread-safe, but looking at the source, it looks like it is thread safe if the right code-paths are taken.

Does anyone know if it is supposed to be thread safe, but just isn't documented to be that way?

like image 276
Richard Levasseur Avatar asked Dec 17 '14 20:12

Richard Levasseur


People also ask

What is Loop Run_in_executor?

run_in_executor is used to manage threads from within an event loop. To this end, it needs to wrap the thread into a Future, which needs to be assigned to an event loop (in one way or another). The reason the method is stored directly in a loop object is probably historical.

Is Asyncio thread safe?

No, asyncio is not thread safe. Generally only one thread should have control over an event loop and/or a resource associated to the event loop.

What is an Asyncio event?

An asyncio event can be used to notify multiple asyncio tasks that some event has happened. An Event object manages an internal flag that can be set to true with the set() method and reset to false with the clear() method. The wait() method blocks until the flag is set to true. The flag is set to false initially.

How do you stop a loop in Python event?

Run the event loop until stop() is called. If stop() is called before run_forever() is called, the loop will poll the I/O selector once with a timeout of zero, run all callbacks scheduled in response to I/O events (and those that were already scheduled), and then exit.


2 Answers

I think it entirely depends on what you give it. It effectively just starts a thread and runs your code, so whether that is thread safe or not depends on what you tell it to do.

like image 52
leech Avatar answered Sep 19 '22 16:09

leech


run_in_executor is supposed to be not threadsafe by specification (sorry, it looks like implicit statement and probably should be clarified in PEP-3156).

Even if concrete implementation is thread safe please don't assume that any PEP-3156 compliant implementation will be thread-safe too.

like image 24
Andrew Svetlov Avatar answered Sep 19 '22 16:09

Andrew Svetlov