asyncio: Wait for event from other thread

Question

I'm designing an application in Python which should access a machine to perform some (lengthy) tasks. The asyncio module seems to be a good choice for everything that is network-related, but now I need to access the serial port for one specific component. I've implemented kind of an abstraction layer for the actual serial port stuff, but can't figure out how to sensibly integrate this with asyncio.

Following setup: I have a thread running a loop, which regularly talks to the machine and decodes the responses. Using a method enqueue_query(), I can put a query string into a queue, which will then be sent to the machine by the other thread and cause a response. By passing in a threading.Event (or anything with a set() method), the caller can perform a blocking wait for the response. This can then look something like this:

f = threading.Event()
ch.enqueue_query('2 getnlimit', f)
f.wait()
print(ch.get_query_responses())

My goal is now to put those lines into a coroutine and have asyncio handle this waiting, so that the application can do something else in the meantime. How could I do this? It would probably work by wrapping the f.wait() into an Executor, but this seems to be a bit stupid, as this would create a new thread only to wait for another thread to do something.

Answer 1

The class Event_ts in Huazuo Gao's answer works well in Python up to 3.9 but not in 3.10. It is because in Python 3.10 the private attribute _loop is initially None.

The following code works in Python 3.10 as well as 3.9 and below. (I added the clear() method as well.)

import asyncio
class Event_ts(asyncio.Event):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        if self._loop is None:
            self._loop = asyncio.get_event_loop()

    def set(self):
        self._loop.call_soon_threadsafe(super().set)

    def clear(self):
        self._loop.call_soon_threadsafe(super().clear)

Answer 2

The simplest way is to do exactly what you suggested - wrap the call to f.wait() in an executor:

@asyncio.coroutine
def do_enqueue():
    f = threading.Event()
    ch.enqueue_query('2 getnlimit', f)
    yield from loop.run_in_executor(None, f.wait)
    print(ch.get_query_responses())

You do incur the overhead of starting up a thread pool (at least for the first call, the pool will stay in memory from that point forward), but any solution that provides an implementation of something like threading.Event() with thread-safe blocking and non-blocking APIs, without relying on any background threads internally, would be quite a bit more work.