Python asyncio: what satisfies `isinstance( (generator-based coroutune), ???) == True`?

Question

I'm baffled to find out neither of typing.Awaitable nor collections.abc.Awaitable cover a generator-based coroutine which is one of awaitables defined in

• https://www.python.org/dev/peps/pep-0492/#await-expression

As of Python 3.6, several asyncio APIs such as sleep() and open_connection() actually return generator-based coroutines. I usually have no problems with applying await keywords to the return values of them, but I'm going to handle a mixture of normal values and awaitables and I'd need to figure out which ones require await to yield an actual value.

So here's my question, what satisfies isinstance(c, ???) == True for an arbitrary generator-based coroutine c? I'm not insisting on using isinstance for this purpose, maybe getattr() can be a solution...

Background

I'm working on a tiny mock utility for unit testing of async function based on https://blog.miguelgrinberg.com/post/unit-testing-asyncio-code which internally has a asyncio.Queue() of mocked return values, and I want to enhance my utility so that the queue can have awaitable elements, each of which triggering await operation. My code will look like

async def case01(loop):
  f = AsyncMock()
  f.side_effect = loop, []
  await f()  # blocks forever on an empty queue

async def case02(loop):
  f = AsyncMock()
  f.side_effect = loop, ['foo']
  await f() # => 'foo'
  await f() # blocks forever

async def case03(loop):
  f = AsyncMock()
  f.side_effect = loop, [asyncio.sleep(1.0, 'bar', loop=loop)]
  await f() # yields 'bar' after 1.0 sec of delay

For usability reason, I don't want to manually wrap the return values with create_task().

I'm not sure my queue will ever legitimely contain normal, non-coroutine generators; still, the ideal solution should be able to distinguish normal generators from generator-based coroutines and skip applying await operation to the former.

Answer 1

I'm not sure what you're trying to test here, but the inspect module has functions for checking most things like this:

>>> async def f(c):
...     await c
>>> co = f()
>>> inspect.iscoroutinefunction(f)
True
>>> inspect.iscoroutine(co)
True
>>> inspect.isawaitable(co)
True

The difference between the last two is that isawaitable is true for anything you can await, not just coroutines.

If you really want to test with isinstance:

isinstance(f) is just types.FunctionType, which isn't very useful. To check whether it's a function that returns a coroutine, you need to also check its flags: f.__code__.co_flags & inspect.CO_COROUTINE (or you could hardcode 0x80 if you don't want to use inspect for some reason).

isinstance(co) is types.CoroutineType, which you could test for, but it's still probably not a great idea.

Answer 2

The documented way to detect objects that can be passed to await is with inspect.isawaitable.

According to PEP 492, await requires an awaitable object, which can be:

• A native coroutine - a Python function defined with async def;
• A generator-based coroutine - a Python generator decorated with @types.coroutine;
• Instance of a Python class that defines __await__;
• Instance of an extension type that implements tp_as_async.am_await.

isinstance(o, collections.abc.Awaitable) covers all except the 2nd one. This could be reported as a bug in Awaitable if it wasn't explicitly documented, pointing to inspect.isawaitable to check for all awaitable objects.

Note that you cannot distinguish generator-based coroutine objects from regular generator-iterators by checking the type. The two have the exact same type because the coroutine decorator doesn't wrap the given generator, it just sets a flag on its code object. The only way to check if the object is a generator-iterator produced by a generator-based coroutine is to check its code flags, which how inspect.isawaitable is implemented.

A related question is why Awaitable only checks for the existence of __await__ and not for other mechanisms that await itself uses. This is unfortunate for code that tries to use Awaitable to check the actual awaitability of an object, but it is not without precedent. A similar discrepancy exists between iterability and the the Iterable ABC:

class Foo:
  def __getitem__(self, item):
      raise IndexError

>>> iter(Foo())
<iterator object at 0x7f2af4ad38d0>
>>> list(Foo())
[]

Despite instances of Foo being iterable, isinstance(Foo(), collections.abc.Iterable) returns false.