Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I write asyncio coroutines that optionally act as regular functions?

I'm writing a library that I'd like end-users to be able to optionally use as if its methods and functions were not coroutines.

For example, given this function:

@asyncio.coroutine
def blah_getter():
    return (yield from http_client.get('http://blahblahblah'))

An end user who doesn't care to use any asynchronous features in their own code, still has to import asyncio and run this:

>>> response = asyncio.get_event_loop().run_until_complete(blah_getter())

It would be cool if I could, inside of blah_getter determine if I was being called as a coroutine or not and react accordingly.

So something like:

@asyncio.coroutine
def blah_getter():
    if magically_determine_if_being_yielded_from():
        return (yield from http_client.get('http://blahblahblah'))
    else:
        el = asyncio.get_event_loop()
        return el.run_until_complete(http_client.get('http://blahblahblah'))
like image 810
Dustin Wyatt Avatar asked May 10 '15 18:05

Dustin Wyatt


People also ask

Which is used to write concurrently in Asyncio?

Using the . run_in_executor() method of an event loop will provide the necessary interoperability between the two future types by wrapping the concurrent. futures. Future type in a call to asyncio.

Which function is used to run Awaitables concurrently in Asyncio?

gather() method - It runs awaitable objects (objects which have await keyword) concurrently.

How do you write async function in Python?

To run an async function (coroutine) you have to call it using an Event Loop. Event Loops: You can think of Event Loop as functions to run asynchronous tasks and callbacks, perform network IO operations, and run subprocesses. Example 1: Event Loop example to run async Function to run a single async function: Python3.

What is coroutine Asyncio?

@coroutine is a functional analogue for async def but it works in Python 3.4+ and utilizes yield from construction instead of await . For practical perspective just never use @coroutine if your Python is 3.5+.


2 Answers

You need two functions -- asynchronous coroutine and synchronous regular function:

@asyncio.coroutine
def async_gettter():
    return (yield from http_client.get('http://example.com'))

def sync_getter()
    return asyncio.get_event_loop().run_until_complete(async_getter())

magically_determine_if_being_yielded_from() is actually event_loop.is_running() but I strongly don't recommend to mix sync and async code in the same function.

like image 178
Andrew Svetlov Avatar answered Oct 12 '22 19:10

Andrew Svetlov


I agree with Andrew's answer, I just want to add that if you're dealing with objects, rather than top-level functions, you can use a metaclass to add synchronous versions of your asynchronous methods automatically. See this example:

import asyncio
import aiohttp

class SyncAdder(type):
    """ A metaclass which adds synchronous version of coroutines.

    This metaclass finds all coroutine functions defined on a class
    and adds a synchronous version with a '_s' suffix appended to the
    original function name.

    """
    def __new__(cls, clsname, bases, dct, **kwargs):
        new_dct = {}
        for name,val in dct.items():
            # Make a sync version of all coroutine functions
            if asyncio.iscoroutinefunction(val):
                meth = cls.sync_maker(name)
                syncname = '{}_s'.format(name)
                meth.__name__ = syncname
                meth.__qualname__ = '{}.{}'.format(clsname, syncname)
                new_dct[syncname] = meth
        dct.update(new_dct)
        return super().__new__(cls, clsname, bases, dct)

    @staticmethod
    def sync_maker(func):
        def sync_func(self, *args, **kwargs):
            meth = getattr(self, func)
            return asyncio.get_event_loop().run_until_complete(meth(*args, **kwargs))
        return sync_func

class Stuff(metaclass=SyncAdder):
    @asyncio.coroutine
    def getter(self, url):
        return (yield from aiohttp.request('GET', url))

Usage:

>>> import aio, asyncio
>>> aio.Stuff.getter_s
<function Stuff.getter_s at 0x7f90459c2bf8>
>>> aio.Stuff.getter
<function Stuff.getter at 0x7f90459c2b70>
>>> s = aio.Stuff()
>>> s.getter_s('http://example.com')
<ClientResponse(http://example.com) [200 OK]>
<CIMultiDictProxy {'ACCEPT-RANGES': 'bytes', 'CACHE-CONTROL': 'max-age=604800', 'DATE': 'Mon, 11 May 2015 15:13:21 GMT', 'ETAG': '"359670651"', 'EXPIRES': 'Mon, 18 May 2015 15:13:21 GMT', 'SERVER': 'ECS (ewr/15BD)', 'X-CACHE': 'HIT', 'X-EC-CUSTOM-ERROR': '1', 'CONTENT-LENGTH': '1270', 'CONTENT-TYPE': 'text/html', 'LAST-MODIFIED': 'Fri, 09 Aug 2013 23:54:35 GMT', 'VIA': '1.1 xyz.com:80', 'CONNECTION': 'keep-alive'}>
>>> asyncio.get_event_loop().run_until_complete(s.getter('http://example.com'))
<ClientResponse(http://example.com) [200 OK]>
<CIMultiDictProxy {'ACCEPT-RANGES': 'bytes', 'CACHE-CONTROL': 'max-age=604800', 'DATE': 'Mon, 11 May 2015 15:25:09 GMT', 'ETAG': '"359670651"', 'EXPIRES': 'Mon, 18 May 2015 15:25:09 GMT', 'SERVER': 'ECS (ewr/15BD)', 'X-CACHE': 'HIT', 'X-EC-CUSTOM-ERROR': '1', 'CONTENT-LENGTH': '1270', 'CONTENT-TYPE': 'text/html', 'LAST-MODIFIED': 'Fri, 09 Aug 2013 23:54:35 GMT', 'VIA': '1.1 xys.com:80', 'CONNECTION': 'keep-alive'}>
like image 24
dano Avatar answered Oct 12 '22 18:10

dano