Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Converting from generator-based to native coroutines

I've used Python for many years but only gradually studied the more obscure features of the language, as most of my code is for data processing. Generators based on yield are part of my routine toolkit, and recently I read about coroutines. I found an example similar to this:

def averager():
    sum = 0.0
    n = 0
    while True:
        value = yield
        sum += value
        n += 1
        print(sum/n)

avg = averager()
next(avg) # prime the coroutine
avg.send(3)
avg.send(4)
avg.send(5)

which prints the average of the values sent to it. I figured something like this might come in useful in data processing pipelines so I resolved to keep it in the back of my head. That is, until I read the following notice in the Python documentation:

Support for generator-based coroutines is deprecated and is scheduled for removal in Python 3.10.

Obviously I'd like to write future-proof code so at this point it's probably useless to start learning generator-based coroutines. My question, then, is: How to implement this example using the native (asyncio) coroutines? I have a much harder time wrapping my head around the native coroutine syntax.

While trying to search for an answer, I found a related question which has a comment and an answer that are basically saying "you can't do it with async, do it with yield-based coroutines instead". But if those are going away, is there going to be any way to do this with coroutines in 3.10+?

like image 289
Jsl Avatar asked Nov 01 '19 11:11

Jsl


People also ask

What differentiates a generator from a coroutine?

Generators produce data for iteration while coroutines can also consume data. whatever value we send to coroutine is captured and returned by (yield) expression. A value can be sent to the coroutine by send() method.

Are coroutines generators?

Coroutines are Generators, but their yield accepts values. Coroutines can pause and resume execution (great for concurrency).

What causes a Python function definition to become a generator or Coroutine?

It is fairly simple to create a generator in Python. It is as easy as defining a normal function, but with a yield statement instead of a return statement. If a function contains at least one yield statement (it may contain other yield or return statements), it becomes a generator function.

Are Python generators asynchronous?

Asynchronous generator functions are part of Python version 3.6, they were introduced by PEP-525. Asynchronous generator functions are much like regular asynchronous functions except that they contain the yield keyword in the function body.


1 Answers

And there come Asynchronous Generators ...

So we still have that power in asynchronous context.
As for theory - the mentioned PEP 525 provides a great description and definitely worth reading.
I'll just post a prepared illustrative example (for asynchronous averager) with initialization and safe finalization included:

import asyncio

async def coro():
    print('running other coroutine in 3 sec ...')
    await asyncio.sleep(3)  # emulate working


async def averager():
    sum_ = n = 0
    while True:
        v = yield
        sum_ += v
        n += 1
        print(sum_ / n)
        await asyncio.sleep(0.1)


async def main():
    agen = averager()
    await agen.asend(None)
    print(agen.__name__, 'initialized ...')

    await agen.asend(3)
    print('another separate processing here ...')

    await coro()

    await agen.asend(4)
    await agen.asend(14)


loop = asyncio.get_event_loop()
try:
    loop.run_until_complete(main())
finally:
    loop.run_until_complete(loop.shutdown_asyncgens())
    loop.close()

Program output:

averager initialized ...
3.0
another separate processing here ...
running other coroutine in 3 sec ...
3.5
7.0
like image 82
RomanPerekhrest Avatar answered Oct 02 '22 19:10

RomanPerekhrest