Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

`async for` in Python 3.4

There's a way to transform a Python 3.5 async for statement in a Python 3.4 code?

PEP 0492 says that async for

async for TARGET in ITER:
    BLOCK
else:
    BLOCK2

is equivalent to

iter = (ITER)
iter = type(iter).__aiter__(iter)
running = True
while running:
    try:
        TARGET = await type(iter).__anext__(iter)
    except StopAsyncIteration:
        running = False
    else:
        BLOCK
else:
    BLOCK2

but __aiter__ does not exists in Python 3.4

like image 991
Marco Sulla Avatar asked Dec 17 '15 15:12

Marco Sulla


People also ask

What is async for in Python?

asyncio is used as a foundation for multiple Python asynchronous frameworks that provide high-performance network and web-servers, database connection libraries, distributed task queues, etc. asyncio is often a perfect fit for IO-bound and high-level structured network code.

How do I run async in Python?

To run an async function (coroutine) you have to call it using an Event Loop. Event Loops: You can think of Event Loop as functions to run asynchronous tasks and callbacks, perform network IO operations, and run subprocesses. Example 1: Event Loop example to run async Function to run a single async function: Python3.

Does Python have async await?

Python's asyncio package (introduced in Python 3.4) and its two keywords, async and await , serve different purposes but come together to help you declare, build, execute, and manage asynchronous code.

Is Python asynchronous or synchronous?

Python code runs at exactly the same speed whether it is written in sync or async style. Aside from the code, there are two factors that can influence the performance of a concurrent application: context-switching and scalability.


1 Answers

No, there is not, async/await (__aiter__, etc as well) was introduced in python 3.5. On py3.4 the closest thing is asyncio.gather (if you can run all tasks at once/in parallel and wait until they are all finished) or pushing results into an asyncio.Queue (which is sequential, just as async for). Edit: see last example for an async for alternative, as described in the question.

Here is an example ala python docs for asyncio.gather:

import asyncio

@asyncio.coroutine
def task(id):
    print("task: {}".format(id))
    yield from asyncio.sleep(random.uniform(1, 3))
    return id

tasks = [
    task("A"),
    task("B"),
    task("C")
]
loop = asyncio.get_event_loop()
results = loop.run_until_complete(asyncio.gather(*tasks))
loop.close()
print(results)

Output:

task: B
task: A
task: C
['A', 'B', 'C']

Here is one for asyncio.Queue:

import asyncio

@asyncio.coroutine
def produce(queue, n):
    for x in range(n):
        print('producing {}/{}'.format(x, n))
        # todo: do something more useful than sleeping :)
        yield from asyncio.sleep(random.random())
        yield from queue.put(str(x))


@asyncio.coroutine
def consume(queue):
    while True:
        item = yield from queue.get()
        print('consuming {}...'.format(item))
        # todo: do something more useful than sleeping :)
        yield from asyncio.sleep(random.random())
        queue.task_done()


@asyncio.coroutine
def run(n):
    queue = asyncio.Queue()
    # schedule the consumer
    consumer = asyncio.ensure_future(consume(queue))
    # run the producer and wait for completion
    yield from produce(queue, n)
    # wait until the consumer has processed all items
    yield from queue.join()
    # the consumer is still awaiting for an item, cancel it
    consumer.cancel()


loop = asyncio.get_event_loop()
loop.run_until_complete(run(10))
loop.close()

Edit: async for alternative as described in the question:

import asyncio
import random

class StopAsyncIteration(Exception):
    """"""

class MyCounter:
    def __init__(self, count):
        self.count = count

    def __aiter__(self):
        return self

    @asyncio.coroutine
    def __anext__(self):
        if not self.count:
            raise StopAsyncIteration

        return (yield from self.do_something())

    @asyncio.coroutine
    def do_something(self):
        yield from asyncio.sleep(random.uniform(0, 1))
        self.count -= 1
        return self.count

@asyncio.coroutine
def getNumbers():
    i = MyCounter(10).__aiter__()
    while True:
        try:
            row = yield from i.__anext__()
        except StopAsyncIteration:
            break
        else:
            print(row)

loop = asyncio.get_event_loop()
loop.run_until_complete(getNumbers())
loop.close()

Note that this can be simplified by removing both __aiter__ and __anext__ and raising a stop exception within the do_something method itself or return a sentinel result when done (usually an invalid value like: None, "", -1, etc)

like image 64
nitely Avatar answered Oct 03 '22 23:10

nitely