asyncio aiohttp progress bar with tqdm

Question

I'm attempting to integrate a tqdm progress bar to monitor POST requests generated with aiohttp in Python 3.5. I have a working progress bar but can't seem to gather results using as_completed(). Pointers gratefully received.

Examples I've found suggest using the following pattern, which is incompatible with Python 3.5 async def definitions:

for f in tqdm.tqdm(asyncio.as_completed(tasks), total=len(coros)):     yield from f 

Working (albeit redacted) async code without the progress bar:

def async_classify(records):      async def fetch(session, name, sequence):         url = 'https://app.example.com/api/v0/search'         payload = {'sequence': str(sequence)}         async with session.post(url, data=payload) as response:             return name, await response.json()      async def loop():         auth = aiohttp.BasicAuth(api_key)         conn = aiohttp.TCPConnector(limit=100)         with aiohttp.ClientSession(auth=auth, connector=conn) as session:             tasks = [fetch(session, record.id, record.seq) for record in records]             responses = await asyncio.gather(*tasks)             return OrderedDict(responses) 

This is my unsuccessful attempt at modifying loop():

async def loop():     auth = aiohttp.BasicAuth(api_key)     conn = aiohttp.TCPConnector(limit=100)     with aiohttp.ClientSession(auth=auth, connector=conn) as session:         tasks = [fetch(session, record.id, record.seq) for record in records]         for f in tqdm.tqdm(asyncio.as_completed(tasks), total=len(tasks)):             await f         responses = await asyncio.gather(f)         print(responses) 

Answer 1

await f returns a single response. Why would you pass an already completed Future to asyncio.gather(f) is unclear.

Try:

responses = [] for f in tqdm.tqdm(asyncio.as_completed(tasks), total=len(tasks)):     responses.append(await f) 

---

Python 3.6 implements PEP 530 -- Asynchronous Comprehensions:

responses = [await f              for f in tqdm.tqdm(asyncio.as_completed(tasks), total=len(tasks))] 

It works inside async def functions now.