Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why it says fetch function was never awaited?

I am trying to perform asynchronous requests as follows:

# Example 2: asynchronous requests

import asyncio
import aiohttp
import time
import concurrent.futures
no = int(input("time of per "))
num_requests = int(input("enter the no of threads "))
no_1 = no
avg = 0
async def fetch():
    async with aiohttp.ClientSession() as session:
        await  session.get('http://google.com')

while no > 0:
    start = time.time()
    async def main():
        with concurrent.futures.ThreadPoolExecutor(max_workers=num_requests) as executor:
            loop = asyncio.get_event_loop()
            futures = [
                loop.run_in_executor(
                    executor,
                    fetch
                )
                for i in range(num_requests)
            ]
        for response in await asyncio.gather(*futures):
            pass
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
    temp = (time.time()-start)
    print(temp)
    avg = avg + temp
    no = no - 1

print("Average is ",avg/no_1)

I get an error

RuntimeWarning: coroutine 'fetch' was never awaited
  handle = None  # Needed to break cycles when an exception occurs

Even though I am using await in fetch function. Why is it so ?

like image 573
vishal choudhary Avatar asked Jan 28 '23 16:01

vishal choudhary


1 Answers

fetch does contain an await, but no one is awaiting fetch() itself. Instead, it is called by run_in_executor, which is designed for synchronous functions. While you can certainly invoke an async function like a synchronous one, it will have no effect unless awaited by a coroutine or submitted to an event loop, and the code in the question is doing neither.

Also, it is not allowed to invoke an asyncio coroutine from a different thread, nor is it necessary to do so. If you need to run coroutines like fetch() "in parallel", submit them to the running loop with create_task() and await them en mass using gather (which you were already almost doing). For example:

async def main():
    loop = asyncio.get_event_loop()
    tasks = [loop.create_task(fetch())
             for i in range(num_requests)]
    for response in await asyncio.gather(*tasks):
        pass  # do something with response

main() can be invoked as in the question:

loop = asyncio.get_event_loop()
while no > 0:
    start = time.time()
    loop.run_until_complete(main())
    ...
    no = no - 1

But it would be somewhat more idiomatic to also create a coroutine for the timing code, and invoke loop.run_until_complete() only once:

async def avg_time():
    while no > 0:
        start = time.time()
        await main()
        ...
        no = no - 1

loop = asyncio.get_event_loop()
loop.run_until_complete(avg_time())

Finally, you probably want to create the ClientSession in main or in everything and pass the same session object to each fetch invocation. A session is typically shared between multiple requests and is not meant to be created for each individual request anew.

like image 71
user4815162342 Avatar answered Mar 07 '23 05:03

user4815162342