I am trying to perform asynchronous requests
as follows:
# Example 2: asynchronous requests
import asyncio
import aiohttp
import time
import concurrent.futures
no = int(input("time of per "))
num_requests = int(input("enter the no of threads "))
no_1 = no
avg = 0
async def fetch():
async with aiohttp.ClientSession() as session:
await session.get('http://google.com')
while no > 0:
start = time.time()
async def main():
with concurrent.futures.ThreadPoolExecutor(max_workers=num_requests) as executor:
loop = asyncio.get_event_loop()
futures = [
loop.run_in_executor(
executor,
fetch
)
for i in range(num_requests)
]
for response in await asyncio.gather(*futures):
pass
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
temp = (time.time()-start)
print(temp)
avg = avg + temp
no = no - 1
print("Average is ",avg/no_1)
I get an error
RuntimeWarning: coroutine 'fetch' was never awaited
handle = None # Needed to break cycles when an exception occurs
Even though I am using await
in fetch
function. Why is it so ?
fetch
does contain an await
, but no one is awaiting fetch()
itself. Instead, it is called by run_in_executor
, which is designed for synchronous functions. While you can certainly invoke an async function like a synchronous one, it will have no effect unless awaited by a coroutine or submitted to an event loop, and the code in the question is doing neither.
Also, it is not allowed to invoke an asyncio coroutine from a different thread, nor is it necessary to do so. If you need to run coroutines like fetch()
"in parallel", submit them to the running loop with create_task()
and await them en mass using gather
(which you were already almost doing). For example:
async def main():
loop = asyncio.get_event_loop()
tasks = [loop.create_task(fetch())
for i in range(num_requests)]
for response in await asyncio.gather(*tasks):
pass # do something with response
main()
can be invoked as in the question:
loop = asyncio.get_event_loop()
while no > 0:
start = time.time()
loop.run_until_complete(main())
...
no = no - 1
But it would be somewhat more idiomatic to also create a coroutine for the timing code, and invoke loop.run_until_complete()
only once:
async def avg_time():
while no > 0:
start = time.time()
await main()
...
no = no - 1
loop = asyncio.get_event_loop()
loop.run_until_complete(avg_time())
Finally, you probably want to create the ClientSession
in main
or in everything
and pass the same session object to each fetch
invocation. A session is typically shared between multiple requests and is not meant to be created for each individual request anew.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With