Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does Python asyncio use a thread pool?

I wrote this bit of code:

import asyncio
import threading
from aiohttp import ClientSession

async def fetch(url):
    async with ClientSession() as session:
        async with session.get(url) as response:
            response = await response.read()
            print(threading.current_thread().name)


loop = asyncio.get_event_loop()

tasks = [asyncio.ensure_future(fetch("http://example.com")) for i in range(5)]

loop.run_until_complete(asyncio.wait(tasks))

It prints out "MainThread" every time. Does this mean that all of the requests are being executed concurrently using the same main thread and it's not using threads from a thread pool to execute each request or are the threads being abstracted?

This post seems to show that in fact there is a thread pool which Python uses for these async tasks: http://skipperkongen.dk/2016/09/09/easy-parallel-http-requests-with-python-and-asyncio/

like image 982
lorenzocastillo Avatar asked Aug 01 '17 01:08

lorenzocastillo


1 Answers

It prints out "MainThread" every time. Does this mean that all of the requests are being executed concurrently using the same main thread and it's not using threads from a thread pool to execute each request or are the threads being abstracted?

It's not using worker threads, asyncio will only use the main thread. Concurrency is achieved through cooperative multitasking by using Python generators (read about coroutines).

This post seems to show that in fact there is a thread pool ...

As well as asyncio module, the blog post you linked explicitly uses the concurrent.futures module. The ThreadPoolExecutor class from that code will spawn worker threads. But the example code in your question will not.

like image 102
wim Avatar answered Sep 23 '22 04:09

wim