Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Aiohttp server max connections

I cannot understand the reason aiohttp (and asyncio in general) server implementation does not provide a way to limit max concurrent connections limit (number of accepted sockets, or number of running requests handlers). (https://github.com/aio-libs/aiohttp/issues/675). Without this limit, it is easy to run out of memory and/or file descriptors.

In the same time, aiohttp client by default limits number of concurrent requests to 100 (https://docs.aiohttp.org/en/stable/client_advanced.html#limiting-connection-pool-size), aiojobs limits number of running tasks and size of pending tasks list, nginx has worker_connections limit, any sync framework is limited by number of worker threads by design.

While aiohttp can handle a lot of concurrent requests, this number is still limited. Docs on aiojobs says "The Scheduler has implied limit for amount of concurrent jobs (100 by default). ... It prevents a program over-flooding by running a billion of jobs at the same time". And still, we can happily spawn "billion" (well, until we run out of resources) aiohttp handlers.

So the question is, why is it implemented the way it is? Am I missing some important detail? I think we can somehow pause requests handlers using Semafor, but the socket is still accepted by aiohttp and coroutine is spawned, in contrast with nginx. Also when deploying behind nginx, the number of worker_connections and aiohttp desired limit will certainly be different.(because nginx may serve static files also)

like image 967
Alexandr Tatarinov Avatar asked May 27 '19 06:05

Alexandr Tatarinov


People also ask

What is session in aiohttp?

Session is a storage for saving temporary data like logged user info. aiohttp_session library actually uses a middleware for session control. Setup session support: import aiohttp_session async def init_app() -> web.

How do I pass headers in aiohttp?

If you need to add HTTP headers to a request, pass them in a dict to the headers parameter. await session. post(url, data='Привет, Мир! ')

How does aiohttp work?

Aiohttp is an HTTP server/client for asyncio. It allows users to create asynchronous servers and clients. Also, the aiohttp package works for Client WebSockets and Server WebSockets.

What is Asyncio semaphore?

From the asyncio docs: A semaphore manages an internal counter which is decremented by each acquire() call and incremented by each release() call. The counter can never go below zero; when acquire() finds that it is zero, it blocks, waiting until some task calls release() .


2 Answers

Based on the developers' comments on the linked issue, the reasons for this choice are the following:

  • The application can return a 4xx or 5xx response if it detects that the number of connections is larger than what it can reasonably handle. (This differs from the Semaphore idiom, which would effectively queue the connection.)

  • Throttling the number of server connections is more complicated than just specifying a number, because the limit might well depend on what your coroutines are doing, i.e. it should at least be path-based. Andrew Svetlov links to NGINX documentation about connection limiting to support this.

  • It is anyway recommended to put aiohttp behind a specialized front server such as NGINX.

More detail than this can only be provided by the developer(s), who have been known to read this tag.

At this point, it appears that the recommended solution is to either use a reverse proxy for limiting, or an application-based limit like this decorator (untested):

REQUEST_LIMIT = 100

def throttle_handle(real_handle):
    _nrequests = 0
    async def handle(request):
        nonlocal _nrequests
        if _nrequests >= REQUEST_LIMIT:
            return aiohttp.web.Response(
                status=429, text="Too many connections")
        _nrequests += 1
        try:
            return await real_handle(request)
        finally:
            _nrequests -= 1
    return handle

@throttle_handle
async def handle(request):
    ... your handler here ...
like image 158
user4815162342 Avatar answered Sep 22 '22 06:09

user4815162342


To limit concurrent connections you can use aiohttp.TCPConnector or aiohttp.ProxyConnector if you using proxy. Just create it in a session instead of using the default.

aiohttp.ClientSession(
    connector=aiohttp.TCPConnector(limit=1)
)
aiohttp.ClientSession(
    connector=aiohttp.ProxyConnector.from_url(proxy_url, limit=1)
)
like image 44
Quietude Avatar answered Sep 22 '22 06:09

Quietude