I've read some posts from 2013 that the Gunicorn team was planning to build a threaded buffering layer worker model, similar to how Waitress works. Is that what the gthread async worker does? The gthread workers were released with version 19.0 in 2014.
Waitress has a master async thread that buffers requests, and enqueues each request to one of its sync worker threads when the request I/O is finished.
Gunicorn gthread doesn't have much documentation, but it sounds similar. From the docs:
The worker gthread is a threaded worker. It accepts connections in the main loop, accepted connections are are added to the thread pool as a connection job.
I only ask because I am not super knowledgeable about python async I/O code, though a cursory reading of the gthread.py seems to indicate that it is a socket-buffering process that protects worker threads from long-I/O requests (and buffers the response I/O as well).
https://github.com/benoitc/gunicorn/blob/master/gunicorn/workers/gthread.py
Gunicorn is a pre-fork model server, which means that a master process is started and then a several worker processes are forked from the master. The worker processes are responsible for handling requests and returning a response to the client. Gunicorn Support multiple worker types: Sync Workers. Async Workers.
Gunicorn should only need 4-12 worker processes to handle hundreds or thousands of requests per second. Gunicorn relies on the operating system to provide all of the load balancing when handling requests. Generally we recommend (2 x $num_cores) + 1 as the number of workers to start off with.
Requests per second is not the same as "concurrent requests". If each request takes exactly 1 millisecond to handle, then a single worker can serve 1000 RPS.
Gunicorn also allows for each of the workers to have multiple threads. In this case, the Python application is loaded once per worker, and each of the threads spawned by the same worker shares the same memory space.
The threaded worker in Gunicorn does not buffer I/O and does not read the request body in the main thread.
The main loop asynchronously handles calling accept()
[1], but then the socket is immediately submitted to the thread pool[2].
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With