Reducing Flask/Gunicorn request queue

Question

I have a Flask/Gunicorn endpoint which takes a few seconds to return and gets hit pretty hard. Gunicorn seems to queue up a lot of requests and eventually process them all, but the requests which happen to be added at the back of the queue end up taking a really long time.

The app runs as gunicorn -w 4 -b :8080.

Is there any way to configure Flask/Gunicorn so that it will only keep X requests in the queue?

Answer 1

Probably you've figured it out by now, but since I ran into this, I may as well answer it.

The "request queue" you mention, is called "backlog" and based on the docs you can change it by passing in the --backlog argument to your command, so it would look like:

gunicorn -w 4 --backlog 1024 -b :8080

Keep in mind though, that if the number of requests exceed the number you put there, your clients will start seeing errors.