Running Gunicorn Flask app in Docker [CRITICAL] WORKER TIMEOUT when starting up

Question

I want to run a Flask web services app with gunicorn in Docker. Upon startup, the app loads a large machine learning model.

However, when I run gunicorn within Docker I received the following timeouts and it just keeps spawning workers.

[2019-12-12 21:52:42 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:1198) 
[2019-12-12 21:52:42 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:1204)
[2019-12-12 21:52:42 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:1210)
[2019-12-12 21:52:42 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:1211)
[2019-12-12 21:52:42 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:1222)
[2019-12-12 21:52:42 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:1223)
[2019-12-12 21:52:42 +0000] [1264] [INFO] Booting worker with pid: 1264
[2019-12-12 21:52:42 +0000] [1265] [INFO] Booting worker with pid: 1265
[2019-12-12 21:52:42 +0000] [1276] [INFO] Booting worker with pid: 1276
[2019-12-12 21:52:42 +0000] [1277] [INFO] Booting worker with pid: 1277
[2019-12-12 21:52:42 +0000] [1278] [INFO] Booting worker with pid: 1278
[2019-12-12 21:52:42 +0000] [1289] [INFO] Booting worker with pid: 1289

Running it as a flask app within Docker or running the flask app with (or without) gunicorn from the command line works fine. It also works with gunicorn if I remove the machine learning model.

For example:

$python app.py

$gunicorn -b 0.0.0.0:8080 --workers=2 --threads=4 app:app

$gunicorn app:app

Here is my Dockerfile with the Flask development server. Works fine.

ADD . /app
WORKDIR /app
RUN pip install -r requirements.txt
CMD python app.py

If I run gunicorn as follows it just keeps spawning workers:

CMD gunicorn -b 0.0.0.0:8080 --workers=2 --threads=4 app:app
or
CMD ["gunicorn", "app:app"]

gunicorn has a --timeout=30 parameter. Defaults to 30 seconds which I increased to 300. This did not appear to have an affect.

Note: I rewrote the app for the Starlette library and received the same results!

Any guidance is appreciated.

Thanks, Jay

Answer 1

I needed to add the gunicorn --timeout as follows:

CMD gunicorn --timeout 1000 --workers 1 --threads 4 --log-level debug --bind 0.0.0.0:8000 app:app

I also ran into problems deploy on Google Cloud Platform. The log only showed a kill message. Increasing the memory in the compute instance solved that problem.

Answer 2

try this CMD["gunicorn", "--timeout", "1000", "--workers=1","-b", "0.0.0.0:8000","--log-level", "debug", "manage"]