FastAPI Gunicorn Uvicorn for Production Deployment with Google Cloud Run (Stress Testing)

Question

here I want to ask to you, what's the difference between running the gunicorn uvicorn with python, and default from tiangolo?

I have tried to stress testing these using JMeter with thread properties:

From these, I got the result::

From above I have tried:

1. Dockerfile with tiangolo base
2. Dockerfile with python:3.8-slim-buster and run it with gunicorn command
3. Dockerfile with python:3.8-slim-buster and run it with python

This is my Dockerfile for case 1 (Tiangolo base):

FROM tiangolo/uvicorn-gunicorn-fastapi:python3.8-slim
RUN apt-get update && apt-get install wget gcc -y
RUN mkdir -p /app
WORKDIR /app
COPY ./requirements.txt /app/requirements.txt
RUN python -m pip install --upgrade pip
RUN pip install --no-cache-dir -r /app/requirements.txt
COPY . /app

This is my Dockerfile for case 2 (Python base with gunicorn command):

FROM python:3.8-slim-buster as builder
RUN apt-get update --fix-missing
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y libgl1-mesa-dev python3-pip git
RUN mkdir /usr/src/app
WORKDIR /usr/src/app
COPY ./requirements.txt /usr/src/app/requirements.txt
RUN pip3 install -U setuptools
RUN pip3 install --upgrade pip
RUN pip3 install -r ./requirements.txt
COPY . /usr/src/app
ENTRYPOINT gunicorn --bind :8080 --workers 1 --threads 8 main:app --worker-class uvicorn.workers.UvicornH11Worker --preload --timeout 60 --worker-tmp-dir /dev/shm

This is my Dockerfile for case 3 (Python base with python command):

FROM python:3.8-slim-buster
RUN apt-get update --fix-missing
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y libgl1-mesa-dev python3-pip git
RUN mkdir /usr/src/app
WORKDIR /usr/src/app
COPY ./requirements.txt /usr/src/app/requirements.txt
RUN pip3 install -U setuptools
RUN pip3 install --upgrade pip
RUN pip3 install -r ./requirements.txt --use-feature=2020-resolver
COPY . /usr/src/app
CMD ["python3", "/usr/src/app/main.py"]

Here I am confused, from the results above it looks like they have fairly the same results, what is the difference between the three methods above? which one is the best for production? I'm sorry, I'm new here in the production deployment API. I need some advice on this case. Thank you

This is my Cloud Run command

gcloud builds submit --tag gcr.io/gaguna3/priceengine

gcloud run deploy backend-pure-python \
    --image="gcr.io/gaguna3/priceengine" \
    --region asia-southeast2 \
    --allow-unauthenticated \
    --platform managed \
    --memory 4Gi \
    --cpu 2 \
    --timeout 900 \
    --project=gaguna3

Answer 1

It was suggested that a 3x speed up over uvicorn could be obtained by adding gunicorn e.g.:

gunicorn -w 4 -k uvicorn.workers.UvicornWorker main:app

for more details and benchmarks see: https://stackoverflow.com/a/63427961/2705777

Answer 2

Based on your examples, I noticed that the first container is using a fine tuned version of gunicorn, also in the tiangolo's github page is mentioned

This image has an "auto-tuning" mechanism included, so that you can just add your code and get that same high performance automatically.

From my perspective, this can be achieved by performing a dynamic scaling on the gunicorn workers and/or using Cpython modules.

The difference between the second and third container is the number of workers that you defined for your service, in the second container you have only 1 worker with 8 threads, if you play with your config you can improve the performance, as is mentioned on this article.

It is not a bad idea to use tiangolo/uvicorn-gunicorn but I recommend you to lock the version of the container, this is in order to prevent that a future change affects your productive environment.

On the other hand, using vanilla python images allow you to customize the image without fear of breaking something, but this requires some time to get the same performance of the tiangolo docker.