Docker build taking too long when installing grpcio via pip

Question

I have a Dockerfile which installs a few packages via pip. Some of them are requiring grpcio, and it takes a few minutes only to build this part.

Does anyone have a tip to speed up this part?

Installing collected packages: python-dateutil, azure-common, azure-nspkg, azure-storage, jmespath, docutils, botocore, s3transfer, boto3, smmap2, gitdb2, GitPython, grpcio, protobuf, googleapis-common-protos, grpc-google-iam-v1, pytz, google-api-core, google-cloud-pubsub
Found existing installation: python-dateutil 2.7.3
  Uninstalling python-dateutil-2.7.3:
    Successfully uninstalled python-dateutil-2.7.3
Running setup.py install for grpcio: started
Running setup.py install for grpcio: still running...
Running setup.py install for grpcio: still running...
Running setup.py install for grpcio: still running...

Thanks.

Answer 1

I had the same issue and it was solved by upgrading pip:

$ pip3 install --upgrade pip

Here's a word from one of the maintainers of grpc project:

• pip grpcio install is (still) very slow #22815

Answer 2

Had the same issue, fixed it by using a virtualenv and a multistage dockerfile :

FROM python:3.7-slim as base

# ---- compile image -----------------------------------------------
FROM base AS compile-image
RUN apt-get update \
  && apt-get install -y --no-install-recommends \
  build-essential \
  gcc

RUN python -m venv /app/env
ENV PATH="/app/env/bin:$PATH"

COPY requirements.txt .
RUN pip install --upgrade pip
# pip install is fast here (while slow without the venv) :
RUN pip install -r requirements.txt

# ---- build image -----------------------------------------------
FROM base AS build-image
COPY --from=compile-image /app/env /app/env

# Make sure we use the virtualenv:
ENV PATH="/app/env/bin:$PATH"
COPY . /app
WORKDIR /app

Here is my requirements.txt :

fastapi==0.27.*
grpcio-tools==1.21.*
uvicorn==0.7.*

Answer 3

Some docker images (looking at you, Alpine) can't pull prebuilt wheels. Use a docker image that can, like Debian.

Check out this nice write-up on why. I'll reproduce a quote from it, that's especially apt:

But for Python, as Alpine doesn't use the standard tooling used for building Python extensions, when installing packages, in many cases Python (pip) won't find a precompiled installable package (a "wheel") for Alpine. And after debugging lots of strange errors you will realize that you have to install a lot of extra tooling and build a lot of dependencies just to use some of these common Python packages. 😩 -- Sebastián Ramírez