How to speed up Docker build

Question

I'm trying to create a Dockerfile for a project I'm working on. Installing all the required packages through apt and pip takes a couple of minutes. Since the required packages don't change, is there a way so I can skip to the steps that do change?

I'm running Docker CE on OS X (version 17.06.2-ce-mac27).

Answer 1

Yes you can. Create two images

Dockerfile-base

FROM python:3.6
RUN pip install selenium

Then build using below

docker build -f Dockerfile-base -t base .

Dockerfile

FROM base
COPY . .

So you won't rebuild base. And keep on working on the main Dockerfile. There are other possible solutions also like deploying local Nexus package manager and using it to cache packages locally. But then too much of effort for a developer machine

Answer 2

If you use the docker cache each layer of the image will only be rebuilt if it has changed or the layer above has changed.

FROM alpine:latest          # First layer
RUN apk add git gcc         # Second layer
RUN apk add another-package # Third layer

If the first or second layers are changed (say you add openssl to the second line for example) the second and third layer will be rebuilt without using the cache.
But if only the third layer is changed, only that layer will have to rebuild, while the first and second layer is built from cache.
So sometimes you can move all the stuff that are supposed to be built rarely to the top of the file, and then let stuff that rebuild often be in its own layer further down (even though more layers increases image size).

If you rather move it into multiple images, you can absolutely do as Tarun says above. If it is only data that you wish to move from one image to another (that is, not installed packages and such) you could check into multi-stage builds which allows you to define multiple images in a single file and let them copy data from the one built before in the file.

To get more information about how the build cache works, check out the docs!