Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is it possible to docker build a multi-staged image in parallel?

I have a rather large Dockerfile that has multiple layers. Some of these layers need quite some time. I noticed that many things do not depend on each other.

So hence the obvious question: Can I docker build in parallel?

The docker build only seems to have options that limit the build speed, instead of speeding it up (e.g. --memory).

Example: Imagine you have a Dockerfile that looks like this. I now want to call a docker build --some-flag that builds all stages in parallel unless they have to work for each other.

FROM someImage AS stage1
# do some task

FROM otherImage AS stage2
# do another taks

FROM yetAnotherImg As stage3
# more work
COPY --from=stage2 ... # stage3 has to wait for stage2 to finish

Do you know if --some-flag exists? Do you know a different way how do achieve the goal?

EDIT:

The only thing I can think about is splitting the Dockerfile up in many more stages and thus making modifications less painful, but this is not really an ideal solution

like image 711
User12547645 Avatar asked May 21 '20 13:05

User12547645


People also ask

Can Docker build in parallel?

Parallel Docker Build The average build using parallel took 128 seconds. That's a 44% drop in overall time for the build, which is pretty significant, especially when you're running this frequently while trying to troubleshoot a problem.

What is a multi stage Docker build?

A multistage build allows you to use multiple images to build a final product. In a multistage build, you have a single Dockerfile, but can define multiple images inside it to help build the final image.

Can a project have multiple Dockerfiles?

As Kingsley Uchnor said, you can have multiple Dockerfile , one per directory, which represent something you want to build.

Can a Docker container contains multiple images?

None. A Docker container is built from one image. It does not contain an image, at least not as part of any Docker process.


Video Answer


1 Answers

Thanks at m303945 for this answer!

This answer gives you an example how to build in parallel:

Dockerfile:

FROM alpine as s1
RUN sleep 10 && echo "s1 done"

FROM alpine as s2
RUN sleep 10 && echo "s2 done"

FROM alpine as s3
RUN sleep 10 && echo "s3 done"

Sequential: docker build . will take around 30 seconds.

Parallel: DOCKER_BUILDKIT=1 docker build . will take around 10 seconds.

Sources:

docker documentation

amazing blog post

like image 54
User12547645 Avatar answered Oct 24 '22 17:10

User12547645