I have a rather large Dockerfile that has multiple layers. Some of these layers need quite some time. I noticed that many things do not depend on each other.
So hence the obvious question: Can I docker build
in parallel?
The docker build only seems to have options that limit the build speed, instead of speeding it up (e.g. --memory
).
Example: Imagine you have a Dockerfile
that looks like this. I now want to call a docker build --some-flag
that builds all stages in parallel unless they have to work for each other.
FROM someImage AS stage1
# do some task
FROM otherImage AS stage2
# do another taks
FROM yetAnotherImg As stage3
# more work
COPY --from=stage2 ... # stage3 has to wait for stage2 to finish
Do you know if --some-flag
exists? Do you know a different way how do achieve the goal?
EDIT:
The only thing I can think about is splitting the Dockerfile up in many more stages and thus making modifications less painful, but this is not really an ideal solution
Parallel Docker Build The average build using parallel took 128 seconds. That's a 44% drop in overall time for the build, which is pretty significant, especially when you're running this frequently while trying to troubleshoot a problem.
A multistage build allows you to use multiple images to build a final product. In a multistage build, you have a single Dockerfile, but can define multiple images inside it to help build the final image.
As Kingsley Uchnor said, you can have multiple Dockerfile , one per directory, which represent something you want to build.
None. A Docker container is built from one image. It does not contain an image, at least not as part of any Docker process.
Thanks at m303945 for this answer!
This answer gives you an example how to build in parallel:
Dockerfile:
FROM alpine as s1
RUN sleep 10 && echo "s1 done"
FROM alpine as s2
RUN sleep 10 && echo "s2 done"
FROM alpine as s3
RUN sleep 10 && echo "s3 done"
Sequential: docker build .
will take around 30 seconds.
Parallel: DOCKER_BUILDKIT=1 docker build .
will take around 10 seconds.
Sources:
docker documentation
amazing blog post
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With