Docker Compose vs Multi-Stage Build

Question

With this new version of Docker, Multi-Stage build gets introduced, at least I'd never heard of it before. And the question I have now is, should I use it like a standard Compose file?

I used docker-compose.yaml to start containers where many images where involved, one for the web server and one for the data base. With this new multi-stage build, can I use one single Dockerfile with two FROM commands and that's it?

Will this Multi-stage build eventually kill Compose (since images are smaller)?

Answer 1

Multi-stage feature allows you to create temporary builds and extract their files to be used in your final build. For e.g. you need gcc to build your libraries but you don't need gcc in production container. Though, you could do multiple builds using few lines of bash scripting, multi-stage feature allows you to do it using a single Dockerfile. Compose only use your final image(s) regardless of how you've built it, so they are unrelated.

Answer 2

Multi-stage builds don't impact the use of docker-compose (though you may want to look into using docker stack deploy with swarm mode to use that compose on a swarm). Compose is still needed to connect multiple microservices together, e.g. running a proxy, a few applications, and an in memory cache. Compose also simplifies passing all the configuration options to a complex docker image, attaching networks and volumes, configuring restart policies, swarm constraints, etc. All of these could be done with lots of scripting, but are made easier by a simple yaml definition.

What multi-stage builds do replace is a multiple step build where you may have a build environment that should be different than a runtime environment. This is all prior to the docker-compose configuration of running your containers.

The popular example is a go binary. That binary is statically compiled so it doesn't really need anything else to run. But the build environment for it is much larger as it pulls in the compiler and various libraries. Here's an example hello.go:

package main
import "fmt"
func main() {
        fmt.Printf("Hello, world.\n")
}

And the corresponding Dockerfile:

ARG GOLANG_VER=1.8
FROM golang:${GOLANG_VER} as builder
WORKDIR /go/src/app
COPY . .
RUN go-wrapper download 
RUN go-wrapper install

FROM scratch
COPY --from=builder /go/bin/app /app
CMD ["/app"]

The two FROM lines that that Dockerfile are what make it a multi-stage build. The first FROM line creates the first stage with the go compiler. The second FROM line is also the last which makes it the default image to tag when you build. In this case, that stage is the runtime of a single binary. Other stages are all cached on the build server but don't get copied with the final image. You can target the build to different stages if you need to build a single piece with the docker build --target=builder . command.

This becomes important when you look at the result of the build:

$ docker build -t test-mult-stage .
Sending build context to Docker daemon  4.096kB
Step 1/9 : ARG GOLANG_VER=1.8
 ---> 
Step 2/9 : FROM golang:${GOLANG_VER} as builder
 ---> a0c61f0b0796
Step 3/9 : WORKDIR /go/src/app
 ---> Using cache
 ---> af5177aae437
Step 4/9 : COPY . .
 ---> Using cache
 ---> 976490d44468
Step 5/9 : RUN go-wrapper download
 ---> Using cache
 ---> e31ac3ce83c3
Step 6/9 : RUN go-wrapper install
 ---> Using cache
 ---> 2630f482fe78
Step 7/9 : FROM scratch
 ---> 
Step 8/9 : COPY --from=builder /go/bin/app /app
 ---> 96b9364cdcdc
Removing intermediate container ed558a4da820
Step 9/9 : CMD /app
 ---> Running in 55db8ed593ac
 ---> 5fd74a4d4235
Removing intermediate container 55db8ed593ac
Successfully built 5fd74a4d4235
Successfully tagged test-mult-stage:latest

$ docker images | grep 2630
<none>          <none>      2630f482fe78    5 weeks ago      700MB

$ docker images | grep test-mult-stage
test-mult-stage latest      5fd74a4d4235    33 seconds ago   1.56MB

Note the runtime image is only 1.5 MB, while the untaged builder image with the compiler is 700MB. Previously to get the same space savings you would need to compile your application outside of docker and deal with all the dependency issues that docker would normally solve for you. Or you could do the build in one container, copy the result out of that container, and use that copied file as the input to another build. The multi-stage build turns this second option into a single reproducible and portable command.