Building and deploying C++ through docker multistage build vs mount

Question

I am trying to build a custom C++ tool using docker that has a number of custom library dependencies.

There are three libraries: libA, libB and libC. libB depends on libA, and libC depends on libA and libB. On my home system, I would usually update the source for libA and then make install everything downstream from it, as libB uses CMake to find libA etc.

What is the best way to achieve this in docker? Is it also to use make install? The final image I don't wish to have the source code in - just binaries and libraries. If I use a multistage build, how do I know I have copied over all the necessary other libraries e.g. curl, protobuf into the final 'layer'. Would it not be better to just mount the source for the build image and then not for the deployment image?

I find there is not a lot of information for docker and C++.

Answer 1

You do not typically want to use volumes for your application code or dependencies. A best practice is for a Docker image to be self-contained. You might run across Docker setups for interpreted languages (especially Node) that make heavy use of volumes and bind mounts, but these really just boil down to running Node on host code via Docker instead of installing Node on the host.

A typical approach for something that needs to be compiled, but only needs the compiled artifacts to run, is to use a Docker multi-stage build. The idea is that you set up an image containing a full build toolchain and compile everything, and then create a second image in the same Dockerfile that COPY --from the first image only the compiled artifacts.

I might try using the standard /usr/local directory tree here, for two reasons: it's often the default installation prefix for tools like Autoconf, and it's empty in the standard Linux distribution Docker images.

# The first stage only builds things.  We'll COPY files out of it
# later.
FROM ubuntu:20.04 AS builder

# Install any dependencies we need here, like the C toolchain.
RUN apt-get update \
 && DEBIAN_FRONTEND=noninteractive \
    apt-get install --assume-yes --no-install-recommends \
      build-essential \
      libbar-dev \
      libfoo-dev \
      libquux-dev

# Build library A and install it into /usr/local.
WORKDIR /usr/src/libA
COPY libA ./
RUN ./configure \
 && make \
 && make install

# Similarly with library B
WORKDIR /usr/src/libB
COPY libB ./
RUN ./configure \
 && make \
 && make install

# ...and library C, and the application proper
...

# The final FROM line is the actual runtime image.  Anything before
# this will be built, and will show up in `docker images`, but isn't
# "the output" of the build.

FROM ubuntu:20.04

# Get the installed libraries and applications from the earlier stage.
COPY --from=builder /usr/local/ /usr/local/

# Regenerate the shared-library cache.
RUN ldconfig

# Set ordinary metadata to run a Docker container.  (The binary is
# probably in /usr/local/bin which is on the default PATH.)
EXPOSE 80
CMD ["myapp"]

If the different libraries have very different build-time dependencies, it's possible to extend this approach to have more build stages. Remember that each FROM line starts anew, and you need to RUN apt-get install or COPY things into each image stage for them to be present.

One open complication here is that C library builds will often include some parts that are only needed for additional C library builds. In your example libB depends on libA, and so the libA installation step will include C header files and often a static library. These can be large (especially the .a static library) and aren't needed at run time. You might want to weed these out, maybe by strategic use of RUN rm at the end of the build stage. In general RUN rm doesn't make an image smaller, but if you RUN rm something in a build stage and then COPY the redacted files into the final stage, only the things that are left behind get copied into the final image.