Menu
I have a Dockerfile which is split into a two-stage multi-stage docker build. The first stage generates a basic gcc build environment in which a number of C and C++ library are compiled. The second stage uses the COPY --from= command to copy the library files from the first stages /usr/local/lib/libproto* to the current image's.
The problem I am seeing is that the first image contains symlinks from a generic library file name to a specific versioned file name. AFAIK this is common practice within Debian and many other Linux systems. Docker's COPY command does not seem to understand symlinks so instead makes two complete copies of the library files. This results in a larger Docker Image size and warnings from later apt-get commands to the tune of ldconfig: /usr/local/lib/libprotobuf.so.17 is not a symbolic link.
My specific file presently looks like:
#Compile any tools we cannot install from packages
FROM gcc:7 as builder
USER 0
RUN \
apt-get -y update && \
apt-get -y install \
clang \
libc++-dev \
libgflags-dev \
libgtest-dev
RUN \
# Protocol Buffer & gRPC
# install protobuf first, then grpc
git clone -b $(curl -L https://grpc.io/release) \
https://github.com/grpc/grpc /var/local/git/grpc && \
cd /var/local/git/grpc && \
git submodule update --init && \
echo "--- installing protobuf ---" && \
cd third_party/protobuf && \
./autogen.sh && ./configure --enable-shared && \
make -j$(nproc) && make install && make clean && ldconfig && \
echo "--- installing grpc ---" && \
cd /var/local/git/grpc && \
make -j$(nproc) && make install && make clean && ldconfig
FROM debian
LABEL \
Description="Basic Debian production environment with a number of libraries configured" \
MAINTAINER="Mr Me"
ARG prefix=/usr/local
ARG binPath=$prefix/bin
ARG libPath=$prefix/lib
# Copy over pre-made tools
# Protocol Buffer
COPY --from=builder /usr/local/lib/libproto* $libPath/
# gRPC
COPY --from=builder /usr/local/lib/libaddress_sorting.so.6.0.0 $libPath/
COPY --from=builder /usr/local/lib/libgpr* $libPath/
COPY --from=builder /usr/local/lib/libgrpc* $libPath/
RUN ldconfig
# Install remaining tools using apt-get
RUN apt-get -y update && \
apt-get -y install \
libhdf5-dev \
libssl1.1 \
uuid-dev;
As you can see I am trying to add the latest versions of gRPC and Protocol Buffer to a Debian based runtime image.
489
With multi-stage builds, you use multiple FROM statements in your Dockerfile. Each FROM instruction can use a different base, and each of them begins a new stage of the build. You can selectively copy artifacts from one stage to another, leaving behind everything you don't want in the final image.
Docker ADD Command The command copies files/directories to a file system of the specified container. It includes the source you want to copy ( <src> ) followed by the destination where you want to store it ( <dest> ). If the source is a directory, ADD copies everything inside of it (including file system metadata).
One approach to keeping Docker images small is using multistage builds. A multistage build allows you to use multiple images to build a final product. In a multistage build, you have a single Dockerfile, but can define multiple images inside it to help build the final image.
In fact, they should not be. You don't need testing dependencies or direct access. They should be as similar as possible, differing only in the environment variables provided to each. Automated tests are executed on staging, but they should not require the setup to be different from production.
Use multi-stage builds. With multi-stage builds, you use multiple FROM statements in your Dockerfile. Each FROM instruction can use a different base, and each of them begins a new stage of the build. You can selectively copy artifacts from one stage to another, leaving behind everything you don’t want in the final image.
When using multi-stage builds, you are not limited to copying from stages you created earlier in your Dockerfile. You can use the COPY --from instruction to copy from a separate image, either using the local image name, a tag available locally or on a Docker registry, or a tag ID.
And indeed, most other information I can gather regarding COPY (except the examples in the above link) rather suggest that COPY is actually meant to copy files from the directory where the docker build command was launched, not from within the build environment.
Acknowledgment : Special thanks to Alex Ellis for granting permission to use his blog post Builder pattern vs. Multi-stage builds in Docker as the basis of the examples below. One of the most challenging things about building images is keeping the image size down.
This is more of a workaround than an answer.
You could tar the files, copy the tarball to the second container and then untar them.
Tar maintains symbolic links by default.
84
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
