Large Docker context slowing down docker-compose build

Question

I have two images that use a two-stage build to build Scala code and copy the artifacts to a final image. To speed up the build, I copy my local ~/.ivy2 to the context directory and from there to the images (~1GB). Unfortunately this means that even when nothing has changed and the images don't need to be re-built, docker-compose build (or docker build) hangs for quite a while to copy Docker context. This happens twice of course, once for each image.

Is there any cleverer way to do this?

Dockerfile:

FROM openjdk:8

RUN apt-get update &&\
    apt-get install -y apt-transport-https gnupg2 &&\
    echo "deb https://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list &&\
    apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 642AC823 &&\
    apt-get update &&\
    apt-get install -y sbt=1.1.6

COPY ivy-cache/ /root/.ivy2

COPY app/source/ /app/source

RUN cd /app/source &&\
    sbt assembly &&\
    cp target/scala-2.11/my-app-*.jar /app/my-app.jar

FROM gettyimages/spark:2.3.1-hadoop-3.0

COPY --from=0 /app/my-app.jar /app/my-app.jar

CMD ["spark-submit", "--master", "local", "/app/my-app.jar"]

Answer 1

With 18.09, docker includes BuildKit. By itself, BuildKit will cache the previous context and only send over the differences with the equivalent of rsync in the background.

For this specific case, you can use some experimental features to mount in your dependency cache as the equivalent of a named volume using the RUN --mount syntax. The cache directory never makes it into the image, but is there for later builds, and when you pull in a new dependency it will behave just like a local build, downloading only the new dependencies.

# syntax=docker/dockerfile:experimental
FROM openjdk:8 as build

RUN apt-get update &&\
    apt-get install -y apt-transport-https gnupg2 &&\
    echo "deb https://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list &&\
    apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 642AC823 &&\
    apt-get update &&\
    apt-get install -y sbt=1.1.6

COPY app/source/ /app/source

RUN --mount=type=cache,target=/root/.ivy2 \
    cd /app/source &&\
    sbt assembly &&\
    cp target/scala-2.11/my-app-*.jar /app/my-app.jar

FROM gettyimages/spark:2.3.1-hadoop-3.0 as release

COPY --from=build /app/my-app.jar /app/my-app.jar

CMD ["spark-submit", "--master", "local", "/app/my-app.jar"]

To use BuildKit under 18.09, you can either export an environment variable:

export DOCKER_BUILDKIT=1

or update the engine with the new default in /etc/docker/daemon.json:

{ "features": {"buildkit": true} }