Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Maven docker cache dependencies

I'm trying to use docker to automate maven builds. The project I want to build takes nearly 20 minutes to download all the dependencies, so I tried to build a docker image that would cache these dependencies, but it doesn't seem to save it. My Dockerfile is

FROM maven:alpine
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
ADD pom.xml /usr/src/app
RUN mvn dependency:go-offline

The image builds, and it does download everything. However, the resulting image is the same size as the base maven:alpine image, so it doesn't seem to have cached the dependencies in the image. When I try to use the image to mvn compile it goes through the full 20 minutes of redownloading everything.

Is it possible to build a maven image that caches my dependencies so they don't have to download everytime I use the image to perform a build?

I'm running the following commands:

docker build -t my-maven .

docker run -it --rm --name my-maven-project -v "$PWD":/usr/src/mymaven -w /usr/src/mymaven my-maven mvn compile

My understanding is that whatever RUN does during the docker build process becomes part of the resulting image.

like image 490
Daniel Watrous Avatar asked Feb 13 '17 16:02

Daniel Watrous


People also ask

Does Maven cache dependencies?

When a Maven build is executed, Maven automatically downloads all the dependency jars into the local repository. Usually this folder is named . m2/repository . To clear/delete your local maven repository cache, simply delete the .

Where is Docker cache stored?

The cache uses the same storage driver as used for image layers. Metadata is stored in databases at /var/lib/docker/buildkit . When using overlay2 driver the layer itself is in /var/lib/docker/overlay2/<ID>/diff/ . For <ID> , see below. /var/lib/docker can vary depending on data-root in your dockerd configuration.

Does Maven have a cache?

At very simple form, the build cache Maven is essentially a hash function which takes Maven project and produces cache key for a project.

Does Docker have a cache?

Whether it's for the next release of your software, or locally during development. Because building images is a common task, Docker provides several tools that speed up builds. The most important feature for improving build speeds is Docker's build cache.


2 Answers

Usually, there's no change in pom.xml file but just some other source code changes when you're attempting to start docker image build. In such circumstance you can do this:

FYI:

FROM maven:3-jdk-8

ENV HOME=/home/usr/app

RUN mkdir -p $HOME

WORKDIR $HOME

# 1. add pom.xml only here

ADD pom.xml $HOME

# 2. start downloading dependencies

RUN ["/usr/local/bin/mvn-entrypoint.sh", "mvn", "verify", "clean", "--fail-never"]

# 3. add all source code and start compiling

ADD . $HOME

RUN ["mvn", "package"]

EXPOSE 8005

CMD ["java", "-jar", "./target/dist.jar"]

So the key is:

  1. add pom.xml file.

  2. then mvn verify --fail-never it, it will download maven dependencies.

  3. add all your source file then, and start your compilation(mvn package).

When there are changes in your pom.xml file or you are running this script for the first time, docker will do 1 -> 2 -> 3. When there are no changes in pom.xml file, docker will skip step 1、2 and do 3 directly.

This simple trick can be used in many other package management circumstances(gradle、yarn、npm、pip).

Edit:

You should also consider using mvn dependency:resolve or mvn dependency:go-offline accordingly as other comments & answers suggest.

like image 115
Kim Avatar answered Nov 08 '22 12:11

Kim


Using BuildKit

From Docker v18.03 onwards you can use BuildKit instead of volumes that were mentioned in the other answers. It allows mounting caches that can persist between builds and you can avoid downloading contents of the corresponding .m2/repository every time.

Assuming that the Dockerfile is in the root of your project:

# syntax = docker/dockerfile:1.0-experimental

FROM maven:3.6.0-jdk-11-slim AS build
COPY . /home/build
RUN mkdir /home/.m2
WORKDIR /home/.m2
USER root
RUN --mount=type=cache,target=/root/.m2 mvn -f /home/build/pom.xml clean compile

target=/root/.m2 mounts cache to the specified place in maven image Dockerfile docs.

For building you can run the following command:

DOCKER_BUILDKIT=1 docker build --rm --no-cache  .   

More info on BuildKit can be found here.

like image 14
Farzad Vertigo Avatar answered Nov 08 '22 11:11

Farzad Vertigo