Docker with yarn

Question

First time trying to get yarn and docker working together. How can I stop yarn from installing the packages every single time I run docker build command?

I've found some solutions like storing node_modules in a temporary directory and then linking it, but with various packages installed I get too many errors to handle. Is there maybe a way to compare my yarn.lock with the one existing inside Docker or any other solutions?

Dockerfile:

FROM node:8.9.1-alpine

COPY package.json yarn.lock /usr/src/
RUN cd /usr/src \
    && yarn install --pure-lockfile

COPY . /usr/src
EXPOSE 3005

With this setup I get a message saying Sending build context to Docker daemon 375.2MB, then the yarn install is run as usual, fetching the packages every single time.

Answer 1

Definitely pay attention to docker caching. Basically you want to run the most stable instructions earlier than less stable ones. Instructions which would result in the same image changes as an earlier run should not be rerun (exluding issues with ENV/ARG instructions). But once an instruction does need to be run, all instructions following will be run regardless of what's in the cache.

.dockerignore will also help, but it can be easy for things to slip in. I've adopted the practice of inverting the file by ignoring everything and then specifying exactly what should be copied.

To minimise fetching from the web, I like using the yarn offline cache. This stores the tarballs of installed dependencies and reuses them for future installs. You get the benefit over npm rebuild of fresh installs every time (if the cache has been invalidated). You configure the offline cache with a .yarnrc file which could be in your home directory but for this purpose you keep one in your repo along with a directory to store the tarballs.

You have the option of checking-in the mirror directory to your repo. A typically large npm module install will still only be in the region of 20 megs.

If you use the files below, customised for your needs, and run yarn locally, the yarn-offline-mirror will contain the tarballs needed to install the app.

.dockerignore

*

!yarn-offline-mirror/
!src/
!package.json
!yarn.lock
!.yarnrc

.yarnrc

# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1

yarn-offline-mirror "./yarn-offline-mirror"

Dockerfile

ENV HOME /usr/src/
WORKDIR $HOME

# copy the tarballs
COPY ["yarn-offline-mirror", "$HOME/yarn-offline-mirror/"]

# copy files needed for the install
COPY ["package.json", "yarn.lock", ".yarnrc", "$HOME/"]

# the offline flag will mean that an error is raised if any
# module needs to be fetched remotely. It can be removed to allow
# yarn to fetch any missing modules if that was to happen.
RUN yarn --offline --frozen-lockfile --link-duplicates

# copy the rest.. could be further broken up into multiple instructions
# for cache optimisation
COPY . $HOME

CMD npm start

Answer 2

You should use the docker cache better.

If you have your Dockerfile prepared as follows:

FROM node:carbon

COPY package.json yarn.lock /app
RUN cd /app \
    && yarn install --pure-lockfile

COPY . /app
CMD doStuff

The docker build won't touch the package.json unless it has changed. The next RUN command won't execute unless the cache has been invalidated in the earlier step.

NOTE: keep the node_modules inside your .dockerignore file