Building Dockerfile - Retaining files between intermediate containers

Question

I am building an extremely basic container using keymetrics official image docker hub link here

However this is not a node.js or pm2 question it's very much a Docker question, particularly docker build

A step in the Dockerfile (npm install) pulls down node packages into a node_modules folder -- IT WORKS. The folder is created and the files download and are shown in the build output (below).

The problem is that after that RUN command completes, the intermediate container is deleted and I lose the node_modules folder.

My dockerfile has a few ls -l commands to clarify what's going on with the files.

The only files that are retained in the working directory (/app) are the files I copied from host with the COPY command

Here is my Dockerfile:

FROM keymetrics/pm2:latest

# Bundle APP files
COPY src src/
COPY package.json .
COPY pm2.json .

# Install app dependencies
ENV NPM_CONFIG_LOGLEVEL info
RUN pwd && ls -l /app && npm install && ls -l

# Show current folder structure in logs
RUN ls -l

CMD [ "pm2-docker", "start", "pm2.json" ]

Build Command: docker build -t localapps/pm2_test_app:0.0.4 .

Build Output

Sending build context to Docker daemon  13.31kB
Step 1/8 : FROM keymetrics/pm2:latest
 ---> 6aa333f957ec
Step 2/8 : COPY src src/
 ---> Using cache
 ---> 6b73b4463af5
Step 3/8 : COPY package.json .
 ---> Using cache
 ---> d27a2e75fdde
Step 4/8 : COPY pm2.json .
 ---> Using cache
 ---> 9864d9dd73a9
Step 5/8 : ENV NPM_CONFIG_LOGLEVEL info
 ---> Using cache
 ---> 9f711fe6bada
Step 6/8 : RUN pwd && ls -l /app && npm install && ls -l
 ---> Running in 668eb2e2c1e8
/app
total 12
-rw-r--r--    1 root     root           323 Nov  8 17:33 package.json
-rw-r--r--    1 root     root           123 Nov  8 16:59 pm2.json
drwxr-xr-x    2 root     root          4096 Nov  8 18:10 src
npm info it worked if it ends with ok
npm info using [email protected]
npm info using [email protected]
... <omitting useless npm logs> ...

added 8 packages in 2.001s
npm info ok
total 20
drwxr-xr-x   10 root     root          4096 Nov  8 19:49 node_modules
-rw-r--r--    1 root     root          1833 Nov  8 19:49 package-lock.json
-rw-r--r--    1 root     root           323 Nov  8 17:33 package.json
-rw-r--r--    1 root     root           123 Nov  8 16:59 pm2.json
drwxr-xr-x    2 root     root          4096 Nov  8 18:10 src
 ---> 0d749171a431
Removing intermediate container 668eb2e2c1e8
Step 7/8 : RUN ls -l
 ---> Running in fc2132121c96
total 12
-rw-r--r--    1 root     root           323 Nov  8 17:33 package.json
-rw-r--r--    1 root     root           123 Nov  8 16:59 pm2.json
drwxr-xr-x    2 root     root          4096 Nov  8 18:10 src
 ---> c689180648c9
Removing intermediate container fc2132121c96
Step 8/8 : CMD pm2-docker start pm2.json
 ---> Running in e70b2b89b3c8
 ---> 299c26f883e8
Removing intermediate container e70b2b89b3c8
Successfully built 299c26f883e8
Successfully tagged localapps/pm2_test_app:0.0.4

^^ Notice the node_modules directory exists and is good to go after step 6 is finished?!

From the build output it's proven that the npm install ran and pulled down the packages to the appropriate directory....

But on step 7 it's gone?! Furthermore on the final image it's gone. App doesn't work because the dependencies in node_modules is clobbered in the final image.

Why? Without the node_modules directory the application won't run, they are dependencies.

The only way I can get this working is to run npm_install outside of the build, and then copy the folder in using a COPY which is absolutely not a valid solution... but it's a proven workaround.

I understand it uses intermediate containers which makes sense, but how can I make files created within a RUN command retain until the end?

EDIT: Further investigation is showing that the source image dockerfile found here declares a VOLUME in its dockerfile -- is this where my problem comes from? Their examples show that just doing an npm install works out of the box, but it doesn't since it wipes out the files that the install downloaded.

Answer 1

The keymetrics/pm2 image contains the following line in their Dockerfile:

VOLUME ["/app"]

That breaks any later steps and child images from modifying the /app directory. From docker's documentation:

Changing the volume from within the Dockerfile: If any build steps change the data within the volume after it has been declared, those changes will be discarded.

You'll need to run your build in a different directory, make a different base image without that line in it, or convince the upstream author to remove that line.

I've blogged about the issues with volumes defined in Dockerfiles before, and this is one of those issues. There's really no upside to having this line in their Dockerfile.