Building Dockerfile fails when touching a file after a mkdir

Question

I'm new to Docker and try to build an image with a simple Dockerfile:

FROM jenkins
USER root
RUN mkdir -pv /home/a/b
RUN touch /home/a/b/test.txt
RUN mkdir -pv /var/jenkins_home/a/b
RUN touch /var/jenkins_home/a/b/test.txt
USER jenkins

When I build it, it fails with the following output:

Step 0 : FROM jenkins
Step 1 : USER root
Step 2 : RUN mkdir -pv /home/a/b
  mkdir: created directory '/home/a'
  mkdir: created directory '/home/a/b'
Step 3 : RUN touch /home/a/b/test.txt
Step 4 : RUN mkdir -pv /var/jenkins_home/a/b
  mkdir: created directory '/var/jenkins_home/a'
  mkdir: created directory '/var/jenkins_home/a/b'
Step 5 : RUN touch /var/jenkins_home/a/b/test.txt
  touch: cannot touch '/var/jenkins_home/a/b/test.txt': No such file or directory

Can anyone tell me, what I am missing here? Why does the first mkdir & touch combination work and the second does not?

Answer 1

Looking at https://registry.hub.docker.com/u/library/jenkins/, it seems that /var/jenkins_home is a volume. You can only create files there while the container is running, presumably with a volume mapping like

docker run ... -v /your/jenkins/home:/var/jenkins_home ...

The docker build process knows nothing about shared volumes.

Answer 2

This is currently investigated in docker/docker/issues/3639, and summarized in this comment:

Okay, I did little research and it seems that volume is non-mutable between Dockerfile instruction.
Here even smaller Dockerfile for testing:

FROM busybox

RUN mkdir /tmp/volume
RUN echo "hello" > /tmp/volume/hello
VOLUME ["/tmp/volume/"]
RUN [[ -f /tmp/volume/hello ]]
RUN rm /tmp/volume/hello
RUN [[ ! -e /tmp/volume/hello ]]

On each instruction we create new volume and copy content from original volume.

Update April 2019:

Use DOCKER_BUILDKIT=1
The new builder does not exhibit this behavior.

Example from dominikzalewski:

That's a very simple Dockerfile that I'm using:

FROM wordpress:latest
ARG UPLOAD_DIR=/var/www/html/wp-content/uploads

RUN mkdir -p $UPLOAD_DIR
RUN ls -lhd $UPLOAD_DIR

Cf. Build Enhancements for Docker

Docker Build enhancements for 18.09 release introduces a much-needed overhaul of the build architecture.
By integrating BuildKit, users should see an improvement on performance, storage management, feature functionality, and security.

• Docker images created with buildkit can be pushed to Docker Hub and DTR just like Docker images created with legacy build
• The Dockerfile format that works on legacy build will also work with buildkit builds
• The new --secret command line option allows the user to pass secret information for building new images with a specified Dockerfile