Can I mount docker host directory as copy on write/overlay?

Question

I'd like to mount a host directory in docker that on the outside is actually read/only. But I'd like it to appear inside the container as read/write.

So that files/directories can be written to but not changed on the outside. Is this possible using some kind of overlay process?

Answer 1

Edit: Check @javabrett's comment:

Upvoted despite this solution having a sunset. See answer regarding overlay-upperdir-on-overlay being disabled on 4.8 kernels and newer.

See: https://stackoverflow.com/a/50917037/644504

---

This is what I do:

On the host:

Load the directory as read only.

docker run --privileged -v /path/on/host:/path/on/client-read-only:ro -it ubuntu /bin/bash

On the client:

On the client use OverlayFS over the read-only directory mounted from the host.

mount -t overlayfs none -o lowerdir=/path/on/client-read-only,upperdir=/path/on/client /path/on/client

Then use /path/on/client to read/write the files.

Edit: if you have a 3.18+ kernel on your host, you may prefer using this on the client:

mount -t overlay overlay -o lowerdir=/path/on/client-read-only,upperdir=/path/on/client,workdir=/path/on/client-workdir /path/on/client

Which isn't overlayfs. With overlayfs I had an issue regarding being unable to use rm. overlay solved this problem for me.

Answer 2

You can do this without running privileged containers, and without any other 3rd party tools, using the local volume driver. The local volume driver will pass any options to the mount syscall, so anything you can do with mount you can do as a volume in docker. The only prerequisite is that you create the overlay directories in advance and clean them up yourself.

First, lets create the directories and some read only data:

$ mkdir -p {ro-data,upper1,upper2,upper3,work1,work2,work3}

$ ls
ro-data  upper1  upper2  upper3  work1  work2  work3

$ vi ro-data/data.txt

$ cat ro-data/data.txt
This is a data file.
It should be read-only on the host upper dir.

Next, lets create a named volume with the overlay options and run a container with it:

$ docker volume create --driver local --opt type=overlay \
  --opt o=lowerdir=${PWD}/ro-data,upperdir=${PWD}/upper1,workdir=${PWD}/work1 \
  --opt device=overlay overlay1
overlay1

$ docker container run -d --rm -v overlay1:/data --name cont1 busybox tail -f /dev/null
a6269cb6c68469aa4f57aae554c5f0823f1103715334b3719c5567abc7d55daa

Then, lets do the same with a --mount option to run, which gets slightly more complicated because of the nested comma separated strings. Escaped quotes works around that:

$ docker run -d --rm \
  --mount type=volume,dst=/data,volume-driver=local,volume-opt=type=overlay,\"volume-opt=o=lowerdir=${PWD}/ro-data,upperdir=${PWD}/upper2,workdir=${PWD}/work2\",volume-opt=device=overlay \
  --name cont2 busybox tail -f /dev/null
7329ae4ba4046782166b045611ecccb129f5e557df7eb4da95ec9063a0fe234e

Finally, let's us a compose file:

$ vi docker-compose.yml

$ cat docker-compose.yml
version: '3'

volumes:
  overlay3:
    driver: local
    driver_opts:
      type: overlay
      o: lowerdir=${PWD}/ro-data,upperdir=${PWD}/upper3,workdir=${PWD}/work3
      device: overlay

services:
  overlay3:
    image: busybox
    command: tail -f /dev/null
    container_name: cont3
    volumes:
    - overlay3:/data

$ docker-compose up -d
Creating network "vol-overlay_default" with the default driver
Creating volume "vol-overlay_overlay3" with local driver
Creating cont3 ... done

Everything is running, lets verify the data file is there:

$ docker exec cont1 ls -l /data
total 4
-rw-r--r--    1 1000     1000            67 Nov  8 16:29 data.txt

$ docker exec cont2 ls -l /data
total 4
-rw-r--r--    1 1000     1000            67 Nov  8 16:29 data.txt

$ docker exec cont3 ls -l /data
total 4
-rw-r--r--    1 1000     1000            67 Nov  8 16:29 data.txt

Next, we can make some changes to the directory in container 1, and delete the file in container 2:

$ echo "container 1 adds lines" | docker exec -i cont1 tee -a /data/data.txt
container 1 adds lines

$ echo "writing to another file" | docker exec -i cont1 tee -a /data/container1.txt
writing to another file

[11:48:30] [bmitch@bmitch-asusr556l:~/data/docker/test/vol-overlay] [master]
$ docker exec cont2 rm /data/data.txt

Verify each container see's the changes, or lack thereof:

$ docker exec cont1 ls -l /data
total 8
-rw-r--r--    1 root     root            24 Nov  8 16:48 container1.txt
-rw-r--r--    1 1000     1000            90 Nov  8 16:47 data.txt

$ docker exec cont2 ls -l /data
total 0

$ docker exec cont3 ls -l /data
total 4
-rw-r--r--    1 1000     1000            67 Nov  8 16:29 data.txt

$ docker exec cont1 cat /data/data.txt
This is a data file.
It should be read-only on the host upper dir.
container 1 adds lines

$ docker exec cont3 cat /data/data.txt
This is a data file.
It should be read-only on the host upper dir.

And show the host directory is unchanged:

$ ls -l ro-data
total 4
-rw-r--r-- 1 bmitch bmitch 67 Nov  8 11:29 data.txt

$ cat ro-data/data.txt
This is a data file.
It should be read-only on the host upper dir.

The changes were all made only to the upper directories:

$ ls -l upper*
upper1:
total 8
-rw-r--r-- 1 root   root   24 Nov  8 11:48 container1.txt
-rw-r--r-- 1 bmitch bmitch 90 Nov  8 11:47 data.txt

upper2:
total 0
c--------- 1 root root 0, 0 Nov  8 11:48 data.txt

upper3:
total 0

After removing the containers and volumes, you'll need to manually remove the upper directories. Just as docker doesn't create them for you, it doesn't delete them either, exactly the same as if you ran the mount command yourself.

Answer 3

Not an option anymore from inside the container (possibly because overlay-over-overlay is disabled in ~ 4.4 kernels)

$ uname -a && \
  docker run --privileged --rm debian:latest sh -c "mkdir upper lower work merged && mount -t overlay overlay -olowerdir=lower,upperdir=upper,workdir=work merged/; dmesg|tail -1"


Linux preprod 4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 (2018-05-07) x86_64 GNU/Linux

mount: wrong fs type, bad option, bad superblock on overlay,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

[288426.860631] overlayfs: filesystem on 'upper' not supported as upperdir

ANYWAY An alternative is to create the overlay on the host and bind it to the guest:

$ mkdir upper lower work merged && \
  touch upper/up lower/low && \
  sudo mount -t overlay overlay -olowerdir=lower,upperdir=upper,workdir=work merged/ && \
  docker run --rm -v $(pwd)/merged:/tmp/merged debian:latest sh -c "touch /tmp/merged/new-from-container"

$ ls upper/ lower/ merged/

lower/:
low

merged/:
low  new-from-container  up

upper/:
new-from-container  up