How to make volumes permanent with Docker Compose v2

Question

I realize other people have had similar questions but this uses v2 compose file format and I didn't find anything for that.

I want to make a very simple test app to play around with MemSQL but I can't get volumes to not get deleted after docker-compose down. If I've understood Docker Docs right, volumes shouldn't be deleted without explicitly telling it to. Everything seems to work with docker-compose up but after going down and then up again all data gets deleted from the database.

As recommended as a good practice, I'm using separate memsqldata service as a separate data layer.

Here's my docker-compose.yml:

version: '2'
services:
    app:
        build: .
        links:
            - memsql
    memsql:
        image: memsql/quickstart
        volumes_from:
            - memsqldata
        ports:
            - "3306:3306"
            - "9000:9000"
    memsqldata:
        image: memsql/quickstart
        command: /bin/true
        volumes:
            - memsqldatavolume:/data

volumes:
    memsqldatavolume:
        driver: local

Answer 1

I realize this is an old and solved thread where the OP was pointing to a directory in the container rather than the volume they had mounted, but wanted to clear up some of the misinformation I'm seeing.

docker-compose down does not remove volumes, you need to run docker-compose down -v if you also want to delete volumes. Here's the help text straight from docker-compose (note the "by default" list):

$ docker-compose down --help
Stops containers and removes containers, networks, volumes, and images
created by `up`.

By default, the only things removed are:

- Containers for services defined in the Compose file
- Networks defined in the `networks` section of the Compose file
- The default network, if one is used

Networks and volumes defined as `external` are never removed.

Usage: down [options]

Options:
...
    -v, --volumes       Remove named volumes declared in the `volumes` section
                        of the Compose file and anonymous volumes
                        attached to containers.
...

$ docker-compose --version
docker-compose version 1.12.0, build b31ff33

Here's a sample yml with a named volume to test and a dummy command:

$ cat docker-compose.vol-named.yml
version: '2'

volumes:
  data:

services:
  test:
    image: busybox
    command: tail -f /dev/null
    volumes:
    - data:/data

$ docker-compose -f docker-compose.vol-named.yml up -d
Creating volume "test_data" with default driver
Creating test_test_1

After starting the container, the volume is initialized empty since the image is empty at that location. I created a quick hello world in that location:

$ docker exec -it test_test_1 /bin/sh
/ # ls -al /data
total 8
drwxr-xr-x    2 root     root          4096 May 23 01:24 .
drwxr-xr-x    1 root     root          4096 May 23 01:24 ..
/ # echo "hello volume" >/data/hello.txt
/ # ls -al /data
total 12
drwxr-xr-x    2 root     root          4096 May 23 01:24 .
drwxr-xr-x    1 root     root          4096 May 23 01:24 ..
-rw-r--r--    1 root     root            13 May 23 01:24 hello.txt
/ # cat /data/hello.txt
hello volume
/ # exit

The volume is visible outside of docker and is still there after a docker-compose down:

$ docker volume ls | grep test_
local               test_data

$ docker-compose -f docker-compose.vol-named.yml down
Stopping test_test_1 ... done
Removing test_test_1 ... done
Removing network test_default

$ docker volume ls | grep test_
local               test_data

Recreating the container uses the old volume with the file still visible inside:

$ docker-compose -f docker-compose.vol-named.yml up -d
Creating network "test_default" with the default driver
Creating test_test_1

$ docker exec -it test_test_1 /bin/sh
/ # cat /data/hello.txt
hello volume
/ # exit

And running a docker-compose down -v finally removes both the container and the volume:

$ docker-compose -f docker-compose.vol-named.yml down -v
Stopping test_test_1 ... done
Removing test_test_1 ... done
Removing network test_default
Removing volume test_data

$ docker volume ls | grep test_

$

If you find your data is only being persisted if you use a stop/start rather than a down/up, then your data is being stored in the container (or possibly an anonymous volume) rather than your named volume, and the container is not persistent. Make sure the location for your data inside the container is correct to avoid this.

To debug where data is being stored in your container, I'd recommend using docker diff on a container. That will show all of the files created, modified, or deleted inside that container which will be lost when the container is deleted. E.g.:

$ docker run --name test-diff busybox \
  /bin/sh -c "echo hello docker >/etc/hello.conf"

$ docker diff test-diff
C /etc
A /etc/hello.conf

Answer 2

You are using docker-compose down and if you look at the docs here

Stop containers and remove containers, networks, volumes, and images created by up. Only containers and networks are removed by default.

You are right, it should not remove volumes (by default). It may be a bug or you may have changed the default configuration. But I think the right command for you is docker-compose stop. I will try to make some tests with simplier cases for down command.

Answer 3

This was traced back to a bad documentation from MemSQL. MemSQL data path in memsql/quickstart container is /memsql and not /var/lib/memsql like in a stand-alone installation (and in MemSQL docs), and definitely not /data like somebody told me.

Answer 4

Not sure if this helps or not. When you use docker-compose up -d the container is downloaded and images created. To stop the docker images, use docker-compose down, and the images will remain and can be restarted with docker-compose start

I was using the up/down commands and kept losing my data until I tried the stop/start and now data persists.