How am I supposed to access a named volume from the host?

Question

I'm new to the Docker world and I'm trying to achieve something one could think is trivial. However, it seems a lot of beginners struggle to persist their data when using Docker.

I've built a custom image using a Dockerfile. The container runs a MySQL server and... yes, you guessed it: I'd like to persist the data.

Here is my Dockerfile:

FROM debian:8.7

ENV MYSQL_ROOT_PASSWORD=test

RUN apt-get update -y && apt-get install -y apt-utils && \
    echo "mysql-server mysql-server/root_password password $MYSQL_ROOT_PASSWORD" | debconf-set-selections && \
    echo "mysql-server mysql-server/root_password_again password $MYSQL_ROOT_PASSWORD" | debconf-set-selections && \
    apt-get install -y mysql-server mysql-client && service mysql start

CMD service mysql start && /bin/bash

VOLUME /var/lib/mysql

EXPOSE 3306

I build and run the image this way:

docker build -t mysql-persist-test:0.1 .
docker run -dt -v database_volume:/var/lib/mysql mysql-persist-test:0.1

So far, everything works as expected, including the database.

However, let's say I want to retrieve the data on my host machine (Windows 10, I installed Docker via the Docker Toolbox).

I "bind" a local folder to the named volume with Kitematic (see below), the container automatically restarts and... everything is broken! All the files in the /var/lib/mysql directory were removed. Some were re-created with the owner staff instead of mysql.

Then I have these errors in /var/log/mysql/error.log:

...
/usr/sbin/mysqld: Table 'mysql.plugin' doesn't exist
170328 16:03:13 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
...
170328 16:03:13  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
...
170328 16:03:13  InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
...
170328 16:03:14 [ERROR] Fatal error: Can't open and lock privilege tables: Table 'mysql.host' doesn't exist

What am I doing wrong?

Answer 1

The volume hierarchy

Data in containers is in sort of a hierarchy. It goes like this.

1. part of the image

This is the lowest level, where data is in the read-only, immutable image itself.

2. in the read-write layer

Once you start a container from an image, a read-write layer is added on top of the existing image layers. If anything is changed, added, or removed in the container, by default it is written here.

Changes in this layer over-ride data in layer 1.

3. Docker volumes

In your example, you have created a volume in Docker, with

VOLUME /var/lib/mysql

This will create a volume within Docker, that can be re-used, persisted, shared among containers, etc. If there was anything at /var/lib/mysql in layer 1, then the contents of this volume override. If you make changes in the container, they are made in the volume (skipping over layer 2).

4. External volumes

Finally, we have external directories that you can mount inside the container. This overrides all the others.

Since they are based on an external directory, any changes made in the container will be easily accessed from outside. That's presumably why you tried this approach.

What happened to you

You started with a Docker volume (level 3) and then changed to an external volume (level 4). Since level 4 overrides level 3, what happens is the contents of your external directory (probably no contents), override the Docker volume. Therefore the container just sees an empty directory.

Your files are still there. Just undo the external mount and go back to the Docker volume; they'll be waiting there.

How to get your files out

EDIT: as Carlos points out in comments, docker cp is simpler, editing to use that approach instead.

docker cp <container-id>:/var/lib/mysql ./mydata

This will copy the contents of /var/lib/mysql into the folder mydata.