Share volumes between separate docker compose files

Question

I am trying to allow nginx to proxy between multiple containers while also accessing the static files from those containers.

To share volumes between containers created using docker compose, the following works correctly:

version: '3.6'

services:
  web:
    build:
      context: .
      dockerfile: ./Dockerfile
    image: webtest
    command: ./start.sh
    volumes:
      - .:/code
      - static-files:/static/teststaticfiles

  nginx:
    image: nginx:1.15.8-alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx-config:/etc/nginx/conf.d
      - static-files:/static/teststaticfiles
    depends_on:
      - web

volumes:
  static-files:

However what I actually require is for the nginx compose file to be in a separate file and also in a completely different folder. In other words, the docker compose up commands would be run separately. I have tried the following:

First compose file:

version: '3.6'

services:
  web:
    build:
      context: .
      dockerfile: ./Dockerfile
    image: webtest
    command: ./start.sh
    volumes:
      - .:/code
      - static-files:/static/teststaticfiles
    networks:
      - directorylocation-nginx_mynetwork

volumes:
  static-files:

networks:
  directorylocation-nginx_mynetwork:
    external: true

Second compose file (ie: nginx):

version: '3.6'

services:
  nginx:
    image: nginx:1.15.8-alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx-config:/etc/nginx/conf.d
      - static-files:/static/teststaticfiles
    networks:
      - mynetwork

volumes:
  static-files:

networks:
  mynetwork:

The above two files work correctly in the sense that the site can be viewed. The problem is that the static files are not available in the nginx container. The site therefore displays without any images etc.

One work around which works correctly found here is to change the nginx container static files volume to instead be as follows:

- /var/lib/docker/volumes/directory_static-files/_data:/static/teststaticfiles

The above works correctly, but it seems 'hacky' and brittle. Is there another way to share volumes between containers which are housed in different compose files without needing to map the /var/lib/docker/volumes directory.

Answer 1

By separating the 2 docker-compose.yml files as you did in your question, 2 different volumes are actually created; that's the reason you don't see data from web service inside volume of nginx service, because there are just 2 different volumes.

Example : let's say you have the following structure :

example/
    |- web/
        |- docker-compose.yml # your first docker compose file
    |- nginx/
        |- docker-compose.yml # your second docker compose file

Running docker-compose up from web folder (or docker-compose -f web/docker-compose.yml up from example directory) will actually create a volume named web_static-files (name of the volume defined in docker-compose.yml file, prefixed by the folder where this file is located).

So, running docker-compose up from nginx folder will actually create nginx_static-files instead of re-using web_static-files as you want.

You can use the volume created by web/docker-compose.yml by specifying in the 2nd docker compose file (nginx/docker-compose.yml) that this is an external volume, and its name :

volumes:
  static-files:
    external:
      name: web_static-files

Note that if you don't want the volume (and all resources) to be prefixed by the folder name (default), but by something else, you can add -p option to docker-compose command :

docker-compose \
    -f web/docker-compose.yml \
    -p abcd \
    up

This command will now create a volume named abcd_static-files (that you can use in the 2nd docker compose file).

You can also define the volumes creation on its own docker-compose file (like volumes/docker-compose.yml) :

version: '3.6'

volumes:
  static-files:

And reference this volume as external, with name volumes_static-files, in web and nginx docker-compose.yml files :

volumes:
  volumes_static-files:
    external: true

Unfortunately, you cannot set the volume name in docker compose, it will be automatically prefixed. If this is really a problem, you can also create the volume manually (docker volume create static-files) before running any docker-compose up command (I do not recommand this solution though because it adds a manual step that can be forgotten if you reproduce your deployment on another environment).