Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Equivalent of volumes_from in Docker Compose v3

Tags:

There are seemingly similar questions here (docker-compose volumes_from equivalent with version 3, How to replace volumes_from in docker-composer v3) but I don't think they answer the question (or at least I don't understand how the answers solve the problem). So let me try to ask again, very specifically.

I have this v2 docker-compose.yml:

version: '2' services:   full-tests:     volumes:       - ..:/opt/project       - ../../../ext-libs:/opt/ext-libs       - ./third-mapping:/opt/third    unit-tests:     volumes_from: full-tests 

The point is that the set of volumes is defined once and I can easily reuse them using volumes_from.

How would you rewrite this in v3?

like image 388
Borek Bernard Avatar asked Jun 23 '17 11:06

Borek Bernard


People also ask

How to use Docker Compose volumes?

Use one/various volumes by one set of services (defined in the same docker-compose.yml file). Use one/various volumes across the Docker installation. In this tutorial, we’ll learn how to use Docker Compose volumes. A GNU Linux/Mac OS/Windows machine with Docker and Docker Compose installed is required to follow this tutorial.

What are network and volume definitions in dockerfile?

Likewise, network and volume definitions are analogous to docker network create and docker volume create. As with docker run, options specified in the Dockerfile, such as CMD , EXPOSE, VOLUME, ENV, are respected by default - you don’t need to specify them again in docker-compose.yml.

What are the different types of Docker host-mounted volumes?

Docker host-mounted volumes Host path can be defined as an absolute or as a relative path. ? 2. Docker named volumes Named volumes can be defined as internal (default) or external. 2.1. Docker internal named volumes Docker compose internal named volumes have the scope of a single Docker-compose file and Docker creates them if they don’t exist.

Is there a way to use named volumes in compose format V3?

But there appears to be no way to do this in compose format v3 and using docker stack. I've seen it said a few times that named volumes "replace" volumes_from, but I don't see any way to use named volumes to deploy data as a versioned artifact as I have done above.


2 Answers

To answer your question - its impossible with v3 - see the section below. v3 shall not be used as a successor ( also a official statement by docker ) it shall be used in "swarm cases".

nevertheless, what you do should do is using named volumes.

You can combine it with host-mount volumes like this

docker volume create --name volume1 -o type=none -o device=/home/$USER/projects/01 -o o=bind 

You can simplify this using the long-syntax introduced in 3.2: https://docs.docker.com/compose/compose-file/#long-syntax-2 so you can define the named volume + bind on the host in the docker-compose file example:

services:    full-tests:     volumes:            - type: volume         source: ../         target: /opt/project       - type: volume         source: ../../../ext-libs         target: /opt/ext-libs 

or in short as you had

services:    full-tests:      volumes:            - ../:/opt/project       - ../../../ext-libs:/opt/ext-libs 

What you cannot do though, putting the long-syntax under the top-level "volumes" definition to give that volume a name and reused it in the volumes section in the services - that is not possible. To do so, you would use a

volumes:   project:     external: true   third-party:     external: true 

And then use the "docker volume create" syntax on the cli to create those volumes with a bind option, as outlines above

but you will never get what volumes_from was doing for you here


There is no equivalent of volumes_from in v3, since v3 is not a successor of v2, its an alternative - please see my comment and the sources here https://github.com/rancher/rancher/issues/3316#issuecomment-310889283

To sum it up - volumes_from and volumes have an overlap in the case volumes_from was just used the wrong way / in the wrong field.

a) If you want data to be persisted across stack upgrades ( down + up ), you pick named volumes - and now, if 2+ services needs to share this, just mount this named volume using volumes:.

b) If you though, do not want the data to persist over stack upgrades ( e.g. because its source code and the image actually includes an upgrades this ) as a in a usual application + httpd/proxy scenario, you will create a anon-volume for this e.g. /var/www in the Dockerfile using Volume /var/www and then use volumes_from to mount it in the httpd service.


the main point with b is, that on stack upgrades, the anon volume will be removed ( `docker-compose down removes anon containers, but not named ones ) and thus the upgrade works as expected, you have a new codebase

Trying to do the same with named volumes will give you a huge suprise on the first upgrade, since the code is on a named volume and that will overlay the codebase on the "fresh" image / new container, thus you will run on the old codebase after the upgrade.

like image 153
Eugen Mayer Avatar answered Oct 10 '22 15:10

Eugen Mayer


You could use extension fields to keep the code short but it isn't quite the same as volumes_from. For example:

version: '3.7' x-volumes:   &my-volumes       - ..:/opt/project       - ../../../ext-libs:/opt/ext-libs       - ./third-mapping:/opt/third  services:   full-tests:     volumes:       *my-volumes    unit-tests:     volumes:       *my-volumes 
like image 24
Dave Avatar answered Oct 10 '22 16:10

Dave