Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Docker Compose merge arrays for YAML aliases and anchors

Tags:

docker

yaml

I have the following broken docker-compose file

version: '3.4'

x-vols1: &vols-1
    - /home/:/home/

x-vols2: &vols-2
    - /tmp/:/tmp/

services:
    app1:
        container_name: app1
        image: app1
        volumes:
            <<: *vols-1
    app2:
        container_name: app2
        image: app2
        volumes:
            <<: *vols-1
            <<: *vols-2

This fails with the following error

$ docker-compose -f test.yaml config
ERROR: yaml.constructor.ConstructorError: while constructing a mapping
  in "./test.yaml", line 14, column 13
expected a mapping for merging, but found scalar
  in "./test.yaml", line 4, column 7

Question 1: How can I merge arrays in docker-compose? The syntax that I am trying to use is the one for merging dicts

Question 2: If there is no way to merge arrays, is there a workaround?

Use case: I have multiple services, some of them map some volumes, others map other volumes, others map all volumes. I would like to not repeat myself.

Thank you!

like image 981
Andrei Cioara Avatar asked Apr 12 '20 20:04

Andrei Cioara


People also ask

How to use YAML anchors in Docker Compose?

Docker Tip #82: Using YAML Anchors and X Properties in Docker Compose YAML anchors let you reuse parts of your YAML file like functions. You can use them to share default settings between services. This is quite handy when you want to DRY out some bits of your docker-compose.yml file.

What is the difference between aliasing and anchor in YAML?

Anchors need to be before aliasing (e.g. in an aliasor merge key), can remain unused and can be re-defined, last one wins at the time of use (flow is from top to bottom). Example-YAMLwith Aliases, Referencesand Merge Keystaken from the Merge Keyworking draft:

What are Docker Compose’s “extension fields?

It’s very handy for setting up common things like logging or various other Docker properties. You can name the x- properties anything you want as long as it’s valid YAML syntax. These x- properties are called “Extension fields” in Docker Compose’s documentation. The basic idea is the extension field just acts as a “place” to define the YAML anchor.

Is it easy to repeat yourself in Docker Compose?

But, it can be easy to end up with repeated blocks in stacks with lots of similar services or configurations. Don’t Repeat Yourself (DRY) by leveraging YAML aliases and anchors, and the upcoming extension fields feature in the Docker Compose file format.


2 Answers

The Yaml merge syntax is for merging mappings, not for arrays. For more on that, see this issue. However, if you are just adding single volumes, you don't need to merge anything. Just insert the alias as an array entry:

version: '3.4'

x-vols1: &vols-1
    "/home/:/home/"

x-vols2: &vols-2
    "/tmp/:/tmp/"

services:
    app1:
        container_name: app1
        image: app1
        volumes:
            - *vols-1
    app2:
        container_name: app2
        image: app2
        volumes:
            - *vols-1
            - *vols-2
like image 191
BMitch Avatar answered Sep 16 '22 16:09

BMitch


The desired behavior can be achieved by using multiple docker-compose files, one for each volume. Note that the anchors and aliases are not required, but keeping them in to align with the question.

base.yaml

version: '3.4'

services:
    app1:
        container_name: app1
        image: app1
    app2:
        container_name: app2
        image: app2

vol1.yaml

version: '3.4'

x-vols1: &vols-1
    volumes:
        - /home/:/home/

services:
    app1:
        container_name: app1
        image: app1
        <<: *vols-1
    app2:
        container_name: app2
        image: app2
        <<: *vols-1

vol2.yaml

version: '3.4'

x-vols2: &vols-2
    volumes:
        - /tmp/:/tmp/

services:
    app2:
        container_name: app2
        image: app2
        <<: *vols-2

Verify as

$ docker-compose -f base.yaml -f vol1.yaml -f vol2.yaml config

Result

services:
  app1:
    container_name: app1
    image: app1
    volumes:
    - /home:/home:rw
  app2:
    container_name: app2
    image: app2
    volumes:
    - /home:/home:rw
    - /tmp:/tmp:rw
version: '3.4'

Additional documentation https://docs.docker.com/compose/extends/

like image 33
Andrei Cioara Avatar answered Sep 17 '22 16:09

Andrei Cioara