How to Merge Docker Compose file with Bash

Question

I'm trying to merge the docker-compose.yml file with the docker-compose2.yml file with bash.

docker-compose.yml :

version: "3"

services:
  nexus:
    image: sonatype/nexus3
    volumes:
      - "/opt/nexus3/nexus-data:/nexus-data"
    ports:
      - "8081:8081"

volumes:
  nexus-data: {}

docker-compose2.yml :

version: "3"

services:
  nexus2:
    image: sonatype/nexus3
    volumes:
      - "/opt/nexus3/nexus-data:/nexus-data"
    ports:
      - "8082:8082"

volumes:
  nexus-data: {}

Output I Want:

version: "3"

services:
  nexus:
    image: sonatype/nexus3
    volumes:
      - "/opt/nexus3/nexus-data:/nexus-data"
    ports:
      - "8081:8081"

  nexus2:
    image: sonatype/nexus3
    volumes:
      - "/opt/nexus3/nexus-data:/nexus-data"
    ports:
      - "8082:8082"
volumes:
  nexus-data: {}

How do I get this output with bash?

Answer 1

The Docker Compose config command does exactly what you need, it takes multiple compose file and merges them.

Just pass them using multiple -f flags:

docker-compose -f docker-compose.yml -f docker-compose2.yml config

or using an environment variable:

COMPOSE_FILE=docker-compose.yml:docker-compose2.yml docker-compose config

---

The same approach is valid for every Docker Compose command, so if your final target is, for example, to set up your project, you can directly run:

docker-compose -f docker-compose.yml -f docker-compose2.yml up

Check the documentation for further details on how to specify multiple compose files.

Answer 2

I don't think you can do this (easily as a one-liner) in native bash without writing a script. I was curious so I did a quick search and found a yaml manipulation tool which supports merging yaml (docker-compose) files and looks like it fits your use-case.

I used brew to install on MacOS but there are instructions for Linux as well - https://mikefarah.github.io/yq/.

brew install yq

Showing existing files:

$ cat file1.yaml
version: "3"

services:
  nexus:
    image: sonatype/nexus3
    volumes:
      - "/opt/nexus3/nexus-data:/nexus-data"
    ports:
      - "8081:8081"

volumes:
  nexus-data: {}

$ cat file2.yaml
version: "3"

services:
  nexus2:
    image: sonatype/nexus3
    volumes:
      - "/opt/nexus3/nexus-data:/nexus-data"
    ports:
      - "8082:8082"

volumes:
  nexus-data: {}

Merge both files outputting to stdout:

$ yq m file1.yaml file2.yaml
services:
  nexus:
    image: sonatype/nexus3
    ports:
    - 8081:8081
    volumes:
    - /opt/nexus3/nexus-data:/nexus-data
  nexus2:
    image: sonatype/nexus3
    ports:
    - 8082:8082
    volumes:
    - /opt/nexus3/nexus-data:/nexus-data
version: "3"
volumes:
  nexus-data: {}

There may be a native way but I just redirected the stdout to a file:

$ yq m file1.yaml file2.yaml > file3.yaml
$ cat file3.yaml
services:
  nexus:
    image: sonatype/nexus3
    ports:
    - 8081:8081
    volumes:
    - /opt/nexus3/nexus-data:/nexus-data
  nexus2:
    image: sonatype/nexus3
    ports:
    - 8082:8082
    volumes:
    - /opt/nexus3/nexus-data:/nexus-data
version: "3"
volumes:
  nexus-data: {}

There are a lot of examples in their documentation for you to explore - https://mikefarah.github.io/yq/merge/.