How to set Docker Compose `env_file` relative to `.yml` file when multiple `--file` option is used?

Question

I am trying to set my env_file configuration to be relative to each of the multiple docker-compose.yml file locations instead of relative to the first docker-compose.yml.

The documentation (https://docs.docker.com/compose/compose-file/compose-file-v3/#env_file) suggests this should be possible:

If you have specified a Compose file with docker-compose -f FILE, paths in env_file are relative to the directory that file is in.

For example, when I issue

docker compose \
  --file docker-compose.yml \
  --file backend/docker-compose.yml \
  --file docker-compose.override.yml up

all of the env_file paths in the second (i.e. backend/docker-compose.yml) and third (i.e. docker-compose.override.yml) are relative to the location of the first file (i.e. docker-compose.yml)

I would like to have the env_file settings in each docker-compose.yml file to be relative to the file that it is defined in.

Is this possible?

Thank you for your time 🙏

In case you are curious about the context:

I would like to have a backend repo that is self-contained and the backend developer can just work on it without needing the frontend container. The frontend repo will pull in the backend repo as a Git submodule, because the frontend container needs the backend container as a dependency. Here are my 2 repo's:

• backend: https://gitlab.com/starting-spark/porter/backend
• frontend: https://gitlab.com/starting-spark/porter/frontend

The backend is organized like this:

/docker-compose.yml
/docker-compose.override.yml

The frontend is organized like this:

/docker-compose.yml
/docker-compose.override.yml
/backend/ # pulled in as a Git submodule
/backend/docker-compose.yml
/backend/docker-compose.override.yml

Everything works if I place my env_file inside the docker-compose.override.yml file. The backend's override env_file will be relative to the backend docker-compose.yml. And the frontend's override env_file will be relative to the frontend docker-compose.yml. The frontend will never use the backend's docker-compose.override.yml.

But I wanted to put the backend's env_file setting in to the backend's docker-compose.yml instead, so that projects needing the backend container can inherit and just use it's defaults. If the depender project wants to override backend's env_file, then it can do so in the depender project's docker-compose.override.yml.

I hope that makes sense.

If there's another pattern to organizing Docker-Compose projects that handles this scenario, please let me know.

• I did want to avoid a mono-repo.

Answer 1

It turns out that there's already an issue and discussion regarding this:

• https://github.com/docker/compose/issues/3874

The thread points out that this is the expected behavior and is documented here: https://docs.docker.com/compose/extends/#understanding-multiple-compose-files

When you use multiple configuration files, you must make sure all paths in the files are relative to the base Compose file (the first Compose file specified with -f). This is required because override files need not be valid Compose files. Override files can contain small fragments of configuration. Tracking which fragment of a service is relative to which path is difficult and confusing, so to keep paths easier to understand, all paths must be defined relative to the base file.

There's a workaround within that discussion that works fairly well: https://github.com/docker/compose/issues/3874#issuecomment-470311052

The workaround is to use a ENV var that has a default:

• ${PROXY:-.}/haproxy/conf:/usr/local/etc/haproxy

Or in my case:

  env_file:
    - ${BACKEND_BASE:-.}/.env

Hope that can be helpful for others 🤞

In case anyone is interested in the full code: backend's docker-compose.yml: https://gitlab.com/starting-spark/porter/backend/-/blob/3.4.3/docker-compose.yml#L13-14
backend's docker-compose.override.yml: https://gitlab.com/starting-spark/porter/backend/-/blob/3.4.3/docker-compose.override.yml#L3-4
backend's .env: https://gitlab.com/starting-spark/porter/backend/-/blob/3.4.3/.env
frontend's docker-compose.yml: https://gitlab.com/starting-spark/porter/frontend/-/blob/3.2.2/docker-compose.yml#L5-6
frontend's docker-compose.override.yml: https://gitlab.com/starting-spark/porter/frontend/-/blob/3.2.2/docker-compose.override.yml#L3-4
frontend's .env: https://gitlab.com/starting-spark/porter/frontend/-/blob/3.2.2/.env#L16