Docker environment variables in multi-stage builds

Question

given this .env file:

TEST=33333

given this docker-compose.yml file:

  service_name:
    image: test
    env_file: .env
    environment:
      TEST: 22222

given this Dockerfile file:

FROM an_image AS builder

FROM another_image
ENV TEST 11111

CMD ["/bin/echo $TEST"]

Whenever I build and run this image in a container, it prints 11111.

If I remove the ENV 11111 line from the Dockerfile, my TEST environment variable is empty...

Is the parent image receiving the environment variables but not the child one?

Thanks!

EDIT:

1. trying ENV TEST ${TEST} didn't work ($TEST is empty)
2. removing ENV TEST didn't work ($TEST is empty)

Answer 1

The problem is not about multi-stage specifically.

It's about differences between Dockerfile ARG & docker-compose YAML build args ("build arguments"); and Dockerfile ENV & docker-compose YAML environment/.env.

The docs were updated (more recently than the original post), and it is fairly clear now:

args

Add build arguments, which are environment variables accessible only during the build process.

Example from the docker-compose docs

Starting simple, just showing the interaction between Dockerfile and the YAML:

ARG buildno
ARG gitcommithash

RUN echo "Build number: $buildno"
RUN echo "Based on commit: $gitcommithash"

build:
  context: .
  args:
    buildno: 1
    gitcommithash: cdc3b19

build:
  context: .
  args:
    - buildno=1
    - gitcommithash=cdc3b19

Example to tie it back to the question:

See the other answer in this thread.

---

Docs & deepening your understanding

Learn one layer of abstraction at a time

I recommend to go from the Dockerfile level of abstraction, upward. Making sure you understand each layer before you add the next layer of abstraction.

1. Dockerfile (and then play with running containers from your Dockerfile ... using default ENV, then playing with --env, then playing with ARG and --build-arg)

2. Then add docker-compose details in, and play with those.

3. Then loop back to Dockerfiles and understanding multi-stage builds.

Dockerfile

A helpful blog-post -- focuses on the Dockerfile but in all cases, it's best to understand Dockerfiles alone before adding the extra layers of abstraction on top of that, such as docker-compose YAML.

https://vsupalov.com/docker-arg-env-variable-guide/

docker-compose

Then docker-compose official docs:

• https://docs.docker.com/compose/environment-variables/
• https://docs.docker.com/compose/env-file/
• https://docs.docker.com/compose/compose-file/#environment
• https://docs.docker.com/compose/compose-file/#env_file

multi-stage Dockerfiles

• https://docs.bitnami.com/containers/how-to/optimize-docker-images-multistage-builds/
• https://medium.com/@tonistiigi/advanced-multi-stage-build-patterns-6f741b852fae
• https://github.com/garethr/multi-stage-build-example

Answer 2

So this is not a multi-stage issue.

It appears ENV variables are only used when running containers (docker-compose up). Not at build time (docker-compose build). So you have to use arguments:

.env:

TEST=11111

docker-compose.yaml:

version: '3'
services:
  test:
    build:
      context: .
      args:
        TEST: ${TEST}

Dockerfile:

FROM nginx:alpine
ARG TEST
ENV TEST ${TEST}
CMD ["sh", "-c", "echo $TEST"]

test command:

docker rmi test_test:latest ; docker-compose build && docker run -it --rm test_test:latest

Seriously the documentation is somewhat lacking.

Reference: https://github.com/docker/compose/issues/1837