Docker-compose not passing environment variables to docker container

Question

I have a Python app that uses environment variables and I want make dev\prod setup with one Dockerfile and one docker-compose.yml file (only change env file with environment variables).

Here are the files I use to start the application:
Dockerfile:

FROM python:3.7-slim-buster
RUN apt-get update

WORKDIR /usr/src/app
RUN mkdir /usr/src/app/excel_users_dump
COPY ./requirements.txt .
RUN pip install -r requirements.txt

COPY . .
RUN python /usr/src/app/myblumbot/main.py

docker-compose.yml:

version: '3'
services:
  bot:
    build:
      context: .
    environment:
      ENV: ${ENV}
      PYTHONPATH: ${PYTHONPATH}
      PYTHONBUFFERED: ${PYTHONBUFFERED}
    volumes:
      - states:/var/myblumbot_states

volumes:
  states:

.env (in the same directory as docker-compose.yml)

PYTHONBUFFERED=1  
PYTHONPATH=/usr/src/app  
ENV=DEV

When I'm running docker-compose up command, it builds and tells me that I didn't have some environment variables so application can't start.

env = os.environ['ENV'] 

KeyError: 'ENV'

But if I add ENV VAR value in Dockerfile, everything works good.

How can I pass variables from docker-compose and .env file?

Answer 1

When you have a setup with both a Dockerfile and a docker-compose.yml file like you show, things run in two phases. During the first phase the image is built, and during the second the container actually gets run. Most of the settings in docker-compose.yml don't have an effect during the build stage; that includes network settings, environment variables, and published ports.

In your Dockerfile you're running your application in a RUN step. That happens as part of the build, not the execution phase; the image that finally gets generated is the filesystem that results after your application exits. Since it's during the build phase, environment variable settings don't take effect.

If you change RUN to CMD, then this will get recorded in the image, and after the build completes, it will run as the main container process with environment variable and other settings.

(In comments you suggest ENTRYPOINT. This will work too, for the same reasons, but it makes a couple of tasks like getting a debug shell harder, and there's a standard Docker first-time setup pattern that needs ENTRYPOINT for its own purposes. I'd prefer CMD here.)