docker-compose use environment variables from .env file

Question

I have some environment variables which I want to use in both docker-compose configurations and Dockerfile. I have an environment file named .env in order to make use of them in docker-compose file (a solution that I used for this issue); but these environment variables are not reachable in node's Dockerfile. Consider JMX_PORT, WS_PORT and IP as variables which are defined in my .env file; I put an echo for each of them in my Dockerfile to check if they are passed to it or not.

I tried to put .env in the env_file section of the node:

version: "2"
services:
  mynode:
    build:
      context: .
      dockerfile: Dockerfile-mynode
    env_file:
      - .env
    ports:
      - "${JMX_PORT}:${JMX_PORT}"
      - "${WS_PORT}:${WS_PORT}"

I also tried:

version: "2"
services:
  mynode:
    build:
      context: .
      dockerfile: Dockerfile-mynode
    environment:
      - IP=${IP}
      - JMX_PORT=${JMX_PORT}
      - WS_PORT=${WS_PORT}
    ports:
      - "${JMX_PORT}:${JMX_PORT}"
      - "${WS_PORT}:${WS_PORT}"

And with a little change:

version: "2"
services:
  mynode:
    build:
      context: .
      dockerfile: Dockerfile-mynode
    environment:
      - IP
      - JMX_PORT
      - WS_PORT
    ports:
      - "${JMX_PORT}:${JMX_PORT}"
      - "${WS_PORT}:${WS_PORT}"

None of the above approaches worked.

I don't want to redefine them in another file. Is there any solution for that?

EDIT 1:

This is my Dockerfile:

FROM mybaseimage
MAINTAINER Zeinab Abbasimazar
RUN echo ${IP}
RUN echo ${JMX_PORT}
RUN echo ${WS_PORT}

EDIT 2:

This is the output of docker-compose config command which implies the docker-compose has access to the environment variables as expected:

services:
  mynode:
    build:
      context: /home/zeinab/Workspace/ZiZi-Docker/Test/test-1
      dockerfile: Dockerfile-WRCore
    environment:
      IP: 171.80.80.80
      JMX_PORT: '1998'
      WS_PORT: '9000'
    ports:
    - 1998:1998
    - 9000:9000
version: '2.0'
volumes: {}

And this is the output of docker-compose build --no-cache command which implies the Dockerfile does not have access to the environment variables as expected:

docker-compose build --no-cache
Building mynode
Step 1 : FROM mybaseimage
 ---> 875978d91e2e
Step 2 : MAINTAINER Zeinab Abbasimazar
 ---> Running in 250eac87d078
 ---> 0e7bd6a18c80
Removing intermediate container 250eac87d078
Step 3 : RUN echo ${IP}
 ---> Running in 4ee6dbf74b2a

 ---> fd2bff1de6b6
Removing intermediate container 4ee6dbf74b2a
Step 4 : RUN echo ${JMX_PORT}
 ---> Running in 01245df43926

 ---> 68b7e90b1c01
Removing intermediate container 01245df43926
Step 5 : RUN echo ${WS_PORT}
 ---> Running in d27db926ecac

 ---> a8c3d0cbade8
Removing intermediate container d27db926ecac
Successfully built a8c3d0cbade8

Answer 1

So what you are doing is wrong. Reason being there is environment variable to be passed to Dockerfile while building. So there are two possible solutions

Use Builder Arguments

FROM mybaseimage
MAINTAINER Zeinab Abbasimazar
ARG IP
ARG JMX_PORT
ARG WS_PORT
RUN echo ${IP}
RUN echo ${JMX_PORT}
RUN echo ${WS_PORT}

And then pass these build arguments using compose

version: "2"
services:
  mynode:
    build:
      context: .
      dockerfile: Dockerfile-mynode
      args:
        - IP=${IP}
        - JMX_PORT=${JMX_PORT}
        - WS_PORT=${WS_PORT}
    environment_file:
      - .env
    ports:
      - "${JMX_PORT}:${JMX_PORT}"
      - "${WS_PORT}:${WS_PORT}"

Also you can load environment variables using environment_file. now this is only available when the container is started and they don't apply to your compose file or to your build file.

Also adding the build arguments will only make them available during the build and not during the container start. If you need to do that you need to define the variable twice

FROM mybaseimage
MAINTAINER Zeinab Abbasimazar
ARG IP
ARG JMX_PORT
ARG WS_PORT
ENV IP=${IP} JMX_PORT=${JMX_PORT} WS_PORT=${WS_PORT}
RUN env

Copying the environment file

.env

export NAME=TARUN

Dockerfile

FROM alpine
COPY .env /tmp/.env
RUN cat /tmp/.env > /etc/profile.d/tarun.sh && chmod +x /etc/profile.d/tarun.sh
SHELL ["sh","-lc"]
RUN env

In the RUN cat /tmp/.env > /etc/profile.d/tarun.sh && chmod +x /etc/profile.d/tarun.sh we create profile file which should be loaded when a shell is executed with profile.

In SHELL ["sh","-lc"] we change default shell from sh -c to sh -l so it can load our profile also.

Output of the last step is

Step 5/5 : RUN env
 ---> Running in 329ec287a5a6
HOSTNAME=e1ede117fb1e
SHLVL=1
HOME=/root
PAGER=less
PS1=\h:\w\$
NAME=TARUN
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
CHARSET=UTF-8
 ---> 6ab95f46e940

As you can see the environment variable is there.

Note: This .env format cannot be loaded in docker-compose as we have an export statement which is a bash command