condas `source activate virtualenv` does not work within Dockerfile

Question

Scenario

I'm trying to setup a simple docker image (I'm quite new to docker, so please correct my possible misconceptions) based on the public continuumio/anaconda3 container.

The Dockerfile:

FROM continuumio/anaconda3:latest

# update conda and setup environment
RUN conda update conda -y \
    && conda env list \
    && conda create -n testenv pip -y \
    && source activate testenv \
    && conda env list

Building and image from this by docker build -t test . ends with the error:

/bin/sh: 1: source: not found

when activating the new virtual environment.

Suggestion 1:

Following this answer I tried:

FROM continuumio/anaconda3:latest

# update conda and setup environment
RUN conda update conda -y \
    && conda env list \
    && conda create -y -n testenv pip \
    && /bin/bash -c "source activate testenv" \
    && conda env list

This seems to work at first, as it outputs: prepending /opt/conda/envs/testenv/bin to PATH, but conda env list as well ass echo $PATH clearly show that it doesn't:

[...]
# conda environments:
#
testenv                  /opt/conda/envs/testenv
root                  *  /opt/conda

---> 80a77e55a11f
Removing intermediate container 33982c006f94
Step 3 : RUN echo $PATH
---> Running in a30bb3706731
/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

The docker files work out of the box as a MWE. I appreciate any ideas. Thanks!

Answer 1

Using the docker ENV instruction it is possible to add the virtual environment path persistently to PATH. Although this does not solve the selected environment listed under conda env list.

See the MWE:

FROM continuumio/anaconda3:latest

# update conda and setup environment
RUN conda update conda -y \
    && conda create -y -n testenv pip

ENV PATH /opt/conda/envs/testenv/bin:$PATH

RUN echo $PATH
RUN conda env list

Answer 2

Method 1: use SHELL with a custom entrypoint script

EDIT: I have developed a new, improved approach which better than the "conda", "run" syntax.

Sample dockerfile available at this gist. It works by leveraging a custom entrypoint script to set up the environment before execing the arguments of the RUN stanza.

Why does this work?

A shell is (put very simply) a process which can act as an entrypoint for arbitrary programs. exec "$@" allows us to launch a new process, inheriting all of the environment of the parent process. In this case, this means we activate conda (which basically mangles a bunch of environment variables), then run /bin/bash -c CONTENTS_OF_DOCKER_RUN.

---

Method 2: SHELL with arguments

Here is my previous approach, courtesy of Itamar Turner-Trauring; many thanks to them!

# Create the environment:
COPY environment.yml .
RUN conda env create -f environment.yml

# Set the default docker build shell to run as the conda wrapped process
SHELL ["conda", "run", "-n", "vigilant_detect", "/bin/bash", "-c"]

# Set your entrypoint to use the conda environment as well
ENTRYPOINT ["conda", "run", "-n", "myenv", "python", "run.py"]

Modifying ENV may not be the best approach since conda likes to take control of environment variables itself. Additionally, your custom conda env may activate other scripts to further modulate the environment.

Why does this work?

This leverages conda run to "add entries to PATH for the environment and run any activation scripts that the environment may contain" before starting the new bash shell.

Using conda can be a frustrating experience, since both tools effectively want to monopolize the environment, and theoretically, you shouldn't ever need conda inside a container. But deadlines and technical debt being a thing, sometimes you just gotta get it done, and sometimes conda is the easiest way to provision dependencies (looking at you, GDAL).