Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Activating a Conda environment in Ansible playbook

I am trying to run a list of tasks (here running airflow but it could be anything really) that require to be executed in a existing Conda environment.

I would like to do these tasks:

- name: activate conda environment
 # does not work, just for the sake of understanding
 command: source activate my_conda_env

- name: initialize the database
  command: airflow initdb

- name: start the web server
  command: 'airflow webserver -p {{ airflow_webserver_port }}'

- name: start the scheduler
  command: airflow scheduler

Of course, this does not work as each task is independent and the conda environment activation in the first task is ignored by the following tasks.

I guess the issue would be the same if using a python virtualenv instead of conda.

How can I achieve each task being run in the Conda environment?

like image 838
jeff73 Avatar asked Jan 03 '17 16:01

jeff73


People also ask

How do you activate a conda environment?

To activate your Conda environment, type source activate <yourenvironmentname> . Note that conda activate will not work on Discovery with this version. To install a specific package, type conda install -n <yourenvironmentname> [package] . To deactivate the current, active Conda environment, type conda deactivate .

Do you have to activate conda environment every time?

Question 1: do i have to activate the conda env every time i open command prompt? No, you do not have to activate conda every time you open CMD. This is simply a matter of choice for the developer.


2 Answers

Each of your commands will be executed in a different process.

source command, on the other hand, is used for reading the environment variables into the current process only (and its children), so it will apply only to the activate conda environment task.

What you can try to do is:

- name: initialize the database
  shell: source /full/path/to/conda/activate my_conda_env && airflow initdb
  args:
    executable: /bin/bash

- name: start the web server
  shell: 'source /full/path/to/conda/activate my_conda_env && airflow webserver -p {{ airflow_webserver_port }}'
  args:
    executable: /bin/bash

- name: start the scheduler
  shell: source /full/path/to/conda/activate my_conda_env && airflow scheduler
  args:
    executable: /bin/bash

Before, check what's the full path to activate on the target machine with which activate (you need to do it before any environment is sourced). If Conda was installed in a user's space, you should use the same user for the Ansible connection.

like image 186
techraf Avatar answered Sep 28 '22 02:09

techraf


Was looking out for something similar. Found a neater solution than having multiple actions:

- name: Run commands in conda environment
  shell: source activate my_conda_env && airflow {{ item }}
  with_items:
    - initdb
    - webserver -p {{ airflow_webserver_port }}
    - scheduler
like image 38
pratpor Avatar answered Sep 28 '22 03:09

pratpor