How to set env variable in Jupyter notebook

Question

To set an env variable in a jupyter notebook, just use a % magic commands, either %env or %set_env, e.g., %env MY_VAR=MY_VALUE or %env MY_VAR MY_VALUE. (Use %env by itself to print out current environmental variables.)

See: http://ipython.readthedocs.io/en/stable/interactive/magics.html

You can also set the variables in your kernel.json file:

My solution is useful if you need the same environment variables every time you start a jupyter kernel, especially if you have multiple sets of environment variables for different tasks.

To create a new ipython kernel with your environment variables, do the following:

• Read the documentation at https://jupyter-client.readthedocs.io/en/stable/kernels.html#kernel-specs
• Run jupyter kernelspec list to see a list with installed kernels and where the files are stored.
• Copy the directory that contains the kernel.json (e.g. named python2) to a new directory (e.g. python2_myENV).
• Change the display_name in the new kernel.json file.
• Add a env dictionary defining the environment variables.

Your kernel json could look like this (I did not modify anything from the installed kernel.json except display_name and env):

{
 "display_name": "Python 2 with environment",
 "language": "python",
 "argv": [
  "/usr/bin/python2",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "env": {"LD_LIBRARY_PATH":""}
}

Use cases and advantages of this approach

• In my use-case, I wanted to set the variable LD_LIBRARY_PATH which effects how compiled modules (e.g. written in C) are loaded. Setting this variable using %set_env did not work.
• I can have multiple python kernels with different environments.
• To change the environment, I only have to switch/ restart the kernel, but I do not have to restart the jupyter instance (useful, if I do not want to loose the variables in another notebook). See -however - https://github.com/jupyter/notebook/issues/2647

If you're using Python, you can define your environment variables in a .env file and load them from within a Jupyter notebook using python-dotenv.

Install python-dotenv:

pip install python-dotenv

Load the .env file in a Jupyter notebook:

%load_ext dotenv
%dotenv

You can setup environment variables in your code as follows:

import sys,os,os.path
sys.path.append(os.path.expanduser('~/code/eol_hsrl_python'))
os.environ['HSRL_INSTRUMENT']='gvhsrl'
os.environ['HSRL_CONFIG']=os.path.expanduser('~/hsrl_config')

This if of course a temporary fix, to get a permanent one, you probably need to export the variables into your ~.profile, more information can be found here

A gotcha I ran into: The following two commands are equivalent. Note the first cannot use quotes. Somewhat counterintuitively, quoting the string when using %env VAR ... will result in the quotes being included as part of the variable's value, which is probably not what you want.

%env MYPATH=C:/Folder Name/file.txt

and

import os
os.environ['MYPATH'] = "C:/Folder Name/file.txt"

If you need the variable set before you're starting the notebook, the only solution which worked for me was env VARIABLE=$VARIABLE jupyter notebook with export VARIABLE=value in .bashrc.

In my case tensorflow needs the exported variable for successful importing it in a notebook.