How to choose your conda environment in Jupyter Notebook

Question

I installed Anaconda 5.3 with Python 3.7 (root environment).
After that I created a new environment (py36) using Python 3.6

I activated the new environment with activate py36
conda env list shows that the environement is active.

But when I start Jupyter Notebook (from the Anaconda prompt with jupyter notebook), it seems to use the root environemt, not the activated environment.

How can I use Jupyter Notebook with the new create environment (py36)?

Answer 1

As @Ista mentioned, the documentation gives an easy solution using notebook extensions.

conda install nb_conda

After installing, you have the option in Jupyter Notebook to 'Change kernel' from the 'Kernel' menu in your Jupyter Notebook.

Answer 2

I managed to find a solution for this in a similar problem. The thing is that IPython is not virtualenv-aware, so a workaround (the one that I found to be most comfortable) is to specify custom IPython kernels to avoid having one Jupyter Notebook installation for each virtualenv (or anaconda environments, in your case).

Jupyter relies on some "kernels" (definitions of where to find the python binary) that are stored somewhere in your OS. These files are something like this:

{
 "display_name": "NameOfTheKernel", 
 "language": "python", 
 "argv": [
  "/usr/bin/python", 
  "-m", 
  "ipykernel_launcher", 
  "-f", 
  "{connection_file}"
 ]

Where /usr/bin/python is the path to the python binary that will be executed. However, as these kernels are defined somewhere in your computer by Jupyter, they don't update when you install some other environments (which is the case for anaconda or virtualenv). I found that the easiest way is to define a custom kernel for each environment that you use. Also, doing this supresses the need to activate the environment every single time you want to use it, as it is loaded directly.

The idea is to define a custom kernel so that Jupyter can "see" the environment that you have created with anaconda. For doing so, execute the following line in bash:

ipython kernel install --user --name=NameOfTheKernel

The "NameOfTheKernel" doesn't actually matter that much. If you don't have the ipython package, install it with pip, anaconda, via sudo apt install or whatever.

What this line will do is to define a custom kernel that will be detected by jupyter. For illustration purposes, in Ubuntu, this will be stored in the folder /home/USERNAME/.local/share/jupyter with this data structure:

/home/USERNAME/.local/share/jupyter/kernels/
└── nameofthekernel
    ├── kernel.json
    ├── logo-32x32.png
    └── logo-64x64.png

Once you have installed the kernel, you have to:

1) Know where your environment has been installed by anaconda. An easy way to do it is to activate your environment in anaconda and, then, write "which python" in the terminal. That will show the full path to the binary.

2) Write that path in the jupyter kernel just created. E.g. using pluma:

pluma /home/USERNAME/.local/share/jupyter/kernels/nameofthekernel/kernel.json

Then, you substitute the path of the python binary you installed with anaconda where /usr/bin/python is.

After this, if Jupyter was running, restart it. This way, the next time you open Jupyter, you can change the kernel (in the notebooks, one of the tabs in the upper part) and you will be using your environment and everything installed alongside that environment.

[TL;DR] I did it with pip, but the steps for anaconda will be more or less the same. The steps are:

#CREATE THE IPYTHON KERNELS
ipython kernel install --user --name=NameOfKernel
#IF PYTHON2 - MODIFY THE KERNELS TO USE THE ANACONDA BINARIES
sed -i -e "s|/usr/bin/python3|/home/${USER}/anaconda/bin/python27|g" /home/$USER/.local/share/jupyter/kernels/nameofkernel/kernel.json
#IF PYTHON3 - MODIFY THE KERNELS TO USE THE ANACONDA BINARIES
sed -i -e "s|/usr/bin/python3|/home/${USER}/anaconda/bin/python36|g" /home/$USER/.local/share/jupyter/kernels/nameofkernel/kernel.json

Or, if you are working with environments:

#CREATE THE IPYTHON KERNELS
ipython kernel install --user --name=NameOfKernel
#IF PYTHON2 - MODIFY THE KERNELS TO USE THE ANACONDA BINARIES
sed -i -e "s|/usr/bin/python3|/home/${USER}/anaconda/envs/nameofenvironment/python27|g" /home/$USER/.local/share/jupyter/kernels/nameofkernel/kernel.json
#IF PYTHON3 - MODIFY THE KERNELS TO USE THE ANACONDA BINARIES
sed -i -e "s|/usr/bin/python3|/home/${USER}/anaconda/envs/nameofenvironment/python36|g" /home/$USER/.local/share/jupyter/kernels/nameofkernel/kernel.json

BEWARE: I DID NOT INSTALL CONDA FOR TESTING THE SOLUTION, SO THE PATHS LEADING TO THE ACTUAL PYTHON BINARIES MIGHT CHANGE. The procedure is the same, however.