Execute Python script within Jupyter notebook using a specific virtualenv

Question

I would like to execute a long running Python script from within a Jupyter notebook so that I can hack on the data structures generated mid-run.

The script has many dependencies and command line arguments and is executed with a specific virtualenv. Is it possible to interactively run a Python script inside a notebook from a specified virtualenv (different to that of the Jupyter installation)?

Thanks!

Answer 1

Here's what worked for me (non conda python): (MacOS, brew version of python. if you are working with system python, you may (will) need prepend each command with sudo)

first activate virtualenv

if starting afresh then, e.g., you could use virtualenvwrapper

$pip install virtualenvwrapper $mkvirtualenv -p python2 py2env  $workon py2env  # This will activate virtualenv  (py2env)$   # Then install jupyter within the active virtualenv (py2env)$ pip install jupyter  # jupyter comes with ipykernel, but somehow you manage to get an error due to ipykernel, then for reference ipykernel package can be installed using: (py2env)$ pip install ipykernel 

Next, set up the kernel

(py2env)$ python -m ipykernel install --user --name py2env --display-name "Python2 (py2env)" 

then start jupyter notebook (the venv need not be activated for this step)

(py2env)$ jupyter notebook # or #$ jupyter notebook 

in the jupyter notebook dropdown menu: Kernel >> Change Kernel >> <list of kernels> you should see Python2 (py2env) kernel

This also makes it easy to identify python version of kernel, and maintain either side by side.

here is the link to detail docs http://ipython.readthedocs.io/en/stable/install/kernel_install.html

Answer 2

A bit more simple solution to get notebook kernels available in other notebooks.

I'm using Linux + virtualenv + virtualenvwrapper. If you are using different setup, change some commands to the appropriate ones, but you should get the idea.

mkvirtualenv jupyter2 workon jupyter2 (jupyter2) pip install jupyter (jupyter2) ipython kernel install --name "jupyter2_Python_2" --user 

last command creates ~/.local/share/jupyter/kernels/jupyter2\ python\ 2/ directory

same stuff for 3

mkvirtualenv -p /usr/bin/python3 jupyter3 // this uses python3 as default python in virtualenv workon jupyter3 (jupyter3) pip install jupyter (jupyter3) ipython kernel install --name "jupyter3_Python_3" --user 

When done you should see both kernels, no matter what env are you using to start jupyter. You can delete links to kernels directly in ~/.local/share/jupyter/kernels/. To specify location provide options to ipython kernel install (--help) or just copy directories from ~/.local/share/jupyter/kernels/ to ~/envs/jupyter3/share/jupyter if you want to run multiple kerenels from one notebook only.