Jupyter Notebook ModuleError after Homebrew Upgrade

Question

I have been using Jupyter for some time now and it has worked just fine. I have Jupyter and Python installed via Homebrew. I am running on MacOS.

Yesterday, I ran the command brew upgrade and now my Jupyter notebook is unable to find any of the installed python packages. I will use Numpy as the example.

When inside of a Jupyter notebook, I try to do

import numpy

I get the message:

ModuleNotFoundError: No module named 'numpy'

If, however, I launch python in a terminal window, then I can import Numpy without issue.

I first checked that the package was installed correctly by re-issuing the install command

brew install numpy

which outputs:

Warning: numpy 1.18.4 is already installed and up-to-date
To reinstall 1.18.4, run `brew reinstall numpy` 

I also ran

pip install numpy

and got:

Requirement already satisfied: numpy in /usr/local/lib/python3.7/site-packages (1.18.4)

Now, this is where I got confused because I expected the path to point to something like /usr/local/Cellar/, so I checked the path inside of the Jupyter notebook:

import sys
sys.path

which outputs:

['/Users/kseuro/Dropbox/Dev/',
 '/usr/local/Cellar/jupyterlab/2.1.2/libexec/lib/python38.zip',
 '/usr/local/Cellar/jupyterlab/2.1.2/libexec/lib/python3.8',
 '/usr/local/Cellar/jupyterlab/2.1.2/libexec/lib/python3.8/lib-dynload',
 '/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8',
 '',
 '/usr/local/Cellar/jupyterlab/2.1.2/libexec/lib/python3.8/site-packages',
 '/usr/local/Cellar/jupyterlab/2.1.2/libexec/lib/python3.8/site-packages/IPython/extensions',
 '/Users/kseuro/.ipython']

Ok, so Homebrew wants Jupyter to use Python3.8? So I tried brew switch python 3.8 and got:

Error: python does not have a version "3.8" in the Cellar.
python's installed versions: 3.7.7

I feel like I'm out of my depth now and need help figuring out what to do next. I don't want to start by just changing paths around.

Suggestions? Thanks so much.

Answer 1

I figured out what to do — posting the solution for my future self and others who may stumble upon this.

Since Jupyerlab is in its own Cellar, the Python packages need to end up in the

/usr/local/Cellar/jupyterlab/x.y.z/libexec/lib/python3.x/site-packages

directory, where x, y, z are integers, so that the Jupyter kernel can find them.

You can do this by calling:

import sys
!{sys.executable} -m pip install 'package-name'

inside of the Jupyer notebook.

All is well, again.