Jupyter Notebook ImportError: No module named 'sklearn'

Question

I am trying to run on my local machine. I get an error ImportError: No module named 'sklearn' only in jupyter notebook It works fine when I use python from the command line both with the carnd-term1 env activated and deactivated.

I have installed sklearn with pip, apt-get and conda. Also tried conda upgrade scikit-learn. Both with the env active and deactivated.

---

(carnd-term1) matt@Malta:~/sdc$ conda upgrade scikit-learn
Fetching package metadata .........
Solving package specifications: .
# All requested packages already installed.
# packages in environment at /home/matt/anaconda3/envs/carnd-term1:
#
scikit-learn 0.18.1 np112py35_1

---

(carnd-term1) matt@Malta:~/sdc$ python3
Python 3.5.2 | packaged by conda-forge | (default, Jan 19 2017, 15:28:33) 
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sklearn
>>>

---

   ...: (carnd-term1) matt@Malta:~/sdc$ ipython
   ...: Python 3.5.2 | packaged by conda-forge | (default, Jan 19 2017, 15:28:33) 
   ...: Type "copyright", "credits" or "license" for more information.
   ...: 
   ...: IPython 5.1.0 -- An enhanced Interactive Python.
   ...: ?         -> Introduction and overview of IPython's features.
   ...: %quickref -> Quick reference.
   ...: help      -> Python's own help system.
   ...: object?   -> Details about 'object', use 'object??' for extra details.
   ...: 
   ...: In [1]: import sklearn
   ...: 
   ...: In [2]: from sklearn.model_selection import train_test_split
   ...: 
   ...: In [3]: (carnd-term1) matt@Malta:~/sdc$ ipython
   ...:    ...: Python 3.5.2 | packaged by conda-forge | (default, Jan 19 2017, 15:28:33) 
   ...:    ...: Type "copyright", "credits" or "license" for more information.
   ...:    ...: 
   ...:    ...: IPython 5.1.0 -- An enhanced Interactive Python.
   ...:    ...: ?         -> Introduction and overview of IPython's features.
   ...:    ...: %quickref -> Quick reference.
   ...:    ...: help      -> Python's own help system.
   ...:    ...: object?   -> Details about 'object', use 'object??' for extra details.
   ...:    ...: 
   ...:    ...: In [1]: import sklearn
   ...:    ...: 
   ...:    ...: In [2]: from sklearn.model_selection import train_test_split
   ...:    ...: 
   ...:    ...: In [3]:

---

Doesn't work from jupyter notebook.

Any ideas?

Answer 1

Let's learn a general approach to solve these sort of problems. The solution is quite straightforward. Basically has three steps:

1. Finding where the pip package is installed.
2. Adding that directory to path.
3. Finally, importing the package.

Finding where the pip package is installed:

!pip show PACKAGE_NAME

Don't forget this ! before the command if you are executing it in jupyter-notebook. This will give you the path to that package (with possibly with other information). Get the path given inside Location.

Adding that directory to path: This following code should go before you import that package in jupyter.

import sys
sys.path.append('path/to/the/package')

Now import the package:

import PACKAGE_NAME

So, for sklearn:

Get the sklearn directory:

!pip show scikit-learn

Add directory:

import sys
sys.path.append('/path/to/sklearn')

For example, if you are using anaconda than the site-packages folder will contain all conda installed packages for that environment. Inside this path there is the folder sklearn which we are trying to import:

import sys
sys.path.append('/home/hafiz031/anaconda3/envs/NLP/lib/python3.8/site-packages')

Now import the desired package as usual:

import sklearn

References:

1. which version and where scikit-learn is installed

2. Python: Best way to add to sys.path relative to the current running script

Answer 2

If you use virtual environment, then you need to install Notebook into your environment:

pip install notebook

Answer 3

This generally means that the two are not the same environment. The best thing to check is sys.executable and make sure that it is what you expect. If it's the notebook that's not using the sys.executable you expect, the first step may be to check your PATHs:

which jupyter
which jupyter-notebook

The most likely issue is that the notebook stack isn't in your conda env, which you can solve with:

conda install notebook

The second most likely is that you have installed a kernelspec (e.g. with ipython kernel install --user) that's overriding your env. You can see where your kernels are with:

jupyter kernelspec list

To make sure you have the IPython kernel installed in the same env, you can do:

conda install ipykernel
ipython kernelspec install --sys-prefix

and check jupyter kernelspec list again after.