No module named 'sklearn.neighbors._base'

Question

I have recently installed imblearn package in jupyter using

!pip show imbalanced-learn

But I am not able to import this package.

from tensorflow.keras import backend
from imblearn.over_sampling import SMOTE

I get the following error

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-20-f19c5a0e54af> in <module>
      1 # from sklearn.utils import resample
      2 from tensorflow.keras import backend
----> 3 from imblearn.over_sampling import SMOTE
      4 
      5 

~/.virtualenvs/p3/lib/python3.6/site-packages/imblearn/__init__.py in <module>
     32     Module which allowing to create pipeline with scikit-learn estimators.
     33 """
---> 34 from . import combine
     35 from . import ensemble
     36 from . import exceptions

~/.virtualenvs/p3/lib/python3.6/site-packages/imblearn/combine/__init__.py in <module>
      3 """
      4 
----> 5 from ._smote_enn import SMOTEENN
      6 from ._smote_tomek import SMOTETomek
      7 

~/.virtualenvs/p3/lib/python3.6/site-packages/imblearn/combine/_smote_enn.py in <module>
      8 from sklearn.utils import check_X_y
      9 
---> 10 from ..base import BaseSampler
     11 from ..over_sampling import SMOTE
     12 from ..over_sampling.base import BaseOverSampler

~/.virtualenvs/p3/lib/python3.6/site-packages/imblearn/base.py in <module>
     14 from sklearn.utils.multiclass import check_classification_targets
     15 
---> 16 from .utils import check_sampling_strategy, check_target_type
     17 
     18 

~/.virtualenvs/p3/lib/python3.6/site-packages/imblearn/utils/__init__.py in <module>
      5 from ._docstring import Substitution
      6 
----> 7 from ._validation import check_neighbors_object
      8 from ._validation import check_target_type
      9 from ._validation import check_sampling_strategy

~/.virtualenvs/p3/lib/python3.6/site-packages/imblearn/utils/_validation.py in <module>
     11 
     12 from sklearn.base import clone
---> 13 from sklearn.neighbors._base import KNeighborsMixin
     14 from sklearn.neighbors import NearestNeighbors
     15 from sklearn.utils.multiclass import type_of_target

ModuleNotFoundError: No module named 'sklearn.neighbors._base'

Other packages in the environment

numpy==1.16.2
pandas==0.24.2
paramiko==2.1.1
matplotlib==2.2.4
scikit-learn==0.22.1
Keras==2.2.4
tensorflow==1.12.0
tensorboard==1.12.0
tensorflow-hub==0.4.0
xlrd==1.2.0
flask==1.0.2
wtforms==2.2.1
bs4==0.0.1
gensim==3.8.1
spacy==2.2.3
nltk==3.4.5 
wordcloud==1.6.0
pymongo==3.10.1    
imbalanced-learn==0.6.1

I checked the sklearn package, it contains base module, not _base. But modifying it may not be the right solution. Any other solution to fix this issue.

Answer 1

If in case you want to persist with the latest version of scikit-learn, add the following code to your script or execute the following code in your environment before installing imblearn

import sklearn.neighbors._base
sys.modules['sklearn.neighbors.base'] = sklearn.neighbors._base

This has to be after

pip install sklearn

or in a notebook environment:

!pip install sklearn

This problem stems from the fact that certain modules are named with an underscore in the newer scikit-learn releases

Answer 2

Previous sklearn.neighbors.base has been renamed to sklearn.neighbors._base in version 0.22.1.
You have probably a version of scikit-learn older than that. Installing the latest release solves the problem:

pip install -U scikit-learn

or

pip install scikit-learn==0.22.1

Answer 3

I had a similar problem trying to import SMOTE from imblearn.over_sampling and my version of scikit-learn was up to date (0.24.1). What worked for me was:

First I downgraded my scikit learn version to 0.22.1 using

 pip install scikit-learn==0.22.1

Next, I updated the imbalanced-learn package using:

pip install -U imbalanced-learn

That uninstalled scikit-learn-0.22.1, installed the updated version (scikit-learn-0.24.1), and updated the imbalanced-learn package. Everything worked fine thereafter.