Exception Type: ValueError: node array from the pickle has an incompatible dtype

Question

So I copied a DjangoML project from Github. The ML model is essentially a diagnosis system that predicts whether or not a patient is at risk based on the values of their blood tests.

I'm running this project locally in VSCode but have yet to deploy the backend/ML model.

I encountered this problem while attempting to forecast the outcome in the web browser.

I'm using Python 3.11.0 for this project.

The requirements.txt file includes:

 asgiref==3.5.0

 Django==4.0.3

  joblib==1.1.0

 numpy~=1.26.4

 pandas==2.2.2

 python-dateutil==2.8.2

 pytz==2022.1

 scikit-learn==1.0.2

 scipy==1.8.0

 six==1.16.0

 sklearn==0.0

 sqlparse==0.4.2 

threadpoolctl==3.1.0 

tzdata==2021.5 

 `Traceback (most recent call last):

File "D:\Django_MachineLearning_HealthcareApp-main\venv\Lib\site-packages\django\core\handlers\exception.py", line 55, in inner response = get_response(request) 

File "D:\Django_MachineLearning_HealthcareApp-main\venv\Lib\site-packages\django\core\handlers\base.py", line 197, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "D:\Django_MachineLearning_HealthcareApp-main\backend\views.py", line 76, in lpredictor result = ValuePredictor(llis, 7, mname) 

File "D:\Django_MachineLearning_HealthcareApp-main\backend\views.py", line 60, in ValuePredictor trained_model = joblib.load(rf'{mdname}_model.pkl') 

File "D:\Django_MachineLearning_HealthcareApp-main\venv\Lib\site-packages\joblib\numpy_pickle.py", line 658, in load obj = _unpickle(fobj, filename, mmap_mode) 

File "D:\Django_MachineLearning_HealthcareApp-main\venv\Lib\site-packages\joblib\numpy_pickle.py", line 577, in _unpickle obj = unpickler.load()

File "C:\Users\vanda\AppData\Local\Programs\Python\Python311\Lib\pickle.py", line 1213, in load dispatch[key[0]](self)

File "D:\Django_MachineLearning_HealthcareApp-main\venv\Lib\site-packages\joblib\numpy_pickle.py", line 402, in load_build Unpickler.load_build(self)

File "C:\Users\vang\AppData\Local\Programs\Python\Python311\Lib\pickle.py", line 1718, in load_build setstate(state) File "sklearn\tree\_tree.pyx", line 865, in sklearn.tree._tree.Tree.setstate <source code not available>

File "sklearn\tree\_tree.pyx", line 1571, in sklearn.tree._tree._check_node_ndarray <source code not available>

Exception Type: ValueError at /diagnose/liver/report

Exception Location:sklearn\tree\_tree.pyx, line 1571, in sklearn.tree._tree._check_node_ndarray Exception Value: node array from the pickle has an incompatible dtype:

expected: {'names': ['left_child', 'right_child', 'feature', 'threshold', 'impurity', 'n_node_samples', 'weighted_n_node_samples', 'missing_go_to_left'], 'formats': ['<i8', '<i8', '<i8', '<f8', '<f8', '<i8', '<f8', 'u1'], 'offsets': [0, 8, 16, 24, 32, 40, 48, 56], 'itemsize': 64}

got     : [('left_child', '<i8'), ('right_child', '<i8'), ('feature', '<i8'), ('threshold', '<f8'), ('impurity', '<f8'), ('n_node_samples', '<i8'), ('weighted_n_node_samples', '<f8')] `

I'm a rookie in Python and machine learning and will appreciate any help and explanation on this.

Answer 1

It seems that you are trying to unpickle a model trained and pickled with older versions of scikit-learn and joblib. The format of models pickled by joblib changed recently and the compatibility with older files has been broken as a result.

The solution is simply to downgrade your version of those packages to match the correct version. Most likely this is scikit-learn==1.0.2 and joblib==1.1.0 as specified in the requirements you posted. You may have to downgrade your version of Python too actually, because scikit-learn 1.0.2 seems to require python 3.10 (not 3.11)