KeyError when loading pickled scikit-learn model using joblib

Question

I have an object that contains within it two scikit-learn models, an IsolationForest and a RandomForestClassifier, that I would like to pickle and later unpickle and use to produce predictions. Apart from the two models, the object contains a couple of StandardScalers and a couple of Python lists.

Pickling this object using joblib is unproblematic, but when I try to unpickle it later I get the following exception:

Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/(...)/python3.5/site-packages/joblib/numpy_pickle.py", line 578, in load
   obj = _unpickle(fobj, filename, mmap_mode)
 File "/home/(...)/python3.5/site-packages/joblib/numpy_pickle.py", line 508, in _unpickle
   obj = unpickler.load()
 File "/usr/lib/python3.5/pickle.py", line 1039, in load
   dispatch[key[0]](self)
KeyError: 0

The same application both pickles and unpickles the object, so the versions of scikit-learn, joblib and other libraries are the same. I'm not sure where to start debugging, given the vague error. Any ideas or pointers?

Answer 1

The solution to this was pretty banal: Without being aware of it I was using the version of joblib in sklearn.externals.joblib for the pickling, but a newer version of joblib for unpickling the object. The problem was resolved when I used the newer version of joblib for both tasks.

Answer 2

With me, happened that I exported the model using from sklearn.externals import joblib and tried to load using import joblib.

Answer 3

Mine was interesting. I was working with git-lfs and thus the files had been changed and joblib couldn't open them. So I needed to run git lfs pull to get actual files. So if you are using compatible joblib versions, make sure your files are not changed somehow!