Pickle , read in data , unsupported pickle protocol: 3 python 2.7

Question

Beginner here, want to read in data with the file ending p.

My code looks like this :

import pickle

training_file = "/home/sk/CarND-Traffic-Sign-Classifier-Project/train.p"
testing_file = "/home/sk/CarND-Traffic-Sign-Classifier-Project/test.p"
with open(training_file, mode='rb') as f:
    train = pickle.load(f)
with open(testing_file, mode='rb') as f:
    test = pickle.load(f)

I get the following error:

ValueError: unsupported pickle protocol: 3

Can someone point out how i can fix it, either changing protocol or reading in the data some other way ?

Answer 1

Had the same issue when i created a pickle file using python3 and then tried loading it in python2. Try running your program with python3 or try creating a pickle file using python2.

Answer 2

Pickle uses different protocols to convert your data to a binary stream.

In python 2 there are 3 different protocols (0, 1, 2) and the default is 0. In python 3 there are 5 different protocols (0, 1, 2, 3, 4) and the default is 3. You must specify in python 3 a protocol lower than 3 in order to be able to load the data in python 2. You can specify the protocol parameter when invoking pickle.dump.

It seems as those files was created with a protocol >=3 (probably 3). So the only option you get is to load it into python 3 and then dump it with a lower protocol.

Answer 3

Evidently pickle protocol 3 was used in whatever python 3 code pickled the object. You can't unpickle with protocol 3 in python 2. You could however write a short python 3 program that loads it and then dumps it with protocol = 2. Then you can load them in python 2.

https://docs.python.org/2/library/pickle.html#usage