Load pickle file(comes from python3) in python2

Question

I have a pickle file, with

>>> with open("wikilinks.pickle", "rb") as f:
...     titles, links = pickle.load(f)
... 
>>> len(titles)
13421

I can load it in python3. However, when I try to load it in python2, I get this message: Traceback (most recent call last):

  File "<stdin>", line 2, in <module>
  File "/usr/lib/python2.7/pickle.py", line 1378, in load
    return Unpickler(file).load()
  File "/usr/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/lib/python2.7/pickle.py", line 886, in load_proto
    raise ValueError, "unsupported pickle protocol: %d" % proto
ValueError: unsupported pickle protocol: 3

So how to load it in python2?

Answer 1

Read in python3 and save it as python2 format

#!/usr/bin/env python3

import pickle

with open("a.pkl", "rb") as f:
    w = pickle.load(f)

pickle.dump(w, open("a_py2.pkl","wb"), protocol=2)

Then you can load a_py2.pkl in python2.

Answer 2

The default pickling protocol used by default in Python 3 is incompatible with the protocol used by Python 2. However, if you can modify the code that produces wikilink.pickle, you can tell it to use one of the earlier protocols (0, 1, or 2) which Python 2 will understand just fine.

Be warned, though, that Python 3 strings will come up as Unicode strings in Python 2, which may be surprising in some use cases.

$ python3
Python 3.4.1 (default, May 19 2014, 17:23:49) 
[GCC 4.9.0 20140507 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> pickle.dumps([1, 2, 'abc'], 2)
b'\x80\x02]q\x00(K\x01K\x02X\x03\x00\x00\x00abcq\x01e.'
>>> 
$ python 
Python 2.7.8 (default, Jul  1 2014, 17:30:21) 
[GCC 4.9.0 20140604 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cPickle
>>> cPickle.loads('\x80\x02]q\x00(K\x01K\x02X\x03\x00\x00\x00abcq\x01e.')
[1, 2, u'abc']