Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Unpickling a python 2 object with python 3

People also ask

Does code Python 2 work with Python 3?

Python 2.6 includes many capabilities that make it easier to write code that works on both 2.6 and 3. As a result, you can program in Python 2 but using certain Python 3 extensions... and the resulting code works on both.

How do you Unpickle an object in Python?

You can use the loads() method to unpickle an object that is pickled in the form of a string using the dumps() method, instead of being stored on a disk via the the dump() method. In the following example the car_list object that was pickled to a car_list string is unpickled via the loads() method.

Can you pickle objects Python?

Python comes with a built-in package, known as pickle , that can be used to perform pickling and unpickling operations. Pickling and unpickling in Python is the process that is used to describe the conversion of objects into byte streams and vice versa - serialization and deserialization, using Python's pickle module.

How do I install a pickle file in Python 3?

Python Pickle load You have to use pickle. load() function to do that. The primary argument of pickle load function is the file object that you get by opening the file in read-binary (rb) mode. Simple!


You'll have to tell pickle.load() how to convert Python bytestring data to Python 3 strings, or you can tell pickle to leave them as bytes.

The default is to try and decode all string data as ASCII, and that decoding fails. See the pickle.load() documentation:

Optional keyword arguments are fix_imports, encoding and errors, which are used to control compatibility support for pickle stream generated by Python 2. If fix_imports is true, pickle will try to map the old Python 2 names to the new names used in Python 3. The encoding and errors tell pickle how to decode 8-bit string instances pickled by Python 2; these default to ‘ASCII’ and ‘strict’, respectively. The encoding can be ‘bytes’ to read these 8-bit string instances as bytes objects.

Setting the encoding to latin1 allows you to import the data directly:

with open(mshelffile, 'rb') as f:
    d = pickle.load(f, encoding='latin1') 

but you'll need to verify that none of your strings are decoded using the wrong codec; Latin-1 works for any input as it maps the byte values 0-255 to the first 256 Unicode codepoints directly.

The alternative would be to load the data with encoding='bytes', and decode all bytes keys and values afterwards.

Note that up to Python versions before 3.6.8, 3.7.2 and 3.8.0, unpickling of Python 2 datetime object data is broken unless you use encoding='bytes'.


Using encoding='latin1' causes some issues when your object contains numpy arrays in it.

Using encoding='bytes' will be better.

Please see this answer for complete explanation of using encoding='bytes'