Unpickling classes from Python 3 in Python 2

Question

If a Python 3 class is pickled using protocol 2, it is supposed to work in Python 2, but unfortunately, this fails because the names of some classes have changed.

Assume we have code called as follows.

Sender

pickle.dumps(obj,2)

Receiver

pickle.loads(atom)

To give a specific case, if obj={}, then the error given is:

ImportError: No module named builtins

This is because Python 2 uses __builtin__ instead.

The question is the best way to fix this problem.

Answer 1

This problem is Python issue 3675. This bug is actually fixed in Python 3.11.

If we import:

from lib2to3.fixes.fix_imports import MAPPING

MAPPING maps Python 2 names to Python 3 names. We want this in reverse.

REVERSE_MAPPING={}
for key,val in MAPPING.items():
    REVERSE_MAPPING[val]=key

We can override the Unpickler and loads

class Python_3_Unpickler(pickle.Unpickler):
    """Class for pickling objects from Python 3"""
    def find_class(self,module,name):
        if module in REVERSE_MAPPING:
            module=REVERSE_MAPPING[module]
        __import__(module)
        mod = sys.modules[module]
        klass = getattr(mod, name)
        return klass

def loads(str):
    file = pickle.StringIO(str)
    return Python_3_Unpickler(file).load()  

We then call this loads instead of pickle.loads.

This should solve the problem.