TypeError on pickle.load with class derived from SimpleNamespace

Question

Consider this Python (version 3.5) code:

import pickle
from types import SimpleNamespace

class MyClass1(list):
    def __init__(self, x):
        self.append(x)

class MyClass2(SimpleNamespace):
    def __init__(self, x):
        self.x = x

a0 = SimpleNamespace(x=99)
a1 = MyClass1(99)
a2 = MyClass2(99)

print('* SimpleNamespace:', pickle.loads(pickle.dumps(a0)))
print('* MyClass1:', pickle.loads(pickle.dumps(a1)))
print('* MyClass2:', pickle.loads(pickle.dumps(a2)))

This works fine for the first two (a0 and a1), but upon handling a2 I get an error:

* SimpleNamespace: namespace(x=99)
* MyClass1: [99]
Traceback (most recent call last):
  File "./picktest.py", line 20, in <module>
    print('* MyClass2:', pickle.loads(pickle.dumps(a2)))
TypeError: __init__() missing 1 required positional argument: 'x'

Observations:

• SimpleNamespace can be pickled.
• A class derived from a different type (list) can be (un)pickled.
• A class derived from SimpleNamespace cannot be unpickled if it has an __init__ requiring arguments. (The error occurs in pickle.loads)

Note that I tried replacing self.x = x by just pass and that didn't change anything.

Is there a way to make this work other than by reimplementing MyClass2 without inheritance?

Answer 1

The issue is that SimpleNamespace defines a __reduce__ that pickle uses to unpickle your object. The __reduce__ defined in SimpleNamespace is not, however, consistent with your __init__. You can define your own __reduce__ to skirt around this:

import pickle
from types import SimpleNamespace

class MyClass1(list):

    def __init__(self, x):
        self.append(x)

class MyClass2(SimpleNamespace):

    def __init__(self, x):
        self.x = x

    def __reduce__(self):
        return (self.__class__, (self.x,))

a0 = SimpleNamespace(x=99)
a1 = MyClass1(99)
a2 = MyClass2(99)

print('* SimpleNamespace:', pickle.loads(pickle.dumps(a0))) 
print('* MyClass1:', pickle.loads(pickle.dumps(a1)))
print('* MyClass2:', pickle.loads(pickle.dumps(a2)))