Cannot pickle empty ordered_set

Question

I'm using version 1.2 (currently the latest) of the ordered_set module linked to from this answer. I've been getting some weird behavior and have traced it to this:

from ordered_set import OrderedSet
import pickle

os_orig = OrderedSet()
print os_orig # 'OrderedSet'
print os_orig.items # '[]'
pickled = pickle.dumps(os_orig)
loaded = pickle.loads(pickled)
print loaded

Which raises AttributeError: 'OrderedSet' object has no attribute 'items'. Everything goes fine if the OrderedSet is not empty.

Unfortunately I am in over my head here when it comes to pickle--what is going wrong?

EDIT: I should add that the module seems to support pickle. From the README: "added a __getstate__ and __setstate__ so it can be pickled"

Answer 1

The pickling support for OrderedSet breaks when the set is empty, because the state returned by __getstate__ is essentially empty:

>>> OrderedSet().__getstate__()
[]

The pickle module ends up not calling __setstate__ when loading the pickle again because __getstate__'s return value is empty. Not calling __setstate__ means the OrderedSet.__init__() method never gets called and you have a broken object. See the __setstate__ documenation:

Note: For new-style classes, if __getstate__() returns a false value, the __setstate__() method will not be called.

An empty list is a false value.

The author must've tested only with pickling non-empty OrderedSet() instances, which works fine.

You can fix the issue by replacing the __getstate__ and __setstate__ methods:

def __getstate__(self):
    return (list(self),)
OrderedSet.__getstate__ = __getstate__

def __setstate__(self, state):
    if isinstance(state, tuple):
        state = state[0]
    self.__init__(state)
OrderedSet.__setstate__ = __setstate__

Now a non-empty, 1-element tuple is returned, forcing pickle to call __setstate__ even for the empty set. The __setstate__ will still accept the previous pickle format, a list object.

I've reported this as a bug with the project, since closed as resolved.