Python Pickling Slots Error

Question

I have a large instance that I've been pickling just fine, but recently I started getting this error when I tried to dump it:

  File "/usr/lib/python2.6/copy_reg.py", line 77, in _reduce_ex
    raise TypeError("a class that defines __slots__ without "
TypeError: a class that defines __slots__ without defining __getstate__ cannot be pickled

I don't understand this error, since all my classes seem to define a __getstate__ method, and none seem to define __slots__. I'm having trouble isolating the change I made the triggered this error.

I can only assume there's some object nested deep within my instance that's causing this. Is there any way to get more info? How do I find the class of the exact object that's triggering this error?

Answer 1

Use a binary protocol for your pickling (instead of the old ASCII one you seem to be defaulting to) and you'll be fine. Observe:

>>> class ws(object):
...   __slots__ = 'a', 'b'
...   def __init__(self, a=23, b=45): self.a, self.b = a, b
... 
>>> x = ws()
>>> import pickle
>>> pickle.dumps(x, -1)
'\x80\x02c__main__\nws\nq\x00)\x81q\x01N}q\x02(U\x01aq\x03K\x17U\x01bq\x04K-u\x86q\x05b.'
>>> pickle.dumps(x)
Traceback (most recent call last):
    [[snip]]
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/copy_reg.py", line 77, in _reduce_ex
    raise TypeError("a class that defines __slots__ without "
TypeError: a class that defines __slots__ without defining __getstate__ cannot be pickled
>>> 

As you see, the -1 protocol (which means "the best, fastest and most compact protocol") works just fine, while the default 0 protocol (the old ascii protocol designed to be compatible all the way back to Python 1.5 and earlier) gives exactly the exception you have observed.

Besides, -1 will be faster and produce more compact results -- you just need to ensure you correctly save and restore the binary strings it produces (so, for example, if you're pickling to file, be sure to open the latter for wb, not just w).

If for some reason this all-around-win solution is not available to you, there are hacks and tricks (e.g., subclass pickle.Pickler, use directly an instance of your subclass rather than the base one as pickle.dumps does, override the save method so it traces the type(obj) before delegating to the superclass), but upgrading to the right, most up-to-date protocol (-1 is guaranteed to be, on any given Python version, the most advanced one that version supports) would be a good idea anyway, if at all feasible.

Answer 2

I had this problem also but with some already pickled data in the Old ASCII protocol . You can use these methods to adapt slot to your object:

class MyAlreadyPickeldObjectWithslots(object):
    ___slots__= ("attr1","attr2",....)
    def __getstate__(self):
        return dict([(k, getattr(self,k,None)) for k in self.__slots__])

    def __setstate__(self,data):
        for k,v in data.items():
            setattr(self,k,v)

That's can be usefull to gain amount of RAM