Why can't I pickle this object?

Question

I have a class (below):

class InstrumentChange(object):
    '''This class acts as the DTO object to send instrument change information from the
       client to the server. See InstrumentChangeTransport below
    '''
    def __init__(self, **kwargs):
        self.kwargs = kwargs
        self._changed = None

    def _method_name(self, text):
        return text.replace(' ','_').lower()

    def _what_changed(self):
        ''' Denotes the column that changed on the instrument returning the column_name of what changed.'''
        if not self._changed:
            self._changed = self._method_name(self.kwargs.pop('What Changed'))

        return self._changed

    def __getattr__(self, attr):
        for key in self.kwargs.iterkeys():
            if self._method_name(key) == attr:
                return self.kwargs[key]

    def __str__(self):
        return "Instrument:%s" % self.kwargs

    __repr__ = __str__

    what_changed = property(_what_changed)

When I run the following test:

def test_that_instrumentchangetransport_is_picklable(self):
        test_dict = {'Updated': 'PAllum', 'Description': 'BR/EUR/BRAZIL/11%/26/06/2017/BD',
        'Ask Q': 500, 'Bbg': 'On', 'C Bid': 72.0, 'Benchmark': 'NL/USD/KKB/7.000%/03/11/2009/BD',
        'ISIN': 'XS0077157575', 'Bid YTM': 0.0, 'Bid Q': 100, 'C Ask': 72.25, 'Ask YTM': 0.0, 'Bid ASW': 0.0,
        'Position': 1280000, 'What Changed': 'C Bid', 'Ask ASW': 0.0}
        ins_change = InstrumentChangeTransport(**test_dict)
        assert isinstance(ins_change, InstrumentChangeTransport)

        # Create a mock filesystem object
        file = open('testpickle.dat', 'w')
        file = Mock()
        pickle.dump(ins_change, file)

I get:

Traceback (most recent call last):
  File "c:\python23\lib\site-packages\nose-0.11.0-py2.3.egg\nose\case.py", line 183, in runTest
    self.test(*self.arg)
  File "C:\Code\branches\demo\tests\test_framework.py", line 142, in test_that_instrumentchangetransport_is_picklable
    pickle.dump(ins_change, file)
  File "C:\Python23\Lib\copy_reg.py", line 83, in _reduce_ex
    dict = getstate()
TypeError: 'NoneType' object is not callable

I've looked at the pickle docs, but I don't quite get it.

Any ideas?

Ben

Answer 1

Your code has several minor "side" issues: the sudden appearance of a 'Transport' in the class name used in the test (it's not the class name that you're defining), the dubious trampling over built-in identifier file as a local variable (don't do that -- it doesn't hurt here, but the habit of trampling over built-in identifiers will cause mysterious bugs one day), the misuses of Mock that has already been noted, the default use of the slowest, grungiest pickling protocol and text rather than binary for the pickle file.

However, at the heart, as @coonj says, is the lack of state control. A "normal" class doesn't need it (because self.__dict__ gets pickled and unpickled by default in classes missing state control and without other peculiarities) -- but since you're overriding __getattr__ that doesn't apply to your class. You just need two more very simple methods:

def __getstate__(self): return self.__dict__
def __setstate__(self, d): self.__dict__.update(d)

which basically tell pickle to treat your class just like a normal one, taking self.__dict__ as representing the whole of the instance state, despite the existence of the __getattr__.

Answer 2

It is failing because it can't find __getstate__() for your object. Pickle needs these to determine how to pickle/unpickle the object. You just need the __getstate__() and __setstate__() methods.

See the TextReader example in the docs: http://docs.python.org/library/pickle.html

Update: I just looked at the sourceforge page for the Mock module, and I think you are also using it incorrectly. You are mocking a file-object, but when pickle tries to read from it, it won't get anything back which is why getattr() returns none.