Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to stop attributes from being pickled in Python [duplicate]

Tags:

I am using gnosis.xml.pickle to convert an object of my own class to xml. The object is initialized so that:

self.logger = MyLogger() 

But when I do dump the object to a string I get an exception stating that the pickler encountered an unpickleable type (thread.lock).

Is there a way to 'tag' the logger attribute so that pickler will know not to try and pickle that attribute?

like image 855
Ries Avatar asked Jun 08 '10 17:06

Ries


People also ask

What objects Cannot be pickled in Python?

With pickle protocol v1, you cannot pickle open file objects, network connections, or database connections.

Is Python pickling slow?

Pickle on the other hand is slow, insecure, and can be only parsed in Python. The only real advantage to pickle is that it can serialize arbitrary Python objects, whereas both JSON and MessagePack have limits on the type of data they can write out.

Can all Python objects be pickled?

Python pickle module is used for serializing and de-serializing a Python object structure. Any object in Python can be pickled so that it can be saved on disk.

Why is pickling done in Python?

Pickle in Python is primarily used in serializing and deserializing a Python object structure. In other words, it's the process of converting a Python object into a byte stream to store it in a file/database, maintain program state across sessions, or transport data over the network.


2 Answers

You can define two methods, __getstate__ and __setstate__, to your class to override the default pickling behavior.

http://docs.python.org/library/pickle.html#object.__getstate__

__getstate__ should return a dict of attributes that you want to pickle.

def __getstate__(self):     d = dict(self.__dict__)     del d['logger']     return d 

__setstate__ should setup your object with the provided dict.

def __setstate__(self, d):     self.__dict__.update(d) # I *think* this is a safe way to do it 

Note that __init__ won't be called when unpickling so you'll have to create your logger in __setstate__

like image 157
FogleBird Avatar answered Sep 30 '22 17:09

FogleBird


This might be a better solution since it will allow an object created via copy.deepcopy to still have a self.logger:

def __getstate__(self):     d = self.__dict__.copy()     if 'logger' in d:         d['logger'] = d['logger'].name     return d  def __setstate__(self, d):     if 'logger' in d:         d['logger'] = logging.getLogger(d['logger'])     self.__dict__.update(d) 
like image 42
iamjk Avatar answered Sep 30 '22 16:09

iamjk