I have a defaultdict that looks like this:
dict1 = defaultdict(lambda: defaultdict(int))
The problem is, I can't pickle it using cPickle. One of the solution that I found here is to use module-level function instead of a lambda. My question is, what is module-level function? How can I use the dictionary with cPickle?
Just use the same syntax as you would on a regular dictionary. del mydict['stock2'] for example, or mydict. pop('stock2') if you want the value returned at the same time. Note that you don't need to be using a defaultdict at all in your example.
A defaultdict works exactly like a normal dict, but it is initialized with a function (“default factory”) that takes no arguments and provides the default value for a nonexistent key. A defaultdict will never raise a KeyError. Any key that does not exist gets the value returned by the default factory.
A defaultdict can be created by giving its declaration an argument that can have three values; list, set or int. According to the specified data type, the dictionary is created and when any key, that does not exist in the defaultdict is added or accessed, it is assigned a default value as opposed to giving a KeyError .
In addition to Martijn's explanation:
A module-level function is a function which is defined at module level, that means it is not an instance method of a class, it's not nested within another function, and it is a "real" function with a name, not a lambda function.
So, to pickle your defaultdict
, create it with module-level function instead of a lambda function:
def dd(): return defaultdict(int) dict1 = defaultdict(dd) # dd is a module-level function
than you can pickle it
tmp = pickle.dumps(dict1) # no exception new = pickle.loads(tmp)
Pickle wants to store all the instance attributes, and defaultdict
instances store a reference to the default
callable. Pickle recurses over each instance attribute.
Pickle cannot handle lambdas; pickle only ever handles data, not code, and lambdas contain code. Functions can be pickled, but just like class definitions only if the function can be imported. A function defined at the module level can be imported. Pickle just stores a string in that case, the full 'path' of the function to be imported and referenced when unpickling again.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With