Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to convert defaultdict of defaultdicts [of defaultdicts] to dict of dicts [of dicts]?

Using this answer, I created a defaultdict of defaultdicts. Now, I'd like to turn that deeply nested dict object back into an ordinary python dict.

from collections import defaultdict  factory = lambda: defaultdict(factory) defdict = factory() defdict['one']['two']['three']['four'] = 5  # defaultdict(<function <lambda> at 0x10886f0c8>, { #             'one': defaultdict(<function <lambda> at 0x10886f0c8>, { #                 'two': defaultdict(<function <lambda> at 0x10886f0c8>, { #                     'three': defaultdict(<function <lambda> at 0x10886f0c8>, { #                         'four': 5})})})}) 

I assume this is not the right solution:

import json  regdict = json.loads(json.dumps(defdict))  # {u'one': {u'two': {u'three': {u'four': 5}}}} 

Also, this answer is inadequate since it does not recurse on the nested dict(s).

like image 701
samstav Avatar asked Oct 21 '14 21:10

samstav


People also ask

What is the difference between Defaultdict and dict?

The main difference between defaultdict and dict is that when you try to access or modify a key that's not present in the dictionary, a default value is automatically given to that key . In order to provide this functionality, the Python defaultdict type does two things: It overrides .

How does Defaultdict work in Python?

A defaultdict works exactly like a normal dict, but it is initialized with a function (“default factory”) that takes no arguments and provides the default value for a nonexistent key. A defaultdict will never raise a KeyError. Any key that does not exist gets the value returned by the default factory.


2 Answers

You can recurse over the tree, replacing each defaultdict instance with a dict produced by a dict comprehension:

def default_to_regular(d):     if isinstance(d, defaultdict):         d = {k: default_to_regular(v) for k, v in d.items()}     return d 

Demo:

>>> from collections import defaultdict >>> factory = lambda: defaultdict(factory) >>> defdict = factory() >>> defdict['one']['two']['three']['four'] = 5 >>> defdict defaultdict(<function <lambda> at 0x103098ed8>, {'one': defaultdict(<function <lambda> at 0x103098ed8>, {'two': defaultdict(<function <lambda> at 0x103098ed8>, {'three': defaultdict(<function <lambda> at 0x103098ed8>, {'four': 5})})})}) >>> default_to_regular(defdict) {'one': {'two': {'three': {'four': 5}}}} 
like image 127
Martijn Pieters Avatar answered Sep 20 '22 15:09

Martijn Pieters


What you're actually trying to do is pickle your recursive defaultdict. And you don't care whether you get back a dict or a defaultdict when unpickling.

While there are a number of ways to solve this (e.g., create a defaultdict subclass with its own pickling, or explicitly override the default one with copyreg), there's one that's dead trivial.

Notice the error you get when you try it:

>>> pickle.dumps(defdict) PicklingError: Can't pickle <function <lambda> at 0x10d7f4c80>: attribute lookup <lambda> on __main__ failed 

You can't pickle lambda-defined functions, because they're anonymous, meaning there's no way they could ever be unpickled.

But there is literally no reason this function needs to be defined by lambda. In particular, you don't even want it to be anonymous, because you're explicitly giving it a name. So:

def factory(): return defaultdict(factory) 

And you're done.

Here it is in action:

>>> from collections import defaultdict >>> def factory(): return defaultdict(factory) >>> defdict = factory() >>> defdict['one']['two']['three']['four'] = 5 >>> import pickle >>> pickle.dumps(defdict) b'\x80\x03ccollections\ndefaultdict\nq\x00c__main__\nfactory\nq\x01\x85q\x02Rq\x03X\x03\x00\x00\x00oneq\x04h\x00h\x01\x85q\x05Rq\x06X\x03\x00\x00\x00twoq\x07h\x00h\x01\x85q\x08Rq\tX\x05\x00\x00\x00threeq\nh\x00h\x01\x85q\x0bRq\x0cX\x04\x00\x00\x00fourq\rK\x05ssss.' 

There are other cases where using lambda instead of def for no good reason will cause problems—you can't introspect your functions as well at runtime, you get worse tracebacks in the debugger, etc. Use lambda when you want an inherently-anonymous function, or a function you can define in the middle of an expression, but don't use it to save three characters of typing.

like image 28
abarnert Avatar answered Sep 20 '22 15:09

abarnert