I've developed a program in python, organizing data in flattened dictionaries. As the size of the dict increases, the program becomes slower due to a intensive keys search. Looking at nested dictionaries structure, seems to me that a "hierarchical" approach may speed up the search of the keys. Am I wrong?
Is the nested dict:
nested_dict = { 'dictA': {'key_1': 'value_1', 'key_2': 'value_2'},
'dictB': {'key_3': 'value_3', 'key_4': 'value_4', 'key_5': 'value_5'},
...
'dictZ': {'key_m': 'value_m', 'key_n': 'value_n'}}
faster than the flattened dict:
dictionary = {'key_1': 'value_1',
'key_2': 'value_2',
...
'key_n': 'value_n'}
Edit: Added few code examples
Below a piece of the code generally I use. The Program is quite big, so there isn't a specific code to be evaluated
Assignment:
dictionary['key_1'] = dictionary2['key_a']
dictionary['key_3'] = dictionary2['key_a']*dictionary['key_4']
Conditional statement:
if( (0 == dictionary['key_1']) and
(dictionary2['key_b'] >= dictionary['key_3']) ):
Looking at nested dictionaries structure, seems to me that a "hierarchical" approach may speed up the search of the keys. Am I wrong?
Yes :-)
The flat dict space has O(1) lookup irrespective of size. That is what makes hash tables so attractive as a data structure.
Adding hierarchy just adds extra hashing steps and lookup steps.
In some contexts, containers do get some cache locality benefits by being small, but in Python the containers have references to objects that are scattered all over memory, so compactness doesn't help much.
In addition, Python is an interpreted language, so adding a extra layer of lookups also entails more opcode evaluations. This would swamp any possible benefits for compactness.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With