size of nested dictionary and list

Question

I have dictionary with following format:

Demo code:

>>> import pprint
>>> pprint.pprint(data)
{'lookup': {'F01': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F02': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F03': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F04': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F05': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F06': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F07': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F08': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F09': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F10': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F11': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F12': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }'},
 'sequence': ['F01',
              'F02',
              'F03',
              'F04',
              'F05',
              'F06',
              'F07',
              'F08',
              'F09',
              'F10',
              'F11',
              'F12']}
>>> import sys
>>> sys.getsizeof(data)
136
>>> sys.getsizeof(data["sequence"])
80
>>> sys.getsizeof(data["lookup"])
520
>>> 

I can not get how nested dictionary is store in memory because size if data is 136 bytes and size of data["sequence"] is 80 bytes and also size of data["lookup"] is 520 bytes.

Also when I do type casting of variable data from dictionary to string then size of string variable is 3587 bytes.

Demo Code:

>>> data_str = str(data)
>>> sys.getsizeof(data_str)
3587

Can any on explain me why?

Answer 1

Dictionaries and lists store references (like every other standard container in Python). sys.getsizeof() doesn't follow references, it gives you the memory footprint of the C structure only. The references are C pointers; their size depends on your specific platform.

Converting a dictionary to a string recursively converts the contents to (repr()) strings too, so all those references are then dereferenced and included in the output. Note that this is not an accurate reflection of the memory size for the original object; strings contain characters, it depends on your exact Python version, OS and range of Unicode codepoints used how much memory each character takes, and the character count has a non-linear relationship to the actual object being reflected.

If you want to know the memory footprint of a dictionary with the contents you need to do so recursively. Take into account that a dictionary can contain references to itself (directly or indirectly), or that any object can have multiple references to it and should only be counted once. I'd use the id() function to track what objects have already been handled.

There are already several posts on Stack Overflow that discuss calculating memory size for containers using recursion or other tools, see Deep version of sys.getsizeof, Python deep getsizeof list with contents?, and Memory-usage of dictionary in Python? for some examples.