Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Memory consumption of a dictionary: set a value None vs. delete the item

Tags:

python

I understand that del d[key] will delete the key-value pair, whereas d[key]=None only de-references the value.

However, in terms of memory management, is there any difference? Does setting a value None trigger garbage collection immediately, assuming that there is no other variable referencing this value?

I ran a little experiment:

In [74]: import sys
In [75]: a = {'a': 'blah'}

In [76]: sys.getsizeof(a)
Out[76]: 280

In [77]: a['a'] = None

In [79]: sys.getsizeof(a)
Out[79]: 280

In [80]: del a['a']

In [81]: sys.getsizeof(a)
Out[81]: 280

Not sure if the approach is valid, but it seems no difference in terms of the size of the dictionary at all. I must miss something here.

like image 614
MLister Avatar asked Nov 10 '12 14:11

MLister


People also ask

Which function or statement delete the dictionary from the memory?

Python's del statement is used to delete variables and objects in the Python program. Iterable objects such as user-defined objects, lists, set, tuple, dictionary, variables defined by the user, etc. can be deleted from existence and from the memory locations in Python using the del statement.

How much memory does a Python dictionary take?

In other words, our dictionary, with nothing in it at all, consumes 240 bytes. Not bad; given how often dictionaries are used in Python, it's good to know that they don't normally consume that much memory.

How much memory does a dictionary use?

That means you need about 1 MB of memory to store the entire dictionary. Of course you can deflate it (zip) and store in memory, with a compression rate of about 90%. That means 100 KB.


2 Answers

sys.getsizeof measures the size of the dict itself; not the size of the values it contains.

None is an object. It requires some memory.

To find the size of a dict including the size of the values it contains, you could use pympler:

In [26]: import pympler.asizeof as asizeof

In [27]: asizeof.asizeof({'a': None})
Out[27]: 168

In [28]: asizeof.asizeof({})
Out[28]: 136

In [29]: import sys

In [30]: 
In [31]: sys.getsizeof({})
Out[31]: 136

In [34]: sys.getsizeof({'a':None})
Out[34]: 136
like image 188
unutbu Avatar answered Nov 12 '22 08:11

unutbu


unutbu is correct. But also the python garbage collector can be a little slow on the uptake sometimes.

Calling del on an object and removing all references to that object means that the garbage collector can collect it whenever it feels ready. This will not usually result in an immediate decrease in the amount of memory used.

you can make use of the gc module to force collection and look at what the garbage collector actually sees.

like image 29
Sheena Avatar answered Nov 12 '22 07:11

Sheena