How to free memory of python deleted object?

Question

It seems python3.5 does not completely free memory of any deleted object, this may because of python internally maintaining some kind of memroy pool for reusing purpose, however, I don't want to reuse them, and I want to free them to make memory available for other programs running on linux.

>>> psutil.Process().memory_info().rss / 2**20
11.47265625
>>> d = {x:x for x in range(10**7)}
>>> psutil.Process().memory_info().rss / 2**20
897.1796875
>>> del d
>>> gc.collect()
0
>>> psutil.Process().memory_info().rss / 2**20
15.5859375

This is just a toy example, the real problem is on a running server, taking 20GB of unfreeable memory.

here is another example: (wd1 is a dict of dict with string keys)

>>> psutil.Process().memory_info().rss / 2**20
28.1796875
>>> wd1 = {x:{i:i for i in d} for x in k}
>>> psutil.Process().memory_info().rss / 2**20
682.78125
>>> del wd1
>>> psutil.Process().memory_info().rss / 2**20
186.21484375

Answer 1

Once you delete an object it is available to garbage collected rather than deleted immediately - so just give it some time and it will free up or trigger a gc.collect() to speed things up.

python.exe
Python 3.5.1 (v3.5.1:37a07cee5969, Dec  6 2015, 01:38:48) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil
>>> import gc
>>> psutil.Process().memory_info().rss / 2**20
13.2890625
>>> d = {x:x for x in range(10**7)}
>>> psutil.Process().memory_info().rss / 2**20
359.13671875
>>> del d
>>> psutil.Process().memory_info().rss / 2**20
13.5234375
>>> gc.collect()
0
>>> psutil.Process().memory_info().rss / 2**20
13.4375
>>>

Just for reference the Python 3 shell is actually more like ipython 2 in that there is a certain amount of storage taken up with history, etc., just for reference:

Python 3.5.1 (v3.5.1:37a07cee5969, Dec  6 2015, 01:38:48) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil
>>> psutil.Process().memory_info().rss / 2**20
13.1875
>>> psutil.Process().memory_info().rss / 2**20
13.20703125
>>> psutil.Process().memory_info().rss / 2**20
13.20703125
>>> psutil.Process().memory_info().rss / 2**20
13.20703125
>>> psutil.Process().memory_info().rss / 2**20
13.20703125
>>> 22*3
66
>>> psutil.Process().memory_info().rss / 2**20
13.25390625
>>> import gc
>>> psutil.Process().memory_info().rss / 2**20
13.25390625
>>> gc.collect()
0
>>> psutil.Process().memory_info().rss / 2**20
13.171875
>>>

Next Morning to check if doing dict update in a function is different:

>>> psutil.Process().memory_info().rss / 2**20
13.1484375
>>> D = {}
>>> psutil.Process().memory_info().rss / 2**20
13.1484375
>>> def UpdateD(d, v):
...     """ Add the text and value for v to dict d """
...     d[v] = str(v)
...
>>> psutil.Process().memory_info().rss / 2**20
13.16015625
>>> for x in range(10**7):
...     UpdateD(D, x)
...
>>> psutil.Process().memory_info().rss / 2**20
666.6328125
>>> del D
>>> psutil.Process().memory_info().rss / 2**20
10.765625
>>> gc.collect()
0
>>> psutil.Process().memory_info().rss / 2**20
12.8984375
>>>

So it looks like your production code might be hanging onto a reference that you still have to track down.