dynamic memory allocation in python

Question

I created a big multidimensional array M with np.zeros((1000,1000)). After certain number of operations, I don't need it anymore. How can I free a RAM dynamically during program's execution? Does M=0 do it for me?

Answer 1

In general you can't. Even if you remove all the references to an object, it is left to the python implementation to re-use or free the memory. On CPython you could call gc.collect() to force a garbage collection run. But while that may reclaim memory, it doesn't necessarily return it to the OS.

But: numpy is an extension module that does its own thing, and manages its own memory.

When I monitor the memory usage of a python process, I see the RAM usage (Resident Set Size) going down after del(M)

In [1]: import numpy as np

In [2]: M = np.zeros((1000,1000))

In [3]: del(M)

In [4]: 

Just after starting IPython:

slackbox:~> ps -u 77778
USER     PID %CPU %MEM    VSZ   RSS TT  STAT STARTED    TIME COMMAND
rsmith 77778  0.0  0.5 119644 22692  0  S+    2:37PM 0:00.39 /usr/local/bin/py

After importing numpy (1):

slackbox:~> ps -u 77778
USER     PID %CPU %MEM    VSZ   RSS TT  STAT STARTED    TIME COMMAND
rsmith 77778  1.0  0.8 168548 32420  0  S+    2:37PM 0:00.49 /usr/local/bin/py

After creating the array (2):

slackbox:~> ps -u 77778
USER     PID %CPU %MEM    VSZ   RSS TT  STAT STARTED    TIME COMMAND
rsmith 77778  0.0  1.0 176740 40328  0  S+    2:37PM 0:00.50 /usr/local/bin/py

After the call to del (3):

slackbox:~> ps -u 77778
USER     PID %CPU %MEM    VSZ   RSS TT  STAT STARTED    TIME COMMAND
rsmith 77778  0.0  0.8 168548 32496  0  S+    2:37PM 0:00.50 /usr/local/bin/py
slackbox:~> 

So in this case using del() can reduce the amount of RAM used.

Note that there is an exception to this with numpy. Numpy can use memory allocated by another extension library. In that case the numpy object is marked that numpy doesn't own the memory, and freeing it is left to the other library.

Answer 2

Use the del statement:

del M

And by the way, a float64 array of shape (1000, 1000) takes only 7 Mb. If you're having memory problems, it's likely that the problem is elsewhere.