How to find total amount of memory used by python process/object in windows

Question

I have some script that loads a lot of data to memory. I want to know how efficient the data stored in memory. So, I want to be able to know how many memory was used by python before I loaded data, and after I loaded data. Also I wondering, if it is some way to check memory usage of complex object. Let say i have nested dictionary with different types of data inside. How can i know how many memory used by all data in this dictionary. Thanks, Alex

Answer 1

As far as I know there is no easy way to see what the memory consumption of a certain object is. It would be a non-trivial thing to do because references could be shared among objects.

Here are my two favourite workarounds:

1. Use the process manager. Have the program pause for before allocation. Write down the memory used before allocation. Allocate. Write down memory after allocation. It's a low-tech method but it works.
2. Alternatively you can use pickle.dump to serialize your data structure. The resulting pickle will be comparable (not identical!) in size to the space needed to store the data structure in memory. For better results, use the binary pickle protocol.

Answer 2

In order to analyze how much memory an object uses, you could use Pympler:

>>> from pympler import asizeof
>>> obj = dict(nested=dict(trash=[1,2,3]))
>>> asizeof.asizeof(obj)
800
>>> asizeof.asizeof(obj['nested'])
480
>>> asizeof.asizeof(obj['nested']['trash'])
160
>>> asizeof.asizeof(obj['nested']['trash'][0])
24