Menu
What's a good way to estimate the memory footprint of an object?
Conversely, what's a good way to measure the footprint?
For example, say I have a dictionary whose values are lists of integer,float tuples:
d['key'] = [ (1131, 3.11e18), (9813, 2.48e19), (4991, 9.11e18) ]
I have 4G of physical memory and would like to figure out approximately how many rows (key:values) I can store in memory before I spill into swap. This is on linux/ubuntu 8.04 and OS X 10.5.6 .
Also, what's the best way to figure out the actual in-memory footprint of my program? How do I best figure out when it's exhausting physical memory and spilling?
532
You can use it by putting the @profile decorator around any function or method and running python -m memory_profiler myscript. You'll see line-by-line memory usage once your script exits.
In python, the usage of sys. getsizeof() can be done to find the storage size of a particular object that occupies some space in the memory. This function returns the size of the object in bytes.
When you create a list object, the list object by itself takes 64 bytes of memory, and each item adds 8 bytes of memory to the size of the list because of references to other objects.
You can detect memory leaks in Python by monitoring your Python app's performance via an Application Performance Monitoring tool such as Scout APM. Once you detect a memory leak, there are multiple ways to solve it.
Guppy has a nice memory profiler (Heapy):
>>> from guppy import hpy
>>> hp = hpy()
>>> hp.setrelheap() # ignore all existing objects
>>> d = {}
>>> d['key'] = [ (1131, 3.11e18), (9813, 2.48e19), (4991, 9.11e18) ]
>>> hp.heap()
Partition of a set of 24 objects. Total size = 1464 bytes.
Index Count % Size % Cumulative % Kind (class / dict of class)
0 2 8 676 46 676 46 types.FrameType
1 6 25 220 15 896 61 str
2 6 25 184 13 1080 74 tuple
...
Heapy is a little underdocumented, so you might have to dig through the web page or source code a little, but it's very powerful. There are also some articles which might be relevant.
50
You can do this with a memory profiler, of which there are a couple I'm aware of:
PySizer - poissibly obsolete, as the homepage now recommends:
Heapy.
This is possibly a duplicate of this question.
22
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
