Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Measure Object Size Accurately in Python - Sys.GetSizeOf not functioning

I am trying to accurately/definitively find the size differences between two different classes in Python. They are both new style classes, save for one not having slots defined. I have tried numerous tests to determine their size difference, but they always end up being identical in memory usage.

So far I have tried sys.GetSizeOf(obj) and heapy's heap() function, with no positive results. Test code is below:

import sys
from guppy import hpy

class test3(object):
    def __init__(self):
        self.one = 1
        self.two = "two variable"

class test4(object):
    __slots__ = ('one', 'two')
    def __init__(self):
        self.one = 1
        self.two = "two variable"

test3_obj = test3()
print "Sizeof test3_obj", sys.getsizeof(test3_obj)

test4_obj = test4()
print "Sizeof test4_obj", sys.getsizeof(test4_obj)

arr_test3 = []
arr_test4 = []

for i in range(3000):
    arr_test3.append(test3())
    arr_test4.append(test4())

h = hpy()
print h.heap()

Output:

Sizeof test3_obj 32
Sizeof test4_obj 32

Partition of a set of 34717 objects. Total size = 2589028 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0  11896  34   765040  30    765040  30 str
     1   3001   9   420140  16   1185180  46 dict of __main__.test3
     2   5573  16   225240   9   1410420  54 tuple
     3    348   1   167376   6   1577796  61 dict (no owner)
     4   1567   5   106556   4   1684352  65 types.CodeType
     5     68   0   105136   4   1789488  69 dict of module
     6    183   1    97428   4   1886916  73 dict of type
     7   3001   9    96032   4   1982948  77 __main__.test3
     8   3001   9    96032   4   2078980  80 __main__.test4
     9    203   1    90360   3   2169340  84 type
<99 more rows. Type e.g. '_.more' to view.>

This is all with Python 2.6.0. I also attempted to override the class's sizeof methods to try determine the size by summing the individual sizeofs but that didn't yield any different results:

class test4(object):
    __slots__ = ('one', 'two')
    def __init__(self):
        self.one = 1
        self.two = "two variable"
    def __sizeof__(self):
        return super(test4, self).__sizeof__() + self.one.__sizeof__() + self.two.__sizeof__()

Results with the sizeof method overridden:

Sizeof test3_obj 80
Sizeof test4_obj 80
like image 857
Zoran Pavlovic Avatar asked Jul 02 '12 21:07

Zoran Pavlovic


1 Answers

As others have stated, sys.getsizeof only returns the size of the object structure that represents your data. So if, for instance, you have a dynamic array that you keep adding elements to, sys.getsizeof(my_array) will only ever show the size of the base DynamicArray object, not the growing size of memory that its elements take up.

pympler.asizeof.asizeof() gives an approximate complete size of objects and may be more accurate for you.

from pympler import asizeof
asizeof.asizeof(my_object)  # should give you the full object size
like image 79
Engineero Avatar answered Sep 19 '22 06:09

Engineero