Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Memory usage of Python base types (particulary int and float)

This is an example from Python 3.8.0 interpreter (however, it is similar in 3.7.5)

>>> import sys
>>> sys.getsizeof(int)
416
>>> sys.getsizeof(float)
416
>>> sys.getsizeof(list)
416
>>> sys.getsizeof(tuple)
416
>>> sys.getsizeof(dict)
416
>>> sys.getsizeof(bool)
416

getsizeof() returns how much bytes Python object consumes together with the garbage collector overhead (see here). What is the reason that basic python classes consume the same amount of memory?

If we take a look at instances of these classes

>>> import sys
>>> sys.getsizeof(int())
24
>>> sys.getsizeof(float())
24

The default argument is 0 and these two instances have the same amount of memory usage for this argument. However, if I try to add an argument

>>> sys.getsizeof(int(1))
28
>>> sys.getsizeof(float(1))
24

and this is where it gets strange. Why does the instance memory usage increase for int but not for float type?

like image 944
vezeli Avatar asked Dec 15 '19 09:12

vezeli


People also ask

How much memory does a Python float use?

The floating-point type (or “float type” for short) is used for numbers that have fractional parts or are too large to store in a long int that takes up a reasonable amount of memory. Typically, eight bytes are used for the Python floating type.

What type of memory does Python use?

Python has a pymalloc allocator optimized for small objects (smaller or equal to 512 bytes) with a short lifetime. It uses memory mappings called “arenas” with a fixed size of 256 KiB.

Does float take more memory than int?

An int and float usually take up "one-word" in memory. Today, with the shift to 64bit systems this may mean that your word is 64 bits, or 8 bytes, allowing the representation of a huge span of numbers. Or, it could still be a 32bit system meaning each word in memory takes up 4 bytes.

How much memory is allocated for int datatype in Python?

An empty list takes 56 bytes, but each additional int adds just 8 bytes, where the size of an int is 28 bytes. A list that contains a long string takes just 64 bytes. The answer is simple.


2 Answers

In short, it all boils down to how Python represents arbitrary long integers. float() types are represented (limited) just as C double.

In CPython implementation, every object (source) begins with a reference count and a pointer to the type object for that object. That's 16 bytes.

Float object stores its data as C double (source), that's 8 bytes. So 16 + 8 = 24 bytes for float objects.

With integers, situation is more complicated. The integer objects are represented as variable sized object (source), which for 16 bytes adds another 8 bytes. Digits are represented as array. Depending on the platform, Python uses either 32-bit unsigned integer arrays with 30-bit digits or 16-bit unsigned integer arrays with 15-bit digits. So for small integers there's only one 32bit integer in the array, so add another 4 bytes = 16 + 8 + 4 = 28 bytes.

If you want to represent larger integer number, the size will grow:

sys.getsizeof(int(2**32))  # prints 32 (24 + 2*4 bytes)
sys.getsizeof(int(2**64))  # prints 36 (24 + 3*4 bytes)

EDIT:

With sys.getsizeof(int) you're getting the size of the class, not of an instance of the class. That's same for float, bool, ...

print(type(int))  # prints <class 'type'>

If you look into the source, there's lot of stuff under the hood. In my version of Python 3.6.9 (Linux/64bit) this prints 400 bytes.

like image 115
Andrej Kesely Avatar answered Oct 21 '22 08:10

Andrej Kesely


Looking at the docs, it's important to observe that:

Only the memory consumption directly attributed to the object is accounted for, not the memory consumption of objects it refers to.

So what can you infer from the fact that the returned value of sys.getsizeof(int(1)) is greater than that of the sys.getsizeof(float(1))?

Simply that it takes more memory to represent an int than it does to represent a float. Is this surprising? Well, possibly not, if we can expect to "do more things" with an int than we can do with a float. We can gauge the "amount of functionality" to the first degree by looking at the number of their attributes:

>>> len(dir(int))
70
>>> len(dir(float))
57
like image 20
gstukelj Avatar answered Oct 21 '22 08:10

gstukelj