How many bytes per element are there in a Python list (tuple)?

Question

For example, how much memory is required to store a list of one million (32-bit) integers?

alist = range(1000000) # or list(range(1000000)) in Python 3.0

Answer 1

"It depends." Python allocates space for lists in such a way as to achieve amortized constant time for appending elements to the list.

In practice, what this means with the current implementation is... the list always has space allocated for a power-of-two number of elements. So range(1000000) will actually allocate a list big enough to hold 2^20 elements (~ 1.045 million).

This is only the space required to store the list structure itself (which is an array of pointers to the Python objects for each element). A 32-bit system will require 4 bytes per element, a 64-bit system will use 8 bytes per element.

Furthermore, you need space to store the actual elements. This varies widely. For small integers (-5 to 256 currently), no additional space is needed, but for larger numbers Python allocates a new object for each integer, which takes 10-100 bytes and tends to fragment memory.

Bottom line: it's complicated and Python lists are not a good way to store large homogeneous data structures. For that, use the array module or, if you need to do vectorized math, use NumPy.

PS- Tuples, unlike lists, are not designed to have elements progressively appended to them. I don't know how the allocator works, but don't even think about using it for large data structures :-)

Answer 2

Useful links:

How to get memory size/usage of python object

Memory sizes of python objects?

if you put data into dictionary, how do we calculate the data size?

However they don't give a definitive answer. The way to go:

1. Measure memory consumed by Python interpreter with/without the list (use OS tools).

2. Use a third-party extension module which defines some sort of sizeof(PyObject).

Update:

Recipe 546530: Size of Python objects (revised)

import asizeof

N = 1000000
print asizeof.asizeof(range(N)) / N
# -> 20 (python 2.5, WinXP, 32-bit Linux)
# -> 33 (64-bit Linux)