Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why does creating a list from a list make it larger?

I'm seeing some inconsistencies when using sys.getsizeof on what should be identical lists. (Python 2.7.5)

>>> lst = [0,1,2,3,4,5,6,7,8,9] >>> sys.getsizeof(lst) 76 >>> lst2 = list(lst) >>> sys.getsizeof(lst2) 104 >>> lst3 = list(lst2) >>> sys.getsizeof(lst3) 104 >>> sys.getsizeof(lst[:]) 76 >>> sys.getsizeof(lst2[:]) 76 

Does anybody have a simple explanation?

like image 464
Mark Ransom Avatar asked Dec 18 '14 19:12

Mark Ransom


People also ask

How do I fix the size of a list in Python?

You could make it a tuple ( tuple([None] * 10) ) to fix its width, but again, you won't be able to change it (not in all cases, only if the items stored are mutable). Another option, closer to your requirement, is not a list, but a collections. deque with a maximum length.

Why is [] faster than list ()?

Why is [] faster than list() ? The biggest reason is that Python treats list() just like a user-defined function, which means you can intercept it by aliasing something else to list and do something different (like use your own subclassed list or perhaps a deque).

Can a list be too big in Python?

According to the source code, the maximum size of a list is PY_SSIZE_T_MAX/sizeof(PyObject*) . On a regular 32bit system, this is (4294967295 / 2) / 4 or 536870912. Therefore the maximum size of a python list on a 32 bit system is 536,870,912 elements.


2 Answers

With a list literal, the VM creates the list with a set length. When passing a sequence to the list() constructor the elements are added one by one (via list.extend()) and as such the list is resized when appropriate. Since the resize operation overallocates in order to amortize the cost, the final list will usually be larger than the source list.

like image 141
Ignacio Vazquez-Abrams Avatar answered Oct 07 '22 20:10

Ignacio Vazquez-Abrams


When you create a list literal, the size reported is the minimum size needed to hold the data. You can see this because the size jumps up if you append a single element. However, when you use list to copy it, it allocates some extra space - it takes a few appends before it reallocates (in your case, I suspect the 8th append will do it - it needs 4 more bytes per element). There is probably a reason why these allocation behaviors are different, but I'm not sure what that might be.

like image 30
Aaron Dufour Avatar answered Oct 07 '22 19:10

Aaron Dufour