Can someone explain to me why the first code block doesn't change the list, but the second does.
a = [1,2,3]
for el in a:
el += 5
This leaves a
as [1,2,3]
. That said if I run
a = [1,2,3]
for i in range(len(a)):
a[i] += 5
then a = [6,7,8]
. My guess is that in the first, when looping over the elements el
is a temporary variable, and not actually the thing that references that element in the list. Not sure why incrementing it doesn't effect the list though.
On incrementing the value of a, now a is reasign to a+1 (id: 1919375104) and other b and c refers to same object (1919375088). Also python does come up with ++/-- operator.
If you are familiar with other programming languages, such as C++ or Javascript, you may find yourself looking for an increment operator. This operator typically is written as a++ , where the value of a is increased by 1. In Python, however, this operator doesn't exist.
In C, C++, Java etc ++ and -- operators increment and decrement value of a variable by 1. In Python these operators won't work.
Python integers are not mutable, but lists are.
In the first case el
references immutable integers, so +=
creates a new integer that only el
refers to.
In the second case the list a
is mutated directly, modifying its elements directly. a[0]
still references an immutable integer, so +=
creates a new integer, but its reference is assigned directly to an element of the mutable list.
Here are examples showing the reference ids of list elements. In the first case, new integers are created, but the original list references are unchanged.
a = [1,2,3]
print [id(x) for x in a]
print a
for el in a:
el += 5 # creates new integer, but only temp name references it
print [id(x) for x in a] # references are not changed.
print a
Output
[36615248, 36615236, 36615224]
[1, 2, 3]
[36615248, 36615236, 36615224]
[1, 2, 3]
In the second case, the list references are updated:
a = [1,2,3]
print [id(x) for x in a]
print a
for i in range(len(a)):
a[i] += 5 # creates new integer, but updates mutable list
print [id(x) for x in a] # references are changed.
print a
Output
[36615248, 36615236, 36615224]
[1, 2, 3]
[36615188, 36615176, 36615164]
[6, 7, 8]
=
(when the left-hand side is just an identifier) is purely a syntactic construct, which binds the name on the left to the object on the right.
All other assignments are short-hand for various method calls.
a[i] = 3
is short for a.__setitem__(i, 3)
a += 3
is short for a = a.__iadd__(3)
a[i] += 3
is short for a.__setitem__(i, a[i]+3)
The end result of each method call depends on how type(a)
implements the method being called. The method might mutate its caller, or it might return a new object.
I first wrote this as a comment but I'd like to expand it a bit, especially to add the tuple example.
Mark Tolonen's answer is correct (and upvoted) in that ordinary integers are immutable (cannot be changed) and lists are mutable (can have elements replaced), but does not mention another couple of key concepts, which show up in slightly scary examples:
Objects get bound to variables.
Ordinary variable assignment like x = 3
simply binds the object on the right—which may be constructed on the spot if needed—to the name on the left.
"In-place" operators like +=
attempt to invoke modifier functions, which allows mutable objects to capture them. For instance, if x
is bound to a class instance, writing x += 3
will actually execute x.__iadd__(3)
, if x
has an __iadd__
.1 If not, it runs x = x + 3
instead, which invokes the __add__
operator: x = x.__add__(3)
. See the operator documentation for all the gory details. In this case, the objects involved—ordinary integers—don't have modifier functions:
>>> (3).__add__
<method-wrapper '__add__' of int object at 0x801c07f08>
>>> (3).__iadd__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute '__iadd__'
so this particular twist is irrelevant for int
, but it's worth remembering.
Indexed assignment, x[i] = expression
, invokes the __setitem__
method. This is how mutable (modifiable) objects mutate themselves. A list object implements __setitem__
:
>>> [].__setitem__
<method-wrapper '__setitem__' of list object at 0x8007767e8>
For completeness, I will note that it also implements __getitem__
to retrieve x[i]
.
Hence, when you write a[i] += 5
, you wind up calling:
a.__setitem__(i, a.__getitem__(i) + 5)
which is how Python manages to add 5 to the i'th element of the list bound to a
.
Here's the slightly scary example. A tuple object is not modifiable, but a list object is. If we embed a list into a tuple:
>>> l = [0]
>>> t = (l,)
we can then use t[0]
to invoke t.__getitem__
and t.__setitem__
. Meanwhile t[0]
is bound to the same list object as l
. This part is obvious enough:
>>> t
([0],)
>>> l.append(1)
>>> t
([0, 1],)
We modified l
in place, so t[0]
, which names the same list as l
, has been modified. But now:
>>> t[0] += [2]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>> t
([0, 1, 2],)
What?! How did t
change when we got an error telling us that t
can't change?
The answer is, t
didn't change, but the list (which we can also access via l
) did change. Lists implement __iadd__
, so the assignment:
t[0] += [2]
"means":
t.__setitem__(0, t.__getitem__(0).__iadd__([2]))
The __getitem__
accessed the list and the __iadd__
augmented the list:
>>> l
[0, 1, 2]
and then the t.__setitem__(0, ...)
raised the TypeError
, but the list was already augmented by then.
Note, by the way, the fact that tweaking the list object bound to l
affects the tuple object bound to t
, because t[0]
is that list object. This—the fact that variables are bound to objects, and elements within data structures like tuples, lists, and dictionaries, can refer to other objects—is critical in reading and writing Python code. Understanding both the binding rules, and when objects get created, is the key to knowing why this is usually a bad idea:
def f(a=[]):
Specifically, the list object here is created at def
time, i.e., just once. Any time someone adds to the list inside f
, with a.append
for instance, it keeps adding to that original list!
See also python: how binding works for (way too) much more about additional binding rules. :-)
1As Duncan points out in a comment on chepner's answer, after invoking __iadd__
, the returned result is re-bound to the object. (All functions return a result; returning without an expression, or "falling off the end" of a function, is defined as returning None
.)
Mutable objects should generally return themselves, and immutable objects don't need to implement __iadd__
in the first place since implementing a mutation on a supposedly-immutable object just seems odd. Nonetheless, we can abuse, and thereby expose, this behavior by writing a class that pretends to be immutable, but is not. Here's an example. This is not meant to be useful, just to illustrate one of the darker corners of Python.
"""
demonstration of iadd behavior
"""
from __future__ import print_function
class Oddity(object):
"""
A very odd class: like a singleton, but there can be
more than one of them. Each one is a list that just
accumulates more items. The __iadd___ (+=) operator
augments the item, then advances to the next instance.
Creating them is tricky as we want to create new ones
up until we "freeze" the class, then start re-using
the instances. We use a __new__ operator a la immutable
objects, plus a boolean in the class itself, even though
each instance is mutable.
"""
def __new__(cls):
if not hasattr(cls, 'frozen'):
cls.frozen = False
if cls.frozen:
whichone = cls.rotator
cls.rotator = (whichone + 1) % len(cls.instances)
return cls.instances[whichone]
self = object.__new__(cls)
if not hasattr(cls, 'instances'):
cls.instances = []
self.whichone = len(cls.instances)
self.values = []
cls.instances.append(self)
print('created', self)
return self
def __str__(self):
return '#{}, containing {}'.format(self.whichone, self.values)
def __iadd__(self, value):
print('iadd to', self)
self.values.append(value)
all_oddities = self.__class__.instances
nextone = (self.whichone + 1) % len(all_oddities)
return all_oddities[nextone]
@classmethod
def freeze(cls):
if not hasattr(cls, 'frozen'):
raise TypeError('need at least one instance to freeze')
cls.frozen = True
cls.rotator = 0
# Now make two instances, and freeze the rest so that
# we can cycle back and forth.
o0 = Oddity()
o1 = Oddity()
Oddity.freeze()
print('o0 is', o0)
o0 += 'first add to o0'
o0 += 'second add to o0'
o0 += 'third add to o0'
print('now o0 is', o0, 'and o1 is', o1)
print('the first object is', Oddity.instances[0])
Once we create the two objects and freeze the class, we invoke __iadd__
three times on o0
, so in the end, o0
and o1
are actually both bound to the second object. The first object—findable only through the class's cls.instances
field—has two items in its list-of-items.
As an exercise, try to predict what this will print out, before running it.
(Extra-advanced exercise: turn Oddity
into a metaclass that can be applied to classes to turn them into freeze-able multi-singletons. [Is there a term for "thing that is like a singleton but allows N of them"?] See also Why Singletons Are Controversial.)
[Edit: this had been bugging me: it turns out there is a name for fixed-set-of-singletons. When the set has just two elements, it's a "doubleton". In Python, True
and False
are the doubletons comprising the bool
type. The generalization to n objects is a multiton, often instantiated / used as a fixed—at least, after "freeze" time—hash table. Note that Python's doubleton boolean instances are immutable, as is Python's singleton None
.]
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With