Processing data by reference or by value in python

Question

Consider the following session. How are the differences explained? I thought that a += b is a syntactical sugar of (and thus equivalent to) a = a + b. Obviously I'm wrong.

>>> import numpy as np
>>> a  = np.arange(24.).reshape(4,6)
>>> print a
[[  0.   1.   2.   3.   4.   5.]
 [  6.   7.   8.   9.  10.  11.]
 [ 12.  13.  14.  15.  16.  17.]
 [ 18.  19.  20.  21.  22.  23.]]
>>> for line in a:
...     line += 100
...
>>> print a #a has been changed
[[ 100.  101.  102.  103.  104.  105.]
 [ 106.  107.  108.  109.  110.  111.]
 [ 112.  113.  114.  115.  116.  117.]
 [ 118.  119.  120.  121.  122.  123.]]
>>>
>>> for line in a:
...     line = line + 999
...
>>> print a #a hasn't been changed
[[ 100.  101.  102.  103.  104.  105.]
 [ 106.  107.  108.  109.  110.  111.]
 [ 112.  113.  114.  115.  116.  117.]
 [ 118.  119.  120.  121.  122.  123.]]

Thank you

Answer 1

Using the + operator results in a call to the special method __add__ which should create a new object and should not modify the original.

On the other hand, using the += operator results in a call to __iadd__ which should modify the object if possible rather than creating a new object.

__add__

These methods are called to implement the binary arithmetic operations (+, -, *, //, %, divmod(), pow(), **, <<, >>, &, ^, |). For instance, to evaluate the expression x + y, where x is an instance of a class that has an __add__() method, x.__add__(y) is called.

__iadd__

These methods are called to implement the augmented arithmetic assignments (+=, -=, *=, /=, //=, %=, **=, <<=, >>=, &=, ^=, |=). These methods should attempt to do the operation in-place (modifying self) and return the result (which could be, but does not have to be, self).

Of course it is possible to implement __add__ and __iadd__ to have some other behaviour if you wanted to, but what you observe is the standard and recommended way. And, yes, it is a little surprising the first time you see it.

Answer 2

You're not wrong, sometimes a += b really is syntactic sugar for a = a + b, but then sometimes it's not, which is one of the more confusing features of Python - see this similar question for more discussion.

The + operator calls the special method __add__, and the += operator tries to call the in-place __iadd__ special method, but I think it's worth expanding on the case where __iadd__ isn't defined.

If the in-place operator isn't defined, for example for immutable types such as strings and integers, then __add__ is called instead. So for these types a += b really is syntactic sugar for a = a + b. This toy class illustrates the point:

>>> class A(object):
...     def __add__(self, other):
...         print "In __add__ (not __iadd__)"
...         return A()
...
>>> a = A()
>>> a = a + 1
In __add__ (not __iadd__)
>>> a += 1
In __add__ (not __iadd__)

This is the behaviour you should expect from any type that is immutable. While this can be confusing, the alternative would be to disallow += on immutable types which would be unfortunate as that would mean you couldn't use it on strings or integers!

For another example, this leads to a difference between lists and tuples, both of which support +=, but only lists can be modified:

>>> a = (1, 2)
>>> b = a
>>> b += (3, 4)   # b = b + (3, 4)   (creates new tuple, doesn't modify)
>>> a
(1, 2)

>>> a = [1, 2]
>>> b = a
>>> b += [3, 4]   # calls __iadd___ so modifies b (and so a also)
>>> a
[1, 2, 3, 4]

Of course the same goes for all the other in-place operators, -=, *=, //=, %=, etc.