Changing the np array does not change the Torch Tensor automatically?

Question

I was going through the basic tutorials of PyTorch and came across conversion between NumPy arrays and Torch tensors. The documentation says:

The Torch Tensor and NumPy array will share their underlying memory locations, and changing one will change the other.

But, this does not seem to be the case in the below code:

import numpy as np

a = np.ones((3,3))
b = torch.from_numpy(a)

np.add(a,1,out=a)
print(a)
print(b)

In the above case I see the changes automatically reflected in the output:

[[2. 2. 2.]
 [2. 2. 2.]
 [2. 2. 2.]]
tensor([[2., 2., 2.],
        [2., 2., 2.],
        [2., 2., 2.]], dtype=torch.float64)

But same doesn't happen when I write something like this:

a = np.ones((3,3))
b = torch.from_numpy(a)

a = a + 1
print(a)
print(b)

I get the following output:

[[2. 2. 2.]
 [2. 2. 2.]
 [2. 2. 2.]]
tensor([[1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.]], dtype=torch.float64)

What am I missing here?

Answer 1

Any time you write a = sign in Python you are creating a new object.

So the right-hand side of your expression in the second case uses the original a and then evaluates to a new object i.e. a + 1, which replaces this original a. b still points to the memory location of the original a, but now a points to a new object in memory.

In other words, in a = a + 1, the expression a + 1 creates a new object, and then Python assigns that new object to the name a.

Whereas, with a += 1, Python calls a's in-place addition method (__iadd__) with the argument 1.

The numpy code: np.add(a,1,out=a) , in the first case takes care of adding that value to the existing array in-place.

(Thanks to @Engineero and @Warren Weckesser for pointing out these explanations in the comments)