Pytorch error when modifying unpacked tensor

Question

I have an error when modifying an unpacked tensor.

import torch
a1, a2 = torch.tensor([1,2], dtype = torch.float64)
b = torch.rand(2, requires_grad = True)

a1 += b.sum()

This code produces the following error:

RuntimeError: A view was created in no_grad mode and is being modified inplace with grad mode enabled. This view is the output of a function that returns multiple views. Such functions do not allow the output views to be modified inplace. You should replace the inplace operation by an out-of-place one.

However, I didn't receive that error when I created a1 and a2 separately as follows:

a1 = torch.tensor([1], dtype = torch.float64)
a1 += b.sum()

I have no idea why unpacking a tensor would lead to that error. Any help is greatly appreciated

Answer 1

The error message is clear and explains that:

• You created two views a1 and a2 in no_grad mode and view b in grad mode enabled.

• Such functions (unpacking) do not allow the output views (a1,a2) to be modified inplace (inplace operation +=)

• Solution: you should replace the inplace operation by an out-of-place one. You can clone a1 to a before this operation as follows:

  a = a1.clone()
  a += b.sum()
  

That is because tensor.clone() creates a copy of the original tensor with requires_grad field. Thus, you will find now a and b have requires_grad. If you print the

print(a.requires_grad)        #True
print(b.requires_grad)        #True
print(a1.requires_grad)       #False

You can read more about inplace operation 1 2