Documentation for PyTorch .to('cpu') or .to('cuda')

Question

I've searched through the PyTorch documenation, but can't find anything for .to() which moves a tensor to CPU or CUDA memory.

I remember seeing somewhere that calling to() on a nn.Module is an in-place operation, but not so on a tensor.

Is there a in-place version for Tensors?

Where do I find the doco for to() for both nn.Module and Tensor (and possibly elsewhere)?

Answer 1

You already found the documentation! great.

.to is not an in-place operation for tensors. However, if no movement is required it returns the same tensor.

In [10]: a = torch.rand(10)

In [11]: b = a.to(torch.device("cuda"))

In [12]: b is a
Out[12]: False

In [18]: c = b.to(torch.device("cuda"))

In [19]: c is b
Out[19]: True

Since b is already on gpu and hence no change is done and c is b results in True.

However, for models, it is an in-place operation which also returns a model.

In [8]: import torch
In [9]: model = torch.nn.Sequential (torch.nn.Linear(10,10))

In [10]: model_new = model.to(torch.device("cuda"))
In [11]: model_new is model
Out[11]: True

It makes sense to keep it in-place for models as parameters of the model need to be moved to another device and not model object. For tensor, it seems new object is created.

Answer 2

I'm still learning to navigate the documentation site (the search isn't great).

Here's what I found:

torch.Tensor.to()

The tensor version either returns:

• self if the tensor is in the target format already, or,
• a copy of the tensor in the target format

There is no in-place version listed.

---

torch.nn.Module.to()

The documentation implies that this is an in-place operation:

Moves and/or casts the parameters and buffers.