Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

PyTorch: What is the difference between tensor.cuda() and tensor.to(torch.device("cuda:0"))?

In PyTorch, what is the difference between the following two methods in sending a tensor (or model) to GPU:

Setup:

X = np.array([[1, 3, 2, 3], [2, 3, 5, 6], [1, 2, 3, 4]]) # X = model()
X = torch.DoubleTensor(X)
Method 1 Method 2
X.cuda() device = torch.device("cuda:0")
X = X.to(device)

(I don't really need a detailed explanation of what is happening in the backend, just want to know if they are both essentially doing the same thing)

like image 417
Leockl Avatar asked Jul 15 '20 04:07

Leockl


People also ask

What does tensor CUDA () do?

cuda. Returns a copy of this object in CUDA memory. If this object is already in CUDA memory and on the correct device, then no copy is performed and the original object is returned.

Is CUDA () the same as .to device?

cuda() and to('cuda') are going to do the same thing, but the later is more flexible. As you can see in your example code, you can specify a device that might be 'cpu' if cuda is unavailable.

What does device CUDA 0 mean?

device('cuda') refers to GPU index 0. Similarly, tensor. cuda() and model. cuda() move the tensor/model to “cuda: 0” by default if not specified. n4tman August 17, 2020, 1:57pm #5.

What does CUDA () do in PyTorch?

cuda is used to set up and run CUDA operations. It keeps track of the currently selected GPU, and all CUDA tensors you allocate will by default be created on that device. The selected device can be changed with a torch. cuda.


2 Answers

There is no difference between the two.
Early versions of pytorch had .cuda() and .cpu() methods to move tensors and models from cpu to gpu and back. However, this made code writing a bit cumbersome:

if cuda_available:
  x = x.cuda()
  model.cuda()
else:
  x = x.cpu()
  model.cpu()

Later versions introduced .to() that basically takes care of everything in an elegant way:

device = torch.device('cuda') if cuda_available else torch.device('cpu')
x = x.to(device)
model = model.to(device)
like image 91
Shai Avatar answered Sep 17 '22 17:09

Shai


Their syntax varies slightly, but they are equivalent:

.to(name) .to(device) .cuda()
CPU to('cpu') to(torch.device('cpu')) cpu()
Current GPU to('cuda') to(torch.device('cuda')) cuda()
Specific GPU to('cuda:1') to(torch.device('cuda:1')) cuda(device=1)

Note: the current cuda device is 0 by default, but this can be set with torch.cuda.set_device().

like image 29
iacob Avatar answered Sep 19 '22 17:09

iacob