In PyTorch, what is the difference between the following two methods in sending a tensor (or model) to GPU:
Setup:
X = np.array([[1, 3, 2, 3], [2, 3, 5, 6], [1, 2, 3, 4]]) # X = model()
X = torch.DoubleTensor(X)
Method 1 | Method 2 |
---|---|
X.cuda() |
device = torch.device("cuda:0") X = X.to(device)
|
(I don't really need a detailed explanation of what is happening in the backend, just want to know if they are both essentially doing the same thing)
cuda. Returns a copy of this object in CUDA memory. If this object is already in CUDA memory and on the correct device, then no copy is performed and the original object is returned.
cuda() and to('cuda') are going to do the same thing, but the later is more flexible. As you can see in your example code, you can specify a device that might be 'cpu' if cuda is unavailable.
device('cuda') refers to GPU index 0. Similarly, tensor. cuda() and model. cuda() move the tensor/model to “cuda: 0” by default if not specified. n4tman August 17, 2020, 1:57pm #5.
cuda is used to set up and run CUDA operations. It keeps track of the currently selected GPU, and all CUDA tensors you allocate will by default be created on that device. The selected device can be changed with a torch. cuda.
There is no difference between the two.
Early versions of pytorch had .cuda()
and .cpu()
methods to move tensors and models from cpu to gpu and back. However, this made code writing a bit cumbersome:
if cuda_available:
x = x.cuda()
model.cuda()
else:
x = x.cpu()
model.cpu()
Later versions introduced .to()
that basically takes care of everything in an elegant way:
device = torch.device('cuda') if cuda_available else torch.device('cpu')
x = x.to(device)
model = model.to(device)
Their syntax varies slightly, but they are equivalent:
⠀ | .to(name) | .to(device) | .cuda() |
---|---|---|---|
CPU | to('cpu') |
to(torch.device('cpu')) |
cpu() |
Current GPU | to('cuda') |
to(torch.device('cuda')) |
cuda() |
Specific GPU | to('cuda:1') |
to(torch.device('cuda:1')) |
cuda(device=1) |
Note: the current cuda device is 0
by default, but this can be set with torch.cuda.set_device()
.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With