RuntimeError: _thnn_mse_loss_forward is not implemented for type torch.cuda.LongTensor

Question

I'm using PyTorch,but I get a error! My error code as following:

for train_data in trainloader:
    example_count += 1
    if example_count == 100:
        break
    optimer.zero_grad()
    image, label = train_data
    image = image.cuda()
    label = label.cuda()
    out = model(image)
    _, out = torch.max(out, 1)
    # print(out.cpu().data.numpy())
    # print(label.cpu().data.numpy())
    # out = torch.zeros(4, 10).scatter_(1, out.cpu(), 1).cuda()
    # label= torch.zeros(4, 10).scatter_(1, label.cpu(), 1).cuda()
    l = loss(out, label)
    l.bakeward()
    optimer.setp()
    j += 1
    count += label.size(0)
    acc += (out == label).sum().item()
    if j % 1000 == 0:
        print(j + ' step:curent accurity is %f' % (acc / count))

the traceback:

    Traceback (most recent call last):
  File "VGG实现.py", line 178, in <module>
    utils.train(testloader,model)
  File "VGG实现.py", line 153, in train
    l=loss(out,label)
  File "/home/tang/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/tang/anaconda3/lib/python3.7/site-packages/torch/nn/modules/loss.py", line 435, in forward
    return F.mse_loss(input, target, reduction=self.reduction)
  File "/home/tang/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py", line 2156, in mse_loss
    ret = torch._C._nn.mse_loss(expanded_input, expanded_target, _Reduction.get_enum(reduction))
RuntimeError: _thnn_mse_loss_forward is not implemented for type torch.cuda.LongTensor

I get a answer whose here Pytorch RuntimeError: "host_softmax" not implemented for 'torch.cuda.LongTensor'

But I don't know how to solve this question.

Answer 1

Look at the documentation of torch.max():

torch.max(input, dim, keepdim=False, out=None) -> (Tensor, LongTensor)

Returns the maximum value of each row of the input tensor in the given dimension dim. The second return value is the index location of each maximum value found (argmax).

Your line of code

_, out = torch.max(out, 1)

Takes the float prediction of the model, out, and uses torch.max() to return the argmax = type long int index of the maximal prediction.
The error message you get is that your loss function (you are using cross entropy with softmax I guess) does not support a long type first argument.
Moreover, you cannot take derivative through argmax - so I don't think converting out to float using .to(torch.float) is going to do you any good either.
The softmax function inside the loss function you are using is taking care of the argmax for you.