pytorch "log_softmax_lastdim_kernel_impl" not implemented for 'torch.LongTensor'

Question

I am trying to use my own dataset to classify text according to https://github.com/bentrevett/pytorch-sentiment-analysis/blob/master/5%20-%20Multi-class%20Sentiment%20Analysis.ipynb. My dataset is a csv of sentences and a class associated with it. there are 6 different classes:

sent                      class
'the fox is brown'        animal
'the house is big'        object
'one water is drinkable'  water
...

When running:

N_EPOCHS = 5

best_valid_loss = float('inf')

for epoch in range(N_EPOCHS):

    start_time = time.time()
    print(start_time)
    train_loss, train_acc = train(model, train_iterator, optimizer, criterion)
    print(train_loss.type())
    print(train_acc.type())
    valid_loss, valid_acc = evaluate(model, valid_iterator, criterion)

    end_time = time.time()

    epoch_mins, epoch_secs = epoch_time(start_time, end_time)

    if valid_loss < best_valid_loss:
        best_valid_loss = valid_loss
        torch.save(model.state_dict(), 'tut5-model.pt')

    print(f'Epoch: {epoch+1:02} | Epoch Time: {epoch_mins}m {epoch_secs}s')
    print(f'\tTrain Loss: {train_loss:.3f} | Train Acc: {train_acc*100:.2f}%')
    print(f'\t Val. Loss: {valid_loss:.3f} |  Val. Acc: {valid_acc*100:.2f}%')

, I receive the following error

RuntimeError: "log_softmax_lastdim_kernel_impl" not implemented for 'torch.LongTensor'

pointing to:

<ipython-input-38-9c6cff70d2aa> in train(model, iterator, optimizer, criterion)
     14         print('pred'+ predictions.type())
     15         #batch.label = batch.label.type(torch.LongTensor)
---> 16         loss = criterion(predictions.long(), batch.label)**

The solution posted here https://github.com/pytorch/pytorch/issues/14224 suggests I need to use long/int.

I had to add .long() at line ** in order to fix this earlier error:

RuntimeError: Expected object of scalar type Long but got scalar type Float for argument #2 'target'

The specific lines of code are:

  def train(model, iterator, optimizer, criterion):
    epoch_loss = 0
    epoch_acc = 0

    model.train()

    for batch in iterator:

        optimizer.zero_grad()

        predictions = model(batch.text)
        print('pred'+ predictions.type())
        #batch.label = batch.label.type(torch.LongTensor)
        loss = criterion(predictions.long(), batch.label)**

        acc = categorical_accuracy(predictions, batch.label)

        loss.backward()

        optimizer.step()

        epoch_loss += loss.item()
        epoch_acc += acc.item()

    return epoch_loss / len(iterator), epoch_acc / len(iterator)

Note, the ** was originally loss = criterion(predictions, batch.label)

Any other suggestions to fix this issue?

Answer 1

criterion is defined as torch.nn.CrossEntropyLoss() in your notebook. As mentioned in documentation of CrossEntropyLoss, it expects probability values returned by model for each of the 'K' classes and corresponding value for ground-truth label as input. Now, probability values are float tensors, while ground-truth label should be a long tensor representing a class (class can not be a float, e.g. 2.3 can not represent a class). hence:

loss = criterion(predictions, batch.label.long())

should work.