Pytorch embedding RuntimeError: Expected object of type torch.LongTensor but found type torch.cuda.LongTensor for argument #3 'index'

Question

I'm getting this error saying

RuntimeError: Expected object of type torch.LongTensor but found type torch.cuda.LongTensor for argument #3 'index'

But what does it mean by argument #3 "index"? I can't find "index" argument in torch.embedding (here source : https://pytorch.org/docs/stable/_modules/torch/nn/modules/sparse.html#Embedding) It seems like I'm passing the embedding the wrong parameters.

I even changed the data type of my input like below but the error persists.

batch['doc_tok'] = batch['doc_tok'].long()
batch['query_tok'] = batch['query_tok'].long()

Any comment (even though it's short!) or just listing keywords to look at will be highly appreciated!

---

Here is a full traceback.

Traceback (most recent call last):
  File "train_v2.py", line 110, in <module>
    main()
  File "train_v2.py", line 81, in main
    model.update(batch)
  File "/home/aerin/Desktop/squad_vteam/src/model.py", line 129, in update
    loss_adv = self.adversarial_loss(batch, loss, self.network.lexicon_encoder.embedding.weight, y)
  File "/home/aerin/Desktop/squad_vteam/src/model.py", line 104, in adversarial_loss
    start, end, _ = self.network(batch)
  File "/home/aerin/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/aerin/Desktop/squad_vteam/src/dreader.py", line 78, in forward
    doc_mask, query_mask = self.lexicon_encoder(batch)
  File "/home/aerin/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/aerin/Desktop/squad_vteam/src/encoder.py", line 116, in forward
    doc_emb, query_emb = emb(doc_tok), emb(query_tok)
  File "/home/aerin/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/aerin/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 108, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)
  File "/home/aerin/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/functional.py", line 1076, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected object of type torch.LongTensor but found type torch.cuda.LongTensor for argument #3 'index'

---

Update: I even sent the whole model.network to cpu but still getting the same error.

batch['doc_tok']=batch['doc_tok'].long().cpu()
batch['query_tok']=batch['query_tok'].long().cpu()
self.network.cpu()

print(batch['doc_tok'].dtype,  batch['query_tok'].dtype) # They are both torch.int64 torch.int64

start, end, _ = self.network(batch)

At this point, I'm suspecting this might be a bug...

model.py code: https://github.com/byorxyz/san_mrc/blob/master/src/model.py

Network defined: https://github.com/byorxyz/san_mrc/blob/master/src/dreader.py

Answer 1

It seems to me like your input/target tensors (batch['doc_tok'], etc.) and your network and its variables (I believe index is an internal tensor for Embedding layers) are on different devices (CPU for your data, GPU for your model?).

If you want everything to run on GPU, you need both to:

• Load your data there, e.g. batch['doc_tok'].cuda()
• Load your model there, e.g. model.network.cuda()

Same goes if you want to run on CPU, replacing with .cpu().

Answer 2

Try this:

batch['doc_tok']=batch['doc_tok'].long().cpu()
batch['query_tok']=batch['query_tok'].long().cpu()