dropout(): argument 'input' (position 1) must be Tensor, not str when using Bert with Huggingface

Question

My code was working fine and when I tried to run it today without changing anything I got the following error:

dropout(): argument 'input' (position 1) must be Tensor, not str

Would appreciate if help could be provided.
Could be an issue with the data loader?

Answer 1

if you use HuggingFace, this information could be useful. I have same error and fix it with adding parameter return_dict=False in model class before dropout: outputs = model(**inputs, return_dict=False)

Answer 2

I was also working on same repo. There is a class probably named Bert_Arch that inherits the nn.Module and this class has a overriden method named forward. Inside forward method just add the parameter 'return_dict=False' to the self.bert() method call. Replace

_, cls_hs = self.bert(sent_id, attention_mask=mask)

with

_, cls_hs = self.bert(sent_id, attention_mask=mask, return_dict=False)

Answer 3

If you are using the Hugging Face transformers library, this error pops up when running code written in v3 on the transformers v4 library. To resolve it, simply add return_dict=False when loading the model as follows:

model = BertModel.from_pretrained("bert-base-cased")
outputs = model(**inputs, return_dict=False)

or

model = BertModel.from_pretrained("bert-base-cased", return_dict=False)
outputs = model(**inputs)

I hope this helps. It worked for me.

Reference: https://huggingface.co/transformers/migration.html