Problem loading pre-trained FinBERT model

Question

I am trying to load a pre-trained FinBERT model in python but am getting an error simply trying to load it. I am following the GitHub docs: https://github.com/ProsusAI/finBERT

Following the tutorial, I have created a directory called bert, downloaded the model pytorch_model.bin and config.json file and dropped them in the folder.

I try and call the model using the following code:

import os
os.chdir(r'bert')

from transformers import BertTokenizer, BertForSequenceClassification
import torch

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')

model = BertForSequenceClassification.from_pretrained(r'bert\pytorch_model.bin',
                                                      config = r'bert\config.json')

The error trying to load is as follows:

RuntimeError: Error(s) in loading state_dict for BertForSequenceClassification:
    size mismatch for classifier.weight: copying a param with shape torch.Size([3, 768]) from checkpoint, the shape in current model is torch.Size([2, 768]).
    size mismatch for classifier.bias: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([2]).

My knowledge of this is very basic and if anyone understands what is going on here it would be greatly appreciated.

Thanks

Answer 1

All you need to do is specify the number of labels:

model = BertForSequenceClassification.from_pretrained(r'bert\pytorch_model.bin',config = r'bert\config.json', num_labels = 3)