Best way of using hugging face's Mask Filling for more than 1 masked token at a time

Question

I am able to use hugging face's mask filling pipeline to predict 1 masked token in a sentence using the below:

!pip install -q transformers
from __future__ import print_function
import ipywidgets as widgets
from transformers import pipeline

nlp_fill = pipeline('fill-mask')
nlp_fill("I am going to guess <mask> in this sentence")

But does anyone have an opinion on what is the best way to do this if I want to predict 2 masked tokens? e.g. if the sentence is instead "I am going to <mask> <mask> in this sentence"?

If i try and put this exact sentence into nlp_fill I get the error "ValueError: only one element tensors can be converted to Python scalars" so it doesn't work automatically.

Any help would be much appreciated!

Answer 1

Sadly, the expectation that there be only one word behind the mask is hardcoded into the FillMaskPipeline class.

For a more constrained problem, of mask-filling given a fixed number of choices to fill the mask, it is possible to extend FitBERT to do this - I have a notebook I can send you if you message me, but it's embarassingly bad code. I am one of the authors of FitBERT, but I haven't had a chance to add this.

EDIT

Here is the notebook , don’t judge me