Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

New posts in huggingface-transformers

Using Hugging Face Transformers library how can you POS_TAG French text

TypeError: not a string | parameters in AutoTokenizer.from_pretrained()

How to get a probability distribution over tokens in a huggingface model?

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! When predicting with my model

Does Huggingface's "resume_from_checkpoint" work?

How to compute sentence level perplexity from hugging face language models?

Print input / output / grad / loss at every step/epoch when training Transformers HuggingFace model

NameError: name 'PartialState' is not defined error while training hugging face wave2vec model

How does one set the pad token correctly (not to eos) during fine-tuning to avoid model not predicting EOS?

How to use Huggingface Trainer with multiple GPUs?

Transformers PreTrainedTokenizer add_tokens Functionality

Using HuggingFace pipeline on pytorch mps device M1 pro

Why does the ProtBERT model generate identical embeddings for all non-whitespace-separated (single token?) inputs?

How to install pip install torch==2.1.2+cu118 in linux?

what is the difference between len(tokenizer) and tokenizer.vocab_size

The last dimension of the inputs to a Dense layer should be defined. Found None. Full input shape received: <unknown>

How can I make sentence-BERT throw an exception if the text exceeds max_seq_length, and what is the max possible max_seq_length for all-MiniLM-L6-v2?

Huggingface MarianMT translators lose content, depending on the model

How to efficiently mean-pool BERT embeddings while excluding padding?

HuggingFace's linear scheduler with warmup parameters