Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Accelerate and bitsandbytes is needed to install but I did

I'm trying to load quantization like

from transformers import LlamaForCausalLM
from transformers import BitsAndBytesConfig

model = '/model/'
model = LlamaForCausalLM.from_pretrained(model, quantization_config=BitsAndBytesConfig(load_in_8bit=True))

but I get the error

ImportError: Using `load_in_8bit=True` requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes `pip install -i https://test.pypi.org/simple/ bitsandbytes` or pip install bitsandbytes` 

But I've installed both, and I get the same error. I shut down and restarted the jupyter kernel I was using this on.


1 Answers

I downgraded transformers library to version 4.30 using the following command:

pip install transformers==4.30
like image 105
jason Avatar answered Sep 13 '25 04:09

jason