ImportError: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install accelerate`

Question

Using HuggingFace Transformer I am trying to create a pipeline, by running below code (code is running on a SageMaker Jupyter Lab):

pipeline = transformers.pipeline(
    "text-generation",
    model="meta-llama/Llama-2-7b-chat-hf",
    torch_dtype=torch.float16,
    device_map="auto",
)

However, it generates following error:

ImportError: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install accelerate`

Reference:

Packages version:print(accelerate.__version__), print(transformers.__version__)
0.21.0
4.31.0

Answer 1

I found the root cause of issue. After installed the accelarate package, we must restart the notebook kernel. That solved the problem.