ImportError: Using `bitsandbytes` 8-bit quantization requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes
Error:
- as following
pip install accelerate bitsandbytesdoesn't work
ImportError: Using `bitsandbytes` 8-bit quantization requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes
Env info
Python 3.12.1
accelerate 0.27.2
bitsandbytes 0.42.0
torch 2.2.1
tqdm 4.65.0
transformers 4.38.2
trl 0.7.11
Someone says we need to downgrade transformers library to version 4.30, on order to fix this error
However, GemmerTokenizer need to upgrade transformers to version 4.38 ...
Can you retry on a fresh new env? Also can you confirm you have access to a GPU device?
I also encountered the same issue
I am having the same issue. I am using Google Colab with an A100 GPU and have tried several versions of bitsandbytes, all >=4.1.3 since only this version supports the quantization method. I had to install version 0.4.2 on Google Colab, and that one worked.
@WQW @HONGMAOLIN I think if you are experimenting this locally. Accelerate requires GPU (Mandatory). without GPU accelerate won't work. You may try that on google colab free T4.
Happening for me on Windows(no GPU) with python 3.11 and:
accelerate 0.28.0
bitsandbytes 0.43.0
transformers 4.38.2
same issue on a mac m1
@thepandapi as far as I know Mac doesn't have a dedicated GPU. Please make sure if you are using the library locally you have a dedicated GPU.
I am having the same issue
packages installed with CUDA 11.8 -
torch - 2.3.0+cu118
llama-index - 0.10.37
llama-index-llms-huggingface - 0.2.0
transformers - 4.39.0
accelerate - 0.27.0
bitsandbytes - 0.43.1
@ybelkada Thank you mate! It worked after restarting the jupyter notebook
Hi everyone, there are news about this issue? I have the same issue In a windows machine with a dedicated GPU, a rtx 3060 of 20gbs.