TheBloke
/

gpt4-alpaca-lora-30B-GPTQ

Text Generation

text2text-generation

text-generation-inference

4-bit precision

Model card Files Files and versions Community

TheBloke commited on Apr 17, 2023

Commit

a548e56

·

1 Parent(s): 1c82f5d

Update README.md

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -61,10 +61,11 @@ cd text-generation-webui
 python server.py --model gpt4-alpaca-lora-30B-GPTQ-4bit-128g --wbits 4 --groupsize 128 --model_type Llama # add any other command line args you want
 ```
-The above commands assume you have installed all dependencies for GPTQ-for-LLaMa and text-generation-webui. Please see their respective repositories for further information.
 If you are on Windows, or cannot use the Triton branch of GPTQ for any other reason, you can instead try the CUDA branch:
 ```
 git clone https://github.com/qwopqwop200/GPTQ-for-LLaMa -b cuda
 cd GPTQ-for-LLaMa
 python setup_cuda.py install --force

 python server.py --model gpt4-alpaca-lora-30B-GPTQ-4bit-128g --wbits 4 --groupsize 128 --model_type Llama # add any other command line args you want
 ```
+The above commands assume you have installed all dependencies for `GPTQ-for-LLaMa` and `text-generation-webui`. Please see their respective repositories for further information.
 If you are on Windows, or cannot use the Triton branch of GPTQ for any other reason, you can instead try the CUDA branch:
 ```
+pip uninstall -y quant_cuda
 git clone https://github.com/qwopqwop200/GPTQ-for-LLaMa -b cuda
 cd GPTQ-for-LLaMa
 python setup_cuda.py install --force