File size: 501 Bytes
8fe92a4 6a16381 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
<p><strong>Information</strong></p>
Alpaca 30B 4-bit. It's working with the newest GPTQ. Quantized using --true-sequential and --act-order optimizations.
This was made using Chansung's 30B Alpaca Lora: https://huggingface.co/chansung/alpaca-lora-30b
<p><strong>Benchmarks</strong></p>
Wikitext2: 4.58
Ptb: 7.71
C4: 6.32
Note: This version does not use grouping, therefore evaluations are minimally higher. However, this version allows fitting the whole model at full context using only 24GB VRAM |