<p><strong>Information</strong></p>
Alpaca 30B 4-bit. It's working with the newest GPTQ. Quantized using --true-sequential and --act-order optimizations.

This was made using Chansung's 30B Alpaca Lora: https://huggingface.co/chansung/alpaca-lora-30b

<p><strong>Benchmarks</strong></p>

Wikitext2: 4.58

Ptb: 7.71

C4: 6.32

Note: This version does not use grouping, therefore evaluations are minimally higher. However, this version allows fitting the whole model at full context using only 24GB VRAM