metadata
base_model:
- zetasepic/Qwen2.5-72B-Instruct-abliterated
EXL2 Quantizations of Qwen2.5-72B-Instruct-abliterated
Using exllamav2 release 0.2.6 for quantization.
Original model: https://huggingface.co/zetasepic/Qwen2.5-72B-Instruct-abliterated
Bits 6.5, lm_head 8.0
"quantization_config": {
"quant_method": "exl2",
"version": "0.2.6",
"bits": 6.5,
"head_bits": 8,
"calibration": {
"rows": 115,
"length": 2048,
"dataset": "(default)"
}