EXL2 quants
#2
by
kim512
- opened
Hi,
I have created exl2 quats of this model:
3.00 bits per weight
3.50 bits per weight
4.00 bits per weight
4.50 bits per weight
5.00 bits per weight
6.00 bits per weight
8.00 bits per weight
Hi,
I have created exl2 quats of this model:
Hey thanks Kim512
As a note, I forgot to include this info in the ReadMe
I am using the defaults from exllamav2/convert.py
head bits = 6
length = 2048
dataset rows = 100
measurement rows = 16
measurement length = 2048
using wikitext-2-v1.parquet
Please let me know if any of these should be changed.