Initial commit, needs testing. leave a comment if anything is off, more IQ2 to come, they take much longer. -IDS
1826d8c
verified
| for i in {19..28} | |
| do | |
| numactl -C0-110 quantize --leave-output-tensor --imatrix imatrix.dat Mistral-11b-v0.1_F32.gguf $i 100 | |
| done | |