Update README.md
Browse files
README.md
CHANGED
@@ -21,7 +21,7 @@ tags:
|
|
21 |
# Quant Infos
|
22 |
|
23 |
- quants done with an importance matrix for improved quantization loss
|
24 |
-
- quantized & generated imatrix from the f32 as f16 is inaccurate when
|
25 |
- K & IQ quants in basically all variants from Q6_K down to IQ1_S
|
26 |
|
27 |
Quantized with [llama.cpp](https://github.com/ggerganov/llama.cpp) commit [4e96a812b3ce7322a29a3008db2ed73d9087b176](https://github.com/ggerganov/llama.cpp/commit/4e96a812b3ce7322a29a3008db2ed73d9087b176) (master from 2024-04-23) with fixes from [this](https://github.com/ggerganov/llama.cpp/pull/6851) branch cherry-picked
|
|
|
21 |
# Quant Infos
|
22 |
|
23 |
- quants done with an importance matrix for improved quantization loss
|
24 |
+
- quantized & generated imatrix from the f32 as f16 is inaccurate when converting from bf16
|
25 |
- K & IQ quants in basically all variants from Q6_K down to IQ1_S
|
26 |
|
27 |
Quantized with [llama.cpp](https://github.com/ggerganov/llama.cpp) commit [4e96a812b3ce7322a29a3008db2ed73d9087b176](https://github.com/ggerganov/llama.cpp/commit/4e96a812b3ce7322a29a3008db2ed73d9087b176) (master from 2024-04-23) with fixes from [this](https://github.com/ggerganov/llama.cpp/pull/6851) branch cherry-picked
|