Update README.md
Browse files
README.md
CHANGED
@@ -22,7 +22,8 @@ huggingface-cli download huihui-ai/Marco-o1-abliterated --local-dir ./huihui-ai/
|
|
22 |
```
|
23 |
python convert_hf_to_gguf.py huihui-ai/Marco-o1-abliterated --outfile huihui-ai/Marco-o1-abliterated/ggml-model-f16.gguf --outtype f16
|
24 |
```
|
25 |
-
3. Use the [llama.cpp](https://github.com/ggerganov/llama.cpp) quantitative program to quantitative model (llama-quantize needs to be compiled.),
|
|
|
26 |
```
|
27 |
llama-quantize huihui-ai/Marco-o1-abliterated/ggml-model-f16.gguf huihui-ai/Marco-o1-abliterated/ggml-model-Q4_K_M.gguf Q4_K_M
|
28 |
```
|
|
|
22 |
```
|
23 |
python convert_hf_to_gguf.py huihui-ai/Marco-o1-abliterated --outfile huihui-ai/Marco-o1-abliterated/ggml-model-f16.gguf --outtype f16
|
24 |
```
|
25 |
+
3. Use the [llama.cpp](https://github.com/ggerganov/llama.cpp) quantitative program to quantitative model (llama-quantize needs to be compiled.),
|
26 |
+
4. [quant option](https://github.com/ggerganov/llama.cpp/blob/master/examples/quantize/quantize.cpp).
|
27 |
```
|
28 |
llama-quantize huihui-ai/Marco-o1-abliterated/ggml-model-f16.gguf huihui-ai/Marco-o1-abliterated/ggml-model-Q4_K_M.gguf Q4_K_M
|
29 |
```
|