Update README.md
Browse files
README.md
CHANGED
@@ -27,7 +27,7 @@ This is a test model on a the following
|
|
27 |
- a private dataset
|
28 |
- slight customization on alpaca chat template
|
29 |
- Works with Ollama create but requires customization to Modelfile
|
30 |
-
- One reason for this was wanted to try doing Q2_K and see if it was actually good(?)
|
31 |
- My examples will be based on unslot.Q2_K.GGUF file, however other quantization should work as well
|
32 |
|
33 |
# HOW TO USE
|
@@ -35,7 +35,14 @@ This is a test model on a the following
|
|
35 |
The whole point of conversion for me was I wanted to be able to to use it through Ollama or (other local options)
|
36 |
For Ollama, it required to be a GGUF file. Once you have this it is pretty straight forward
|
37 |
|
38 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
39 |
- You must already have Ollama running in your setting
|
40 |
- Download the unsloth.Q2_K.gguf model from Files
|
41 |
- In the same directory create a file call "Modelfile"
|
|
|
27 |
- a private dataset
|
28 |
- slight customization on alpaca chat template
|
29 |
- Works with Ollama create but requires customization to Modelfile
|
30 |
+
- One reason for this was wanted to try doing Q2_K and see if it was actually good(?) -> Exceeds Expectation!!
|
31 |
- My examples will be based on unslot.Q2_K.GGUF file, however other quantization should work as well
|
32 |
|
33 |
# HOW TO USE
|
|
|
35 |
The whole point of conversion for me was I wanted to be able to to use it through Ollama or (other local options)
|
36 |
For Ollama, it required to be a GGUF file. Once you have this it is pretty straight forward
|
37 |
|
38 |
+
If you want to try it first, the Q2_K version of this is available in Ollama => deeokay/gemma2custom
|
39 |
+
|
40 |
+
```python
|
41 |
+
ollama pull deeokay/gemma2custom
|
42 |
+
```
|
43 |
+
|
44 |
+
|
45 |
+
# Quick Start:
|
46 |
- You must already have Ollama running in your setting
|
47 |
- Download the unsloth.Q2_K.gguf model from Files
|
48 |
- In the same directory create a file call "Modelfile"
|