Update README.md
Browse files
README.md
CHANGED
|
@@ -139,10 +139,14 @@ Special thanks to [Jon Durbin](https://huggingface.co/jondurbin), [Intel](https:
|
|
| 139 |
|
| 140 |
This model uses a context window of 8k. I recommend using it with the Mistral Instruct chat template (works perfectly with LM Studio).
|
| 141 |
|
|
|
|
|
|
|
| 142 |
It is one of the very best 7B models in terms of instructing following and reasoning abilities and can be used for conversations, RP, and storytelling. Note that it tends to have a quite formal and sophisticated style, but it can be changed by modifying the prompt.
|
| 143 |
|
| 144 |
## ⚡ Quantized models
|
| 145 |
|
|
|
|
|
|
|
| 146 |
* **GGUF**: https://huggingface.co/mlabonne/AlphaMonarch-7B-GGUF
|
| 147 |
* **GPTQ**: https://huggingface.co/LoneStriker/AlphaMonarch-7B-GPTQ
|
| 148 |
* **AWQ**: https://huggingface.co/LoneStriker/AlphaMonarch-7B-AWQ
|
|
|
|
| 139 |
|
| 140 |
This model uses a context window of 8k. I recommend using it with the Mistral Instruct chat template (works perfectly with LM Studio).
|
| 141 |
|
| 142 |
+
If you use SillyTavern, you might want to tweak the inference parameters. Here's what LM Studio uses as a reference: `temp` 0.8, `top_k` 40, `top_p` 0.95, `min_p` 0.05, `repeat_penalty` 1.1.
|
| 143 |
+
|
| 144 |
It is one of the very best 7B models in terms of instructing following and reasoning abilities and can be used for conversations, RP, and storytelling. Note that it tends to have a quite formal and sophisticated style, but it can be changed by modifying the prompt.
|
| 145 |
|
| 146 |
## ⚡ Quantized models
|
| 147 |
|
| 148 |
+
Thanks to [LoneStriker](https://huggingface.co/LoneStriker) for the GPTQ, AWQ, and EXL2 quants.
|
| 149 |
+
|
| 150 |
* **GGUF**: https://huggingface.co/mlabonne/AlphaMonarch-7B-GGUF
|
| 151 |
* **GPTQ**: https://huggingface.co/LoneStriker/AlphaMonarch-7B-GPTQ
|
| 152 |
* **AWQ**: https://huggingface.co/LoneStriker/AlphaMonarch-7B-AWQ
|