Text Generation
GGUF
creative
creative writing
fiction writing
plot generation
sub-plot generation
story generation
scene continue
storytelling
fiction story
science fiction
romance
all genres
story
writing
vivid prosing
vivid writing
fiction
roleplaying
bfloat16
swearing
role play
128k context
horror
mistral small instruct 2409
Inference Endpoints
imatrix
conversational
Update README.md
Browse files
README.md
CHANGED
@@ -62,9 +62,9 @@ Please refer to the original model card for this model from MistralAI for additi
|
|
62 |
|
63 |
<B>Imatrix Notes:</b>
|
64 |
|
65 |
-
Imatrix
|
66 |
|
67 |
-
Due to the parameter count of this model, even IQ2s will work very well.
|
68 |
|
69 |
Q8 is not uploaded here because Imatrix has no effect on this quant.
|
70 |
|
|
|
62 |
|
63 |
<B>Imatrix Notes:</b>
|
64 |
|
65 |
+
Imatrix quants perform best at IQ3s and IQ4s, then Q4s, lower on Q5, and tappers off at Q6.
|
66 |
|
67 |
+
Due to the parameter count of this model, even IQ2s quants will work very well.
|
68 |
|
69 |
Q8 is not uploaded here because Imatrix has no effect on this quant.
|
70 |
|