Deci

/

DeciLM-7B

like 225

Follow

Deci AI 112

@HR1777 https://huggingface.co/Deci/DeciLM-7B-instruct-GGUF

@HR1777 https://huggingface.co/Deci/DeciLM-7B-instruct-GGUF

Thank you for that, but is there any chance we could get some lighter versions like Q6_K too? I know Q8_0 is more accurate, but it runs slow on average computers especially those without GPU.

No need for Q6_K as i am sure you are experts and Q8_0 is better. I close this thread.

@HR1777 Lower quantization than Q8_0 did not preserve accuracy in a usable manner.
Q8_0 was the lower checkpoint that generated relevant outputs. Anything below could only answer 1+1=2, etc - but not generate real-world grade text.

Yes, i understand it. Thank you so much.