Context size clarification
#1
by
notafraud
- opened
Hi! I saw you comment in https://huggingface.co/lmstudio-community/c4ai-command-a-03-2025-GGUF/discussions/1 about the context there. I assume the context window is correct (256k) in your quantized variants, right?
The default limit is set to 16k, cohere updated after the fact to 128k, but in any case you should be able to safely override it to whatever you'd like !
Thank you! Alas, we hoped for actual 256k (64k at least!). I was told that this model is a solid replacement for Mistral Large (finetunes even).
It should support the full 256k! You just need to set it at load time (lm studio should work, just have to edit the box to the value you want instead of using the slider)