"Supports a context length of 160k through yarn settings."

#1
by mclassHF2023 - opened

How would this be achieved precisely? I thought with GGUF files there is no config.json anymore?

Supports a context length of 160k through yarn settings.

Btw: I would be happy with about 16k context already, it doesn't have to be 160k context.

LM Studio Community org

I think you should just need to set the RoPE Frequency Scale when loading the model in LM Studio (if you hold alt while loading it you should be able to set a higher context length and use the RoPE frequency scale for better scaling)

if you want 16k context I think you want a frequency scale of 4

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment