Issue with all GGUF? Model format wrong?
Hi! I have this issue that I opened also here https://github.com/open-webui/open-webui/discussions/6299
Basically, if I import the model in owbui (either via the new "ollama pull hf.co/bartowski/Hermes-2-Theta-Llama-3-8B-GGUF:Q6_K" or via the cmd ollama pull for a model that's in Ollama website), the llm is NOT aware of the context, although it's being passed succesfully to it.
On the other hand, if I manually download a gguf file from here, and I import in owbui manually via the "experimental" import gguf feature, the model is AWARE of the context and provides good responses, BUT the formatting is bad.
The standard owbui modelfile format is:
TEMPLATE """{{ .System }}
USER: {{ .Prompt }}
ASSISTANT: """
PARAMETER num_ctx 4096
PARAMETER stop ""
PARAMETER stop "USER:"
PARAMETER stop "ASSISTANT:"
What could be the issue?