GGUF conversion issue
#1
by
shigurd
- opened
Hi, im having issues with converting this model into GGUF. Im using llama.ccp convert_hf_to_gguf_update.py. Initially got an error due to "- no" in the readme file, but solved it by modifying it (I can see that this has been patched now).
I was able to create the GGUF files, but they end up outputting gibberish (for instance: 122222223555555555555555555541111111111111111111111111111111) for both f32, f16 and Q8_0 when used in ollama. I have tested the llama.ccp script for GGUF conversion of other llama3.2-3B models without any issues like this.
Also tested the model in HF transformers which otherwise works perfectly fine. Anyone have any advice on what im doing wrong?