Model produces garbage
#2
by
WaveCut
- opened
I'm using the Q4_1 quant variant, and regardless of what settings I've tried, I consistently receive the same output of endless @ until I stop the generation.
There is something wrong with it. How do I resolve this issue?
Unfortunately, there are ongoing issues with llama3 tokenization in GGUF and I'm regenerating quants as potential fixes come in. Until then, I would recommend the mlx quants if you're on max otherwise exl2.