not working with webui and koldbold cpp
#1
by
gopi87
- opened
hi this model is not working with webui and koldbold cpp any idea why its happened ?
hey @gopi87 ,
I've just tested it in the latest llama.cpp
and it appears to work:
$ llama.cpp/main -m Higgs-Llama-3-70B.Q2_K.gguf -ngl 60 --special --prompt "<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\nwhat is 1337 ^ 2?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n"
<|begin_of_text|><|begin_of_text|><|start_header_id|>user<|end_header_id|>
what is 1337 ^ 2?<|eot_id|><|start_header_id|>assistant<|end_header_id|>
1337^2 is equal to 1,785,769. This is because when you multiply any number by itself (in this case 1337), you are squaring that number. So 1337 x 1337 results in the calculation of 1,785,769. This value represents the area of a square with sides of length 1337.<|eot_id|> [end of text]
Can you confirm that you're running on the latest koboldcpp?
As a sidenote, I did observe that the model at low quants refused to generate text if the header of assistant
(<|start_header_id|>assistant<|end_header_id|>\n\n
) was not present in the prompt. Maybe you can try adding that to your prompt
hi sorry for this its working well in kolboldcpp
gopi87
changed discussion status to
closed