Model only outputs "!!!!!!!!!!"
#1
by
mrhendrey
- opened
Launched the model using vLLM (v0.6.4.post1), but the model only generates a string of "!!!!!" until the context window fills up. Any thoughts on what might be causing this behavior? Thanks in advance.
Hi
@mrhendrey
- could you try updating the version of vllm you're using? You'll also need to set the dtype to float16 when running the model.
Thanks