vLLM compatible?

#10
by nickandbro - opened

Would be nice to see this in vLLM

hello @nickandbro

Please make sure that you've enabled vLLM in your local apps at https://huggingface.co/settings/local-apps

Afterwards, you should be able to see vLLM snippet under Use this Model dropdown on https://huggingface.co/nvidia/Llama-3_1-Nemotron-51B-Instruct

Please let me know if it works

Sign up or log in to comment