Draft Model of Speculative Decoding

#6
by nagug - opened

Do you have any suggestions of which draft models would play nicely with this mode. BTW. Qwen2.5 7B instruct seem to have different vocab size and not working. May be i am doing something wrong.

Sign up or log in to comment