Can you provide a FP8 version?
#11
by
xjpang85
- opened
Can you provide a FP8 version for less GPUs.
Can the int8 or int4 version meet the requirements?
int4 ok
Hi
@xjpang85
, our vllm support has been merged today. Feel free to use it with our Text-01 model
https://github.com/vllm-project/vllm/pull/13454