How serving?
#218
by
yongho1213
- opened
It is to big for serving.
when i serving using h100*16 by vllm.
It occured oom.
is it need minimum gpu is h100 * 32?
It is to big for serving.
when i serving using h100*16 by vllm.
It occured oom.
is it need minimum gpu is h100 * 32?