Text Generation
Transformers
PyTorch
gpt_bigcode
code
Eval Results
text-generation-inference

How much RAM and VRAM do I need at least to run this model?

#15
by zhwongbo - opened

How much RAM and VRAM do I need at least to run this model?

I'm running it on an A100 40GB and most of the time it's using 33GB of VRAM, peaking at 38GB.
RAM also uses 30+GB.

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment