Hugging Face's logo

lucadiliello
/
opt-30b-deepspeed-inference-fp16-shard-8

Text Generation
Transformers
opt
Inference Endpoints
Model card Files Community
1

The conversion script to 8 shards and 8 tensor parrallels

#1
by RobertKe - opened
Discussion
RobertKe
•
edited Jul 28, 2023

May I ask how do you convert the model to 8 shards and 8 tps? Thank you very much~

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment