Hugging Face's logo

ChenMnZ
/
Llama-3-70b-instruct-EfficientQAT-w2g64-GPTQ

Text Generation
Transformers
Safetensors
llama
conversational
text-generation-inference
Inference Endpoints
2-bit
gptq
Model card Files Community
1

not run

#1
by rakmik - opened
Discussion
rakmik

in colab t4
https://github.com/werruww/EfficientQAT-w2g128-GPTQ-cpu/blob/main/%D9%92%D9%92XXLlama_3_70b_instruct_EfficientQAT_w2g64_GPTQ.ipynb

Is there a solution to run
ChenMnZ/Llama-3-70b-instruct-EfficientQAT-w2g64-GPTQ
on gpu 16 gb?

rakmik

not run never

colab t4

https://github.com/werruww/EfficientQAT-w2g128-GPTQ-cpu/blob/main/XLlama_3_70b_instruct_EfficientQAT_w2g64_GPTQ.ipynb

Sign up or log in to comment