Quantization?

#1
by YaTharThShaRma999 - opened

Is it possible to quantize this model to gptq or gguf? Also how much context can this model handle?

Thanks for releasing such a model(:

YaTharThShaRma999 changed discussion title from Quantizarion? to Quantization?

We will release the gptq quantized model along with the finetuned chat model following up.

Oh ok, Thanks a lot!

I'm closing this for now. If you have any further question, feel free to create an issue under our github repo. 🤗

FancyZhao changed discussion status to closed

Sign up or log in to comment