No support to quantize gemma model in autoawq
#1
by
Abhinav6310
- opened
how do you quantize in autoawq since there is no support to quantize in awq using autoawq
Casper-Hansen previously created a branch that added support for Gemma2-9b in AutoAWQ, but I'm not sure why he didn't merge that change into the main branch. However, there is now a pull request that implements this support. You can check it out here: https://github.com/casper-hansen/AutoAWQ/blob/main/awq/models/gemma2.py
Thanks man
Abhinav6310
changed discussion status to
closed