compressed-llm

/

llama-2-13b-awq

like 0

Follow

Compressed Large Language Models 10

Hi, I came here from the leaderboard and it says this is the AWQ3 bit quantization of llama2-13b.
Since I did not find anything related to the config, and in the paper, you utilized the https://github.com/mit-han-lab/llm-awq lib for AWQ, which currently only supports 4bit quantization,
I'd like to ask for some more details on how 3bit quantization was performed.