metadata
license: llama3.1
EXL2 quants of Llama-3.1 8B instruct
This model requires the dev branch of ExLlamaV2 for now. New release coming soon with the necessary changes.
3.00 bits per weight
3.50 bits per weight
4.00 bits per weight
4.50 bits per weight
5.00 bits per weight
6.00 bits per weight
8.00 bits per weight