bdambrosio
/

Mixtral-8x22b-instruct-oh-6.0bpw-exl2

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Mixtral-8x22b-instruct-oh-6.0bpw-exl2 / README.md

bdambrosio's picture

Update README.md

a00e6dc verified 10 months ago

|

history blame contribute delete

182 Bytes

metadata

license: apache-2.0

6.0 bit exl2 quant (8 vut head) of Fireworks Hermes 2.5 fine tune of Mixtral-8x22b

Use Vicuna prompt template

needs ~ 120GB vRam (2xA100 or 3X RTX 6000)