Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss 3.5bpw

Exllama quant of NeverSleep/Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss

You will need 24gb of vram to run this model at about half context (16k, you can probably go a bit higher too)

Prompt format: Chatml

<|im_start|>system
{sysprompt}<|im_end|>
<|im_start|>user
{input}<|im_end|>
<|im_start|>assistant
{output}<|im_end|>

Contact

Kooten on discord.

Downloads last month: 20

Inference Examples

Text Generation

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Collection including Kooten/Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss-3.5bpw-exl2

NeverSleep / Noromaid

Collection

Great RP model, if you don't know where to start get one of these. • 20 items • Updated Feb 1