W4A16 gs32 no act order (desc_act=false) GPTQ quant of cyberagent/Mistral-Nemo-Japanese-Instruct-2408 w/ GPTQModel 1.7.2 using augmxnt/ultra-orca-boros-en-ja-v1 as calibration set

Downloads last month: 5

Safetensors

Model size

3.09B params

Tensor type

I32

BF16

FP16

Inference Providers NEW

This model is not currently available via any of the supported third-party Inference Providers, and HF Inference API was unable to determine this model's library.

Model tree for shisa-ai/Mistral-Nemo-Japanese-Instruct-2408-GPTQ-W4A16-gs32-noact

Base model

mistralai/Mistral-Nemo-Base-2407

Finetuned

cyberagent/Mistral-Nemo-Japanese-Instruct-2408

Quantized

(10)

this model

Collection including shisa-ai/Mistral-Nemo-Japanese-Instruct-2408-GPTQ-W4A16-gs32-noact

Quantization Tests

Collection

Comparing Efficiency and Quality of various formats • 9 items • Updated 14 days ago