Mistral-Small-24B-Base-2501-GGUF

This repo provides two GGUF quantizations of mistralai/Mistral-Small-24B-Base-2501:

Filename	File size	Description	TLDR
Mistral-Small-24B-Base-2501-q8_0-q4_K_S.gguf	14.05GB	q4_K_S quantization using q8_0 for token embeddings and output tensors	Good quality, smaller size
Mistral-Small-24B-Base-2501-q8_0-q6_K.gguf	19.67GB	q6_K quantization using q8_0 for token embeddings and output tensors	Practically perfect quality, larger size

Downloads last month: 105

GGUF

Model size

23.6B params

Architecture

llama

4-bit

6-bit

Inference Providers NEW

This model is not currently available via any of the supported third-party Inference Providers, and HF Inference API was unable to determine this model's library.

Model tree for ddh0/Mistral-Small-24B-Base-2501-GGUF

Base model

mistralai/Mistral-Small-24B-Base-2501

Quantized

(24)

this model