Edit Models filters

Inference status

Misc

4-bit precision

AutoTrain Compatible

text-generation-inference

Inference Endpoints

8-bit precision

Mixture of Experts

text-embeddings-inference

Carbon Emissions

Models

21,155

Full-text search

Active filters: 4-bit

mlx-community/Llama-3.2-3B-Instruct-4bit

Text Generation • Updated Sep 25, 2024 • 789k • 22

mlx-community/Llama-3.2-1B-Instruct-4bit

Text Generation • Updated Oct 7, 2024 • 1.89M • 9

unsloth/Llama-3.2-1B-bnb-4bit

Text Generation • Updated Dec 10, 2024 • 44.9k • 8

unsloth/Llama-3.2-1B-Instruct-bnb-4bit

Text Generation • Updated Dec 9, 2024 • 74.3k • 12

unsloth/Llama-3.2-3B-bnb-4bit

Text Generation • Updated Dec 10, 2024 • 37.8k • 8

MaziyarPanahi/Llama-3.2-1B-Instruct-GGUF

Text Generation • Updated Sep 25, 2024 • 2.47M • 7

shuyuej/Llama-3.2-1B-Instruct-GPTQ

Updated Sep 25, 2024 • 3.01k • 2

SeanScripts/Llama-3.2-11B-Vision-Instruct-nf4

Image-Text-to-Text • Updated Sep 26, 2024 • 1.09k • 8

cyan2k/molmo-7B-D-bnb-4bit

Text Generation • Updated Sep 26, 2024 • 3.48k • 15

MikeRoz/ArliAI_Mistral-Small-22B-ArliAI-RPMax-v1.1-4.0bpw-h6-exl2

Updated Sep 26, 2024 • 10 • 3

SeanScripts/Molmo-72B-0924-nf4

Image-Text-to-Text • Updated Oct 2, 2024 • 224 • 7

ToastyPigeon/mistral-small-springdragon-qlora

Updated Oct 7, 2024 • 5 • 1

suayptalha/Komodo-7B-Instruct

Text Generation • Updated 22 days ago • 159 • 5

Shaleen123/llama3.2-3b-medical

Text Generation • Updated Oct 6, 2024 • 499 • 2

ibrahimkettaneh/Qwen2.5-Coder-7B-Instruct-abliterated-AWQ

Updated Oct 8, 2024 • 11 • 2

ibrahimkettaneh/Qwen2.5-72B-Instruct-abliterated-AWQ-Marlin

Updated Oct 9, 2024 • 7 • 2

thanhkt/codegemma-7B-ManimGen

Text Generation • Updated Oct 14, 2024 • 122 • 4

AIDC-AI/Ovis1.6-Gemma2-9B-GPTQ-Int4

Image-Text-to-Text • Updated Nov 25, 2024 • 292 • 9

hugging-quants/gemma-2-9b-it-AWQ-INT4

Text Generation • Updated Oct 17, 2024 • 1.01k • 6

mlx-community/Ministral-8B-Instruct-2410-4bit

Updated Oct 17, 2024 • 193 • 7

unsloth/Llama-3.1-Nemotron-70B-Instruct-bnb-4bit

Text Generation • Updated Oct 17, 2024 • 653 • 19

luigi86/Midnight-Miqu-70B-v1.5_mlx-4bpw

Text Generation • Updated Oct 18, 2024 • 34 • 3

MaziyarPanahi/Llama-3.1-8B-Lexi-Uncensored-V2-GGUF

Text Generation • Updated Oct 20, 2024 • 184 • 2

mlx-community/Qwen2.5-7B-Instruct-Uncensored-4bit

Text Generation • Updated Oct 21, 2024 • 158 • 3

OpenLLM-Ro/RoLlama3.1-8b-Instruct-DPO-4Bit-BB

Updated 29 days ago • 10 • 1

OpenLLM-Ro/RoGemma2-9b-Instruct-DPO-4Bit-BB

Updated 29 days ago • 14 • 1

ibnzterrell/Nvidia-Llama-3.1-Nemotron-70B-Instruct-HF-AWQ-INT4

Text Generation • Updated Dec 7, 2024 • 156 • 2

yunguks/walk1009-4bit

Text Generation • Updated Oct 27, 2024 • 10 • 1

unsloth/SmolLM2-1.7B-bnb-4bit

Text Generation • Updated Oct 31, 2024 • 336 • 1

MaziyarPanahi/Llama-3.2-1B-GGUF

Text Generation • Updated Nov 2, 2024 • 65 • 1