Hugging Face's logo

uisikdag
/
SmolVLM-Instruct-4bit-bitsnbytes-nf4

Image-Text-to-Text
Transformers
Safetensors
English
idefics3
conversational
4-bit precision
bitsandbytes
Model card Files Community

4bit nf4 quantized version, you can find the quantized version generation code below.

from transformers import BitsAndBytesConfig


nf4_config = BitsAndBytesConfig(
   load_in_4bit=True,
   bnb_4bit_quant_type="nf4",
   bnb_4bit_use_double_quant=True,
   bnb_4bit_compute_dtype=torch.bfloat16
)

model_nf4 = AutoModelForVision2Seq.from_pretrained("HuggingFaceTB/SmolVLM-Instruct", quantization_config=nf4_config)
Downloads last month
29
Safetensors
Model size
1.26B params
Tensor type
F32
·
FP16
·
U8
·
Inference Providers NEW
Image-Text-to-Text
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for uisikdag/SmolVLM-Instruct-4bit-bitsnbytes-nf4

Base model

HuggingFaceTB/SmolLM2-1.7B
Quantized
HuggingFaceTB/SmolLM2-1.7B-Instruct
Quantized
HuggingFaceTB/SmolVLM-Instruct
Quantized
(4)
this model

Collection including uisikdag/SmolVLM-Instruct-4bit-bitsnbytes-nf4