File size: 193 Bytes
85b8e47
 
 
 
 
0be1d16
 
 
 
1
2
3
4
5
6
7
8
9
---
license: gemma
base_model:
- google/gemma-3-12b-it
---
FP8-Dynamic quantization using llmcompressor. Run with:
```bash
vllm serve leon-se/gemma-3-12b-it-FP8-Dynamic --max-model-len 4096
```