leon-se's picture
Update README.md
0be1d16 verified
metadata
license: gemma
base_model:
  - google/gemma-3-12b-it

FP8-Dynamic quantization using llmcompressor. Run with:

vllm serve leon-se/gemma-3-12b-it-FP8-Dynamic --max-model-len 4096