| license: gemma | |
| base_model: | |
| - google/gemma-3-12b-it | |
| FP8-Dynamic quantization using llmcompressor. Run with: | |
| ```bash | |
| vllm serve leon-se/gemma-3-12b-it-FP8-Dynamic --max-model-len 4096 | |
| ``` |
| license: gemma | |
| base_model: | |
| - google/gemma-3-12b-it | |
| FP8-Dynamic quantization using llmcompressor. Run with: | |
| ```bash | |
| vllm serve leon-se/gemma-3-12b-it-FP8-Dynamic --max-model-len 4096 | |
| ``` |