Update README.md
Browse files
README.md
CHANGED
|
@@ -3,4 +3,7 @@ license: gemma
|
|
| 3 |
base_model:
|
| 4 |
- google/gemma-3-27b-it
|
| 5 |
---
|
| 6 |
-
FP8-Dynamic quantization using llmcompressor.
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
base_model:
|
| 4 |
- google/gemma-3-27b-it
|
| 5 |
---
|
| 6 |
+
FP8-Dynamic quantization using llmcompressor. Run with:
|
| 7 |
+
```bash
|
| 8 |
+
vllm serve leon-se/gemma-3-12b-it-FP8-Dynamic --max-model-len 4096
|
| 9 |
+
```
|