---
license: other
license_name: gemma-terms-of-use
license_link: https://ai.google.dev/gemma/terms
tags:
- gemma
- gguf
---

# Gemma 2B Instruct GGUF

Contains Q4 & Q8 quantized GGUFs for [google/gemma](https://huggingface.co/collections/google/gemma-release-65d5efbccdbb8c4202ec078b)

## Perf

| Variant | Device | Perf |
| - | - | - |
| Q4 |  M1 Pro 10-core GPU | 90 tok/s |
|  |  Snapdragon 778G CPU | 10 tok/s |
| | RTX 2070S | 40 tok/s |
| Q8 | M1 Pro 10-core GPU | 54 tok/s |
|  |  Snapdragon 778G CPU | 6 tok/s |
| | RTX 2070S | 25 tok/s |
| F16 | M1 Pro 10-core GPU | 30 tok/s|
|  |  Snapdragon 778G CPU | <1 tok/s |