legraphista
commited on
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
@@ -18,6 +18,7 @@ tags:
|
|
18 |
- imat
|
19 |
- imatrix
|
20 |
- static
|
|
|
21 |
- 8bit
|
22 |
- 6bit
|
23 |
- 5bit
|
@@ -32,7 +33,7 @@ _Llama.cpp imatrix quantization of google/datagemma-rag-27b-it_
|
|
32 |
|
33 |
Original Model: [google/datagemma-rag-27b-it](https://huggingface.co/google/datagemma-rag-27b-it)
|
34 |
Original dtype: `BF16` (`bfloat16`)
|
35 |
-
Quantized by: llama.cpp [
|
36 |
IMatrix dataset: [here](https://gist.githubusercontent.com/bartowski1182/eb213dccb3571f863da82e99418f81e8/raw/b2869d80f5c16fd7082594248e80144677736635/calibration_datav3.txt)
|
37 |
|
38 |
- [Files](#files)
|
@@ -68,6 +69,8 @@ Link: [here](https://huggingface.co/legraphista/datagemma-rag-27b-it-IMat-GGUF/b
|
|
68 |
### All Quants
|
69 |
| Filename | Quant type | File Size | Status | Uses IMatrix | Is Split |
|
70 |
| -------- | ---------- | --------- | ------ | ------------ | -------- |
|
|
|
|
|
71 |
| [datagemma-rag-27b-it.Q8_0.gguf](https://huggingface.co/legraphista/datagemma-rag-27b-it-IMat-GGUF/blob/main/datagemma-rag-27b-it.Q8_0.gguf) | Q8_0 | 28.94GB | ✅ Available | ⚪ Static | 📦 No
|
72 |
| [datagemma-rag-27b-it.Q6_K.gguf](https://huggingface.co/legraphista/datagemma-rag-27b-it-IMat-GGUF/blob/main/datagemma-rag-27b-it.Q6_K.gguf) | Q6_K | 22.34GB | ✅ Available | ⚪ Static | 📦 No
|
73 |
| [datagemma-rag-27b-it.Q5_K.gguf](https://huggingface.co/legraphista/datagemma-rag-27b-it-IMat-GGUF/blob/main/datagemma-rag-27b-it.Q5_K.gguf) | Q5_K | 19.41GB | ✅ Available | ⚪ Static | 📦 No
|
|
|
18 |
- imat
|
19 |
- imatrix
|
20 |
- static
|
21 |
+
- 16bit
|
22 |
- 8bit
|
23 |
- 6bit
|
24 |
- 5bit
|
|
|
33 |
|
34 |
Original Model: [google/datagemma-rag-27b-it](https://huggingface.co/google/datagemma-rag-27b-it)
|
35 |
Original dtype: `BF16` (`bfloat16`)
|
36 |
+
Quantized by: llama.cpp [b3750](https://github.com/ggerganov/llama.cpp/releases/tag/b3750)
|
37 |
IMatrix dataset: [here](https://gist.githubusercontent.com/bartowski1182/eb213dccb3571f863da82e99418f81e8/raw/b2869d80f5c16fd7082594248e80144677736635/calibration_datav3.txt)
|
38 |
|
39 |
- [Files](#files)
|
|
|
69 |
### All Quants
|
70 |
| Filename | Quant type | File Size | Status | Uses IMatrix | Is Split |
|
71 |
| -------- | ---------- | --------- | ------ | ------------ | -------- |
|
72 |
+
| datagemma-rag-27b-it.BF16 | BF16 | - | ⏳ Processing | ⚪ Static | -
|
73 |
+
| datagemma-rag-27b-it.FP16 | F16 | - | ⏳ Processing | ⚪ Static | -
|
74 |
| [datagemma-rag-27b-it.Q8_0.gguf](https://huggingface.co/legraphista/datagemma-rag-27b-it-IMat-GGUF/blob/main/datagemma-rag-27b-it.Q8_0.gguf) | Q8_0 | 28.94GB | ✅ Available | ⚪ Static | 📦 No
|
75 |
| [datagemma-rag-27b-it.Q6_K.gguf](https://huggingface.co/legraphista/datagemma-rag-27b-it-IMat-GGUF/blob/main/datagemma-rag-27b-it.Q6_K.gguf) | Q6_K | 22.34GB | ✅ Available | ⚪ Static | 📦 No
|
76 |
| [datagemma-rag-27b-it.Q5_K.gguf](https://huggingface.co/legraphista/datagemma-rag-27b-it-IMat-GGUF/blob/main/datagemma-rag-27b-it.Q5_K.gguf) | Q5_K | 19.41GB | ✅ Available | ⚪ Static | 📦 No
|