qwp4w3hyb
/

gemma-2-9b-it-iMat-GGUF

Text Generation

Inference Endpoints

Model card Files Files and versions Community

qwp4w3hyb commited on Jun 28, 2024

Commit

056a9fb

·

verified ·

1 Parent(s): 1efcf9c

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -19,8 +19,8 @@ base_model: google/gemma-2-9b-it
   - WIP new version should have better metadata as its quantized from scratch with llama.cpp
 - Wide coverage of different gguf quant types from Q\_8\_0 down to IQ1\_S
   - experimental custom quant types
-    - `_L` with `--output-tensor-type f16 --token-embedding-type f16` (same as bartowski)
-    - `_XL` with `--output-tensor-type bf16 --token-embedding-type bf16` (same size as _L, in theory even higher numerical accuracy)
 - Quantized with [llama.cpp](https://github.com/ggerganov/llama.cpp) release [b3259](https://github.com/ggerganov/llama.cpp/releases/tag/b3259)
 - Imatrix generated with [this](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8) multi-purpose dataset by [bartowski](https://huggingface.co/bartowski).
   ```

   - WIP new version should have better metadata as its quantized from scratch with llama.cpp
 - Wide coverage of different gguf quant types from Q\_8\_0 down to IQ1\_S
   - experimental custom quant types
+    - `_L` with `--output-tensor-type f16 --token-embedding-type f16` (same as bartowski's)
+    - `_XL` with `--output-tensor-type bf16 --token-embedding-type bf16` (same size as _L, in theory even closer to the source model)
 - Quantized with [llama.cpp](https://github.com/ggerganov/llama.cpp) release [b3259](https://github.com/ggerganov/llama.cpp/releases/tag/b3259)
 - Imatrix generated with [this](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8) multi-purpose dataset by [bartowski](https://huggingface.co/bartowski).
   ```