Update README.md

README.md

@@ -15,8 +15,26 @@ tags:
 license: apache-2.0
 language:
 - en
+library_name: exllamav2
 ---
-
+# Granite-3.1-8B-Reasoning-exl2
+Original model: [granite-3.1-8b-Reasoning](https://huggingface.co/ruslanmv/granite-3.1-8b-Reasoning) by [https://huggingface.co/ruslanmv](https://huggingface.co/ruslanmv)  
+Based on: [granite-3.1-8b-instruct](https://huggingface.co/ibm-granite/granite-3.1-8b-instruct) by [Granite Team, IBM](https://huggingface.co/ibm-granite)
+
+## Quants
+[4bpw h6 (main)](https://huggingface.co/cgus/granite-3.1-8b-Reasoning-exl2/tree/main)  
+[4.5bpw h6](https://huggingface.co/cgus/granite-3.1-8b-Reasoning-exl2/tree/4.5bpw-h6)  
+[5bpw h6](https://huggingface.co/cgus/granite-3.1-8b-Reasoning-exl2/tree/5bpw-h6)  
+[6bpw h6](https://huggingface.co/cgus/granite-3.1-8b-Reasoning-exl2/tree/6bpw-h6)  
+[8bpw h8](https://huggingface.co/cgus/granite-3.1-8b-Reasoning-exl2/tree/8bpw-h8)  
+
+## Quantization notes
+Made with Exllamav2 0.2.8 with default dataset. These quants require Exllamav2 0.2.7 or newer.  
+They meant to be used with apps that support exl2 models such as TabbyAPI, Text-Generation-WebUI and others.  
+On Windows it requires a Nvidia RTX2xxx or newer GPU, on Linux it can be used with Nvidia RTX or AMD ROCm cards.  
+Models are required to be fully loaded into GPU, native RAM offloading isn't supported.  
+If you need RAM offloading or have some other GPU, try GGUF quants instead.
+# Original model card
 # Granite-3.1-8B-Reasoning (Fine-Tuned for Advanced Reasoning)
 
 ## Model Overview
@@ -162,5 +180,4 @@ If you use this model in your research or applications, please cite:
   year={2025},
   url={https://huggingface.co/ruslanmv/granite-3.1-8b-Reasoning}
 }
-```
-
+```