Update README.md

README.md

@@ -2,14 +2,30 @@
 pipeline_tag: text-generation
 inference: false
 license: apache-2.0
-library_name: transformers
+library_name: exllamav2
 tags:
 - language
 - granite-3.2
 base_model:
-- ibm-granite/granite-3.1-2b-instruct
+- ibm-granite/granite-3.2-2b-instruct
 ---
-
+# Granite-3.2-2B-Instruct-exl2
+Original model: [granite-3.2-2b-instruct](https://huggingface.co/ibm-granite/granite-3.2-2b-instruct)  
+Created by: [Granite Team, IBM](https://huggingface.co/ibm-granite)
+
+## Quants
+[4bpw h6 (main)](https://huggingface.co/cgus/granite-3.2-2b-instruct-exl2/tree/main)  
+[4.5bpw h6](https://huggingface.co/cgus/granite-3.2-2b-instruct-exl2/tree/4.5bpw-h6)  
+[5bpw h6](https://huggingface.co/cgus/granite-3.2-2b-instruct-exl2/tree/5bpw-h6)  
+[6bpw h6](https://huggingface.co/cgus/granite-3.2-2b-instruct-exl2/tree/6bpw-h6)  
+[8bpw h8](https://huggingface.co/cgus/granite-3.2-2b-instruct-exl2/tree/8bpw-h8)  
+
+## Quantization notes
+Made with Exllamav2 0.2.8 with the default dataset. Granite3 exl2 models require Exllamav2 0.2.7 or newer.  
+Exl2 models have to be fully loaded into GPU VRAM, native RAM offloading isn't supported.  
+These models require Nvidia RTX on Windows or Nvidia RTX/AMD ROCm on Linux.
+
+# Original model card
 # Granite-3.2-2B-Instruct
 
 **Model Summary:**