JamePeng2023 commited on
Commit
eeeeeea
·
verified ·
1 Parent(s): 72d7fe9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -3,6 +3,10 @@ license: llama3.2
3
  base_model:
4
  - huihui-ai/Llama-3.2-3B-Instruct-abliterated
5
  ---
6
- Quantize the model from https://huggingface.co/huihui-ai/Llama-3.2-3B-Instruct-abliterated using the SpinQuant quantization method from https://github.com/facebookresearch/SpinQuant for on-device deployment in an Android app with Executorch.
 
 
 
 
7
 
8
  2025-02-16 19:40:24,099 - spinquant - INFO - wiki2 ppl is: 11.502239227294922
 
3
  base_model:
4
  - huihui-ai/Llama-3.2-3B-Instruct-abliterated
5
  ---
6
+ Using the SpinQuant quantization method from https://github.com/facebookresearch/SpinQuant, I quantized the Llama-3.2-3B-Instruct-abliterated model from https://huggingface.co/huihui-ai/Llama-3.2-3B-Instruct-abliterated.
7
+
8
+ This quantization is for on-device deployment to Android apps with Executorch.
9
+
10
+ To make it easier for everyone to quickly test and deploy the Executorch on-device demo, I've also converted the quantized PTH file to PTE format and uploaded it.
11
 
12
  2025-02-16 19:40:24,099 - spinquant - INFO - wiki2 ppl is: 11.502239227294922