Upload folder using huggingface_hub

README.md

@@ -9,5 +9,42 @@ tags:
 # Qwen-VL-Chat-MNN
 
 ## Introduction
+This model is a 4-bit quantized version of the MNN model exported from [Qwen-VL-Chat](https://modelscope.cn/models/qwen/Qwen-VL-Chat/summary) using [llmexport](https://github.com/alibaba/MNN/tree/master/transformers/llm/export).
 
-This model is a 4-bit quantized version of the MNN model exported from Qwen-VL-Chat using [llm-export](https://github.com/wangzhaode/llm-export).
+## Download
+```bash
+# install huggingface
+pip install huggingface
+```
+```bash
+# shell download
+huggingface download --model 'taobao-mnn/Qwen-VL-Chat-MNN' --local_dir 'path/to/dir'
+```
+```python
+# SDK download
+from huggingface_hub import snapshot_download
+model_dir = snapshot_download('taobao-mnn/Qwen-VL-Chat-MNN')
+```
+
+```bash
+# git clone
+git clone https://www.modelscope.cn/taobao-mnn/Qwen-VL-Chat-MNN
+```
+
+## Usage
+```bash
+# clone MNN source
+git clone https://github.com/alibaba/MNN.git
+
+# compile
+cd MNN
+mkdir build && cd build
+cmake .. -DMNN_LOW_MEMORY=true -DMNN_CPU_WEIGHT_DEQUANT_GEMM=true -DMNN_BUILD_LLM=true -DMNN_SUPPORT_TRANSFORMER_FUSE=true
+make -j
+
+# run
+./llm_demo /path/to/Qwen-VL-Chat-MNN/config.json prompt.txt
+```
+
+## Document
+[MNN-LLM](https://mnn-docs.readthedocs.io/en/latest/transformers/llm.html#)

config.json

@@ -4,5 +4,11 @@
     "backend_type": "cpu",
     "thread_num": 4,
     "precision": "low",
-    "memory": "low"
+    "memory": "low",
+    "mllm": {
+        "backend_type": "cpu",
+        "thread_num": 4,
+        "precision": "low",
+        "memory": "low"
+    }
 }

llm.mnn

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9b82d2d344a1c53950074f8b0a1e6f8fa9fd0fe70b99a25c0ff164dee05e9759
-size 1567904
+oid sha256:f88cc299570943361d8abc41015c4fe89e501254c6b3bbe7315bb6edffa2b984
+size 2630400

llm.mnn.weight

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7ce25adc9b09b6fe0c8e7fe3155b409cfb27c7b8d4b658a2f0f65965001e35e9
+oid sha256:7dfcd413c69dbdeca8b3456016136aa55857d547d71b69be049a08c3ea5c6fc3
 size 3994391386

visual.mnn

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5ff8ad74df0aa7846f364cfdacbe68e01c8b24aa62f4109ba18d285e6ccf67ea
-size 17084864
+oid sha256:ec0bcf800a7a370d4ab731c6ed2dba1106728fe34374ef198f2be3cbf43df5b5
+size 17015352