qaihm-bot commited on
Commit
d105c01
·
verified ·
1 Parent(s): ee5533b

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +234 -0
README.md ADDED
@@ -0,0 +1,234 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ datasets:
3
+ - imagenet-1k
4
+ - imagenet-22k
5
+ library_name: pytorch
6
+ license: bsd-3-clause
7
+ pipeline_tag: image-classification
8
+ tags:
9
+ - backbone
10
+ - quantized
11
+ - android
12
+
13
+ ---
14
+
15
+ ![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/vit_quantized/web-assets/model_demo.png)
16
+
17
+ # VITQuantized: Optimized for Mobile Deployment
18
+ ## Imagenet classifier and general purpose backbone
19
+
20
+ VIT is a machine learning model that can classify images from the Imagenet dataset. It can also be used as a backbone in building more complex models for specific use cases.
21
+
22
+ This model is an implementation of VITQuantized found [here]({source_repo}).
23
+ This repository provides scripts to run VITQuantized on Qualcomm® devices.
24
+ More details on model performance across various devices, can be found
25
+ [here](https://aihub.qualcomm.com/models/vit_quantized).
26
+
27
+
28
+ ### Model Details
29
+
30
+ - **Model Type:** Image classification
31
+ - **Model Stats:**
32
+ - Model checkpoint: Imagenet
33
+ - Input resolution: 224x224
34
+ - Number of parameters: 86.6M
35
+ - Model size: 85.9 MB
36
+
37
+ | Model | Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Precision | Primary Compute Unit | Target Model
38
+ |---|---|---|---|---|---|---|---|---|
39
+ | VITQuantized | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | QNN | 5.499 ms | 0 - 30 MB | INT8 | NPU | [VITQuantized.so](https://huggingface.co/qualcomm/VITQuantized/blob/main/VITQuantized.so) |
40
+ | VITQuantized | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | ONNX | 43.244 ms | 0 - 5 MB | INT8 | NPU | [VITQuantized.onnx](https://huggingface.co/qualcomm/VITQuantized/blob/main/VITQuantized.onnx) |
41
+ | VITQuantized | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 3.591 ms | 0 - 57 MB | INT8 | NPU | [VITQuantized.so](https://huggingface.co/qualcomm/VITQuantized/blob/main/VITQuantized.so) |
42
+ | VITQuantized | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | ONNX | 32.761 ms | 0 - 763 MB | INT8 | NPU | [VITQuantized.onnx](https://huggingface.co/qualcomm/VITQuantized/blob/main/VITQuantized.onnx) |
43
+ | VITQuantized | RB3 Gen 2 (Proxy) | QCS6490 Proxy | QNN | 22.428 ms | 0 - 8 MB | INT8 | NPU | Use Export Script |
44
+ | VITQuantized | QCS8550 (Proxy) | QCS8550 Proxy | QNN | 4.939 ms | 0 - 1 MB | INT8 | NPU | Use Export Script |
45
+ | VITQuantized | SA8255 (Proxy) | SA8255P Proxy | QNN | 4.931 ms | 0 - 1 MB | INT8 | NPU | Use Export Script |
46
+ | VITQuantized | SA8775 (Proxy) | SA8775P Proxy | QNN | 4.937 ms | 0 - 1 MB | INT8 | NPU | Use Export Script |
47
+ | VITQuantized | QCS8450 (Proxy) | QCS8450 Proxy | QNN | 6.251 ms | 0 - 57 MB | INT8 | NPU | Use Export Script |
48
+ | VITQuantized | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | QNN | 3.394 ms | 0 - 72 MB | INT8 | NPU | Use Export Script |
49
+ | VITQuantized | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | ONNX | 27.705 ms | 0 - 324 MB | INT8 | NPU | [VITQuantized.onnx](https://huggingface.co/qualcomm/VITQuantized/blob/main/VITQuantized.onnx) |
50
+ | VITQuantized | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 5.356 ms | 0 - 0 MB | INT8 | NPU | Use Export Script |
51
+ | VITQuantized | Snapdragon X Elite CRD | Snapdragon® X Elite | ONNX | 58.341 ms | 228 - 228 MB | INT8 | NPU | [VITQuantized.onnx](https://huggingface.co/qualcomm/VITQuantized/blob/main/VITQuantized.onnx) |
52
+
53
+
54
+
55
+
56
+ ## Installation
57
+
58
+ This model can be installed as a Python package via pip.
59
+
60
+ ```bash
61
+ pip install qai-hub-models
62
+ ```
63
+
64
+
65
+ ## Configure Qualcomm® AI Hub to run this model on a cloud-hosted device
66
+
67
+ Sign-in to [Qualcomm® AI Hub](https://app.aihub.qualcomm.com/) with your
68
+ Qualcomm® ID. Once signed in navigate to `Account -> Settings -> API Token`.
69
+
70
+ With this API token, you can configure your client to run models on the cloud
71
+ hosted devices.
72
+ ```bash
73
+ qai-hub configure --api_token API_TOKEN
74
+ ```
75
+ Navigate to [docs](https://app.aihub.qualcomm.com/docs/) for more information.
76
+
77
+
78
+
79
+ ## Demo off target
80
+
81
+ The package contains a simple end-to-end demo that downloads pre-trained
82
+ weights and runs this model on a sample input.
83
+
84
+ ```bash
85
+ python -m qai_hub_models.models.vit_quantized.demo
86
+ ```
87
+
88
+ The above demo runs a reference implementation of pre-processing, model
89
+ inference, and post processing.
90
+
91
+ **NOTE**: If you want running in a Jupyter Notebook or Google Colab like
92
+ environment, please add the following to your cell (instead of the above).
93
+ ```
94
+ %run -m qai_hub_models.models.vit_quantized.demo
95
+ ```
96
+
97
+
98
+ ### Run model on a cloud-hosted device
99
+
100
+ In addition to the demo, you can also run the model on a cloud-hosted Qualcomm®
101
+ device. This script does the following:
102
+ * Performance check on-device on a cloud-hosted device
103
+ * Downloads compiled assets that can be deployed on-device for Android.
104
+ * Accuracy check between PyTorch and on-device outputs.
105
+
106
+ ```bash
107
+ python -m qai_hub_models.models.vit_quantized.export
108
+ ```
109
+ ```
110
+ Profiling Results
111
+ ------------------------------------------------------------
112
+ VITQuantized
113
+ Device : Samsung Galaxy S23 (13)
114
+ Runtime : QNN
115
+ Estimated inference time (ms) : 5.5
116
+ Estimated peak memory usage (MB): [0, 30]
117
+ Total # Ops : 903
118
+ Compute Unit(s) : NPU (903 ops)
119
+ ```
120
+
121
+
122
+ ## How does this work?
123
+
124
+ This [export script](https://aihub.qualcomm.com/models/vit_quantized/qai_hub_models/models/VITQuantized/export.py)
125
+ leverages [Qualcomm® AI Hub](https://aihub.qualcomm.com/) to optimize, validate, and deploy this model
126
+ on-device. Lets go through each step below in detail:
127
+
128
+ Step 1: **Compile model for on-device deployment**
129
+
130
+ To compile a PyTorch model for on-device deployment, we first trace the model
131
+ in memory using the `jit.trace` and then call the `submit_compile_job` API.
132
+
133
+ ```python
134
+ import torch
135
+
136
+ import qai_hub as hub
137
+ from qai_hub_models.models.vit_quantized import
138
+
139
+ # Load the model
140
+
141
+ # Device
142
+ device = hub.Device("Samsung Galaxy S23")
143
+
144
+
145
+ ```
146
+
147
+
148
+ Step 2: **Performance profiling on cloud-hosted device**
149
+
150
+ After compiling models from step 1. Models can be profiled model on-device using the
151
+ `target_model`. Note that this scripts runs the model on a device automatically
152
+ provisioned in the cloud. Once the job is submitted, you can navigate to a
153
+ provided job URL to view a variety of on-device performance metrics.
154
+ ```python
155
+ profile_job = hub.submit_profile_job(
156
+ model=target_model,
157
+ device=device,
158
+ )
159
+
160
+ ```
161
+
162
+ Step 3: **Verify on-device accuracy**
163
+
164
+ To verify the accuracy of the model on-device, you can run on-device inference
165
+ on sample input data on the same cloud hosted device.
166
+ ```python
167
+ input_data = torch_model.sample_inputs()
168
+ inference_job = hub.submit_inference_job(
169
+ model=target_model,
170
+ device=device,
171
+ inputs=input_data,
172
+ )
173
+ on_device_output = inference_job.download_output_data()
174
+
175
+ ```
176
+ With the output of the model, you can compute like PSNR, relative errors or
177
+ spot check the output with expected output.
178
+
179
+ **Note**: This on-device profiling and inference requires access to Qualcomm®
180
+ AI Hub. [Sign up for access](https://myaccount.qualcomm.com/signup).
181
+
182
+
183
+
184
+ ## Run demo on a cloud-hosted device
185
+
186
+ You can also run the demo on-device.
187
+
188
+ ```bash
189
+ python -m qai_hub_models.models.vit_quantized.demo --on-device
190
+ ```
191
+
192
+ **NOTE**: If you want running in a Jupyter Notebook or Google Colab like
193
+ environment, please add the following to your cell (instead of the above).
194
+ ```
195
+ %run -m qai_hub_models.models.vit_quantized.demo -- --on-device
196
+ ```
197
+
198
+
199
+ ## Deploying compiled model to Android
200
+
201
+
202
+ The models can be deployed using multiple runtimes:
203
+ - TensorFlow Lite (`.tflite` export): [This
204
+ tutorial](https://www.tensorflow.org/lite/android/quickstart) provides a
205
+ guide to deploy the .tflite model in an Android application.
206
+
207
+
208
+ - QNN (`.so` export ): This [sample
209
+ app](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/sample_app.html)
210
+ provides instructions on how to use the `.so` shared library in an Android application.
211
+
212
+
213
+ ## View on Qualcomm® AI Hub
214
+ Get more details on VITQuantized's performance across various devices [here](https://aihub.qualcomm.com/models/vit_quantized).
215
+ Explore all available models on [Qualcomm® AI Hub](https://aihub.qualcomm.com/)
216
+
217
+
218
+ ## License
219
+ * The license for the original implementation of VITQuantized can be found [here](https://github.com/pytorch/vision/blob/main/LICENSE).
220
+ * The license for the compiled assets for on-device deployment can be found [here](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/Qualcomm+AI+Hub+Proprietary+License.pdf)
221
+
222
+
223
+
224
+ ## References
225
+ * [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://arxiv.org/abs/2010.11929)
226
+ * [Source Model Implementation](https://github.com/pytorch/vision/blob/main/torchvision/models/vision_transformer.py)
227
+
228
+
229
+
230
+ ## Community
231
+ * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
232
+ * For questions or feedback please [reach out to us](mailto:[email protected]).
233
+
234
+