Hugging Face's logo

STMicroelectronics
/
resnetv1

Image Classification
Model card Files Community
FBAGSTM commited on
Commit
26a5002
·
verified ·
1 Parent(s): ac927c2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +167 -6
README.md CHANGED
@@ -1,6 +1,167 @@
1
- ---
2
- license: other
3
- license_name: sla0044
4
- license_link: >-
5
- https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/LICENSE.md
6
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: sla0044
4
+ license_link: >-
5
+ https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/LICENSE.md
6
+ pipeline_tag: image-classification
7
+ ---
8
+ # ResNet v1
9
+
10
+ ## **Use case** : `Image classification`
11
+
12
+ # Model description
13
+
14
+ ResNet models perform image classification - they take images as input and classify the major object in the image into a
15
+ set of pre-defined classes. ResNet models provide very high accuracies with affordable model sizes. They are ideal for cases when high accuracy of classification is required.
16
+ ResNet models consist of residual blocks and came up to counter the effect of deteriorating accuracies with more layers due to network not learning the initial layers.
17
+ ResNet v1 uses post-activation for the residual blocks. The models below have 8 and 32 layers with ResNet v1 architecture.
18
+ (source: https://keras.io/api/applications/resnet/)
19
+ The model is quantized in int8 using tensorflow lite converter.
20
+
21
+ In addition, we introduce a new model family inspired from ResNet v1 which takes benefit from hybrid quantization.
22
+ Later on, they are named as ST ResNet 8 Hybrid v1 and ST ResNet 8 Hybrid v2.
23
+ By hybrid quantization, we mean that whenever it is possible, some network layers are quantized for weights and/or activations on less than 8 bits.
24
+ We used Larq library to define and train these models. In particular, in our topology some layers/activations are kept in 8 bits while others are in binary.
25
+ Please note that since this quantization is performed during training (Quantization Aware Training), these networks no longer need to be converted with tensorflow lite.
26
+ STM32Cube.AI is able to import them directly in .h5 format and to generate the corresponding optimized FW code.
27
+ Even if many layers are in binary, these models provide comparable accuracy to the full 8-bit ResNet v1 8 but have a significantly lower inference time.
28
+
29
+
30
+ ## Network information
31
+
32
+ | Network Information | Value |
33
+ |-------------------------|-------------------------------------------------------------------------|
34
+ | Framework | TensorFlow Lite |
35
+ | Quantization | int8 |
36
+ | Provenance | https://www.tensorflow.org/api_docs/python/tf/keras/applications/resnet |
37
+ | Paper | https://arxiv.org/abs/1512.03385 |
38
+
39
+ The models are quantized using tensorflow lite converter.
40
+
41
+ ## Network inputs / outputs
42
+
43
+ For an image resolution of NxM and P classes
44
+
45
+ | Input Shape | Description |
46
+ |----------------|-------------------------------------------------------------|
47
+ | (1, N, M, 3) | Single NxM RGB image with UINT8 values between 0 and 255 |
48
+
49
+ | Output Shape | Description |
50
+ |----------------|-------------------------------------------------------------|
51
+ | (1, P) | Per-class confidence for P classes in FLOAT32 |
52
+
53
+ ## Recommended Platforms
54
+
55
+ | Platform | Supported | Optimized |
56
+ |----------|-----------|-----------|
57
+ | STM32L0 | [] | [] |
58
+ | STM32L4 | [x] | [] |
59
+ | STM32U5 | [x] | [] |
60
+ | STM32H7 | [x] | [x] |
61
+ | STM32MP1 | [x] | [x]* |
62
+ | STM32MP2 | [x] | [] |
63
+ | STM32N6 | [x] | [] |
64
+
65
+ * Only for Cifar 100 models
66
+
67
+ # Performances
68
+
69
+ ## Metrics
70
+
71
+ Measures are done with default STM32Cube.AI configuration with enabled input / output allocated option.
72
+
73
+ ### Reference **MCU** memory footprint based on Cifar 10 dataset (see Accuracy for details on dataset)
74
+
75
+ | Model | Format | Resolution | Series | Activation RAM | Runtime RAM | Weights Flash | Code Flash | Total RAM | Total Flash | STM32Cube.AI version |
76
+ |---------------------------------------------------------------------------------------------------------------------------------------|--------|-------------|---------|----------------|-------------|---------------|------------|-----------|-------------|-----------------------|
77
+ | [ResNet v1 8 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/resnet_v1_8_32_tfs/resnet_v1_8_32_tfs_int8.tflite) | Int8 | 32x32x3 | STM32H7 | 62.51 KiB | 7.21 KiB | 76.9 KiB | 56.45 KiB | 69.72 KiB | 133.35 KiB | 10.0.0 |
78
+ | [ST ResNet 8 Hybrid v1 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/st_resnet_8_hybrid_v1_32_tfs/st_resnet_8_hybrid_v1_32_tfs.h5) | Hybrid | 32x32x3 | STM32H7 | 77.84 KiB | 18.38 KiB | 85.79 KiB | 61.75 KiB | 96.22 KiB | 147.54 KiB | 10.0.0 |
79
+ | [ST ResNet 8 Hybrid v2 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/st_resnet_8_hybrid_v2_32_tfs/st_resnet_8_hybrid_v2_32_tfs.h5) | Hybrid | 32x32x3 | STM32H7 | 78.99 KiB | 18.38 KiB | 66.28 KiB | 60.99 KiB | 97.37 KiB | 127.27 KiB | 10.0.0 |
80
+
81
+ ### Reference **MCU** inference time based on Cifar 10 dataset (see Accuracy for details on dataset)
82
+
83
+ | Model | Format | Resolution | Board | Execution Engine | Frequency | Inference time (ms) | STM32Cube.AI version |
84
+ |----------------------------------|--------|-------------|------------------|------------------|--------------|---------------------|-----------------------|
85
+ | [ResNet v1 8 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/resnet_v1_8_32_tfs/resnet_v1_8_32_tfs_int8.tflite) | Int8 | 32x32x3 | STM32H747I-DISCO | 1 CPU | 400 MHz | 28.67 ms | 10.0.0 |
86
+ | [ST ResNet 8 Hybrid v1 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/st_resnet_8_hybrid_v1_32_tfs/st_resnet_8_hybrid_v1_32_tfs.h5) | Hybrid | 32x32x3 | STM32H747I-DISCO | 1 CPU | 400 MHz | 28.93 ms | 10.0.0 |
87
+ | [ST ResNet 8 Hybrid v2 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/st_resnet_8_hybrid_v2_32_tfs/st_resnet_8_hybrid_v2_32_tfs.h5) | Hybrid | 32x32x3 | STM32H747I-DISCO | 1 CPU | 400 MHz | 25.2 ms | 10.0.0 |
88
+
89
+
90
+ ### Reference **MPU** inference time based on Flowers dataset (see Accuracy for details on dataset)
91
+ | Model | Format | Resolution | Quantization | Board | Execution Engine | Frequency | Inference time (ms) | %NPU | %GPU | %CPU | X-LINUX-AI version | Framework |
92
+ |---------------------------------------------------------------------------------------------------------------------------------------|----------|------------|---------------|-------------------|------------------|-----------|---------------------|-------|-------|------|--------------------|-----------------------|
93
+ | [ResNet v1 8 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/resnet_v1_8_32_tfs/resnet_v1_8_32_tfs_int8.tflite) | Int8 | 32x32x3 | per-channel** | STM32MP257F-DK2 | NPU/GPU | 800 MHz | 2.02 ms | 12.26 | 87.74 | 0 | v5.1.0 | OpenVX |
94
+ | [ST ResNet 8 Hybrid v1 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/st_resnet_8_hybrid_v1_32_tfs/st_resnet_8_hybrid_v1_32_tfs.h5) | Hybrid | 32x32x3 | per-channel** | STM32MP257F-DK2 | NPU/GPU | 800 MHz | TBD ms | 0 | 0 | 0 | v5.1.0 | OpenVX |
95
+ | [ST ResNet 8 Hybrid v2 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/st_resnet_8_hybrid_v2_32_tfs/st_resnet_8_hybrid_v2_32_tfs.h5) | Hybrid | 32x32x3 | per-channel** | STM32MP257F-DK2 | NPU/GPU | 800 MHz | TBD ms | 0 | 0 | 0 | v5.1.0 | OpenVX |
96
+ | [ResNet v1 8 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/resnet_v1_8_32_tfs/resnet_v1_8_32_tfs_int8.tflite) | Int8 | 32x32x3 | per-channel | STM32MP157F-DK2 | 2 CPU | 800 MHz | 6.50 ms | NA | NA | 100 | v5.1.0 | TensorFlowLite 2.11.0 |
97
+ | [ST ResNet 8 Hybrid v1 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/st_resnet_8_hybrid_v1_32_tfs/st_resnet_8_hybrid_v1_32_tfs.h5) | Hybrid | 32x32x3 | per-channel | STM32MP157F-DK2 | 2 CPU | 800 MHz | TBD ms | NA | NA | 100 | v5.1.0 | TensorFlowLite 2.11.0 |
98
+ | [ST ResNet 8 Hybrid v2 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/st_resnet_8_hybrid_v2_32_tfs/st_resnet_8_hybrid_v2_32_tfs.h5) | Hybrid | 32x32x3 | per-channel | STM32MP157F-DK2 | 2 CPU | 800 MHz | TBD ms | NA | NA | 100 | v5.1.0 | TensorFlowLite 2.11.0 |
99
+ | [ResNet v1 8 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/resnet_v1_8_32_tfs/resnet_v1_8_32_tfs_int8.tflite) | Int8 | 32x32x3 | per-channel | STM32MP135F-DK2 | 1 CPU | 1000 MHz | 10.77 ms | NA | NA | 100 | v5.1.0 | TensorFlowLite 2.11.0 |
100
+ | [ST ResNet 8 Hybrid v1 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/st_resnet_8_hybrid_v1_32_tfs/st_resnet_8_hybrid_v1_32_tfs.h5) | Hybrid | 32x32x3 | per-channel | STM32MP135F-DK2 | 1 CPU | 1000 MHz | TBD ms | NA | NA | 100 | v5.1.0 | TensorFlowLite 2.11.0 |
101
+ | [ST ResNet 8 Hybrid v2 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/st_resnet_8_hybrid_v2_32_tfs/st_resnet_8_hybrid_v2_32_tfs.h5) | Hybrid | 32x32x3 | per-channel | STM32MP135F-DK2 | 1 CPU | 1000 MHz | TBD ms | NA | NA | 100 | v5.1.0 | TensorFlowLite 2.11.0 |
102
+
103
+ ** **To get the most out of MP25 NPU hardware acceleration, please use per-tensor quantization**
104
+
105
+ ### Reference **MCU** memory footprint based on Cifar 100 dataset (see Accuracy for details on dataset)
106
+
107
+ | Model | Format | Resolution | Series | Activation RAM | Runtime RAM | Weights Flash | Code Flash | Total RAM | Total Flash |
108
+ |----------------------------------------------------------------------------------------------------------------------|--------|-------------|---------|----------------|-------------|---------------|------------|-------------|-------------|
109
+ | [ResNet v1 32 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar100/resnet_v1_32_32_tfs/resnet_v1_32_32_tfs_int8.tflite) | Int8 | 32x32x3 | STM32H7 | 45.41 KiB | 24.98 KiB | 464.38 KiB | 78.65 KiB | 70.39 KiB | 543.03 KiB |
110
+
111
+
112
+ ### Reference **MCU** inference time based on Cifar 100 dataset (see Accuracy for details on dataset)
113
+
114
+ | Model | Format | Resolution | Board | Execution Engine | Frequency | Inference time (ms) |
115
+ |----------------------------------------------------------------------------------------------------------------------|--------|------------|------------------|------------------|--------------|---------------------|
116
+ | [ResNet v1 32 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar100/resnet_v1_32_32_tfs/resnet_v1_32_32_tfs_int8.tflite) | Int8 | 32x32x3 | STM32H747I-DISCO | 1 CPU | 400 MHz | 177.7 ms |
117
+
118
+
119
+ ### Reference **MPU** inference time based on Flowers dataset (see Accuracy for details on dataset)
120
+ | Model | Format | Resolution | Quantization | Board | Execution Engine | Frequency | Inference time (ms) | %NPU | %GPU | %CPU | X-LINUX-AI version | Framework |
121
+ |---------------------------------------------------------------------------------------------------------------------|----------|------------|---------------|-------------------|------------------|-----------|---------------------|-------|-------|------|--------------------|-----------------------|
122
+ |[ResNet v1 32 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar100/resnet_v1_32_32_tfs/resnet_v1_32_32_tfs_int8.tflite) | Int8 | 32x32x3 | per-channel | STM32MP257F-DK2 | NPU/GPU | 800 MHz | 9.160 ms | 14.75 | 85.25 | 0 | v5.1.0 | OpenVX |
123
+ |[ResNet v1 32 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar100/resnet_v1_32_32_tfs/resnet_v1_32_32_tfs_int8.tflite) | Int8 | 32x32x3 | per-channel | STM32MP157F-DK2 | 2 CPU | 800 MHz | 34.78 ms | NA | NA | 100 | v5.1.0 | TensorFlowLite 2.11.0 |
124
+ |[ResNet v1 32 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar100/resnet_v1_32_32_tfs/resnet_v1_32_32_tfs_int8.tflite) | Int8 | 32x32x3 | per-channel | STM32MP135F-DK2 | 1 CPU | 1000 MHz | 55.32 ms | NA | NA | 100 | v5.1.0 | TensorFlowLite 2.11.0 |
125
+
126
+
127
+ ### Accuracy with Cifar10 dataset
128
+
129
+ Dataset details: [link](https://www.cs.toronto.edu/~kriz/cifar.html) ,
130
+ License [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) , Quotation[[1]](#1) , Number of classes: 10, Number of
131
+ images: 60 000
132
+
133
+ | Model | Format | Resolution | Top 1 Accuracy |
134
+ |------------------------------------------------------------------------------------------------------------------|----------|-------------|----------------|
135
+ | [ResNet v1 8 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/resnet_v1_8_32_tfs/resnet_v1_8_32_tfs.h5) | Float | 32x32x3 | 87.01 % |
136
+ | [ResNet v1 8 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/resnet_v1_8_32_tfs/resnet_v1_8_32_tfs_int8.tflite) | Int8 | 32x32x3 | 85.59 % |
137
+ | [ST ResNet 8 Hybrid v1 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/st_resnet_8_hybrid_v1_32_tfs/st_resnet_8_hybrid_v1_32_tfs.h5) | Hybrid | 32x32x3 | 86 % |
138
+ | [ST ResNet 8 Hybrid v2 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar10/st_resnet_8_hybrid_v2_32_tfs/st_resnet_8_hybrid_v2_32_tfs.h5) | Hybrid | 32x32x3 | 84.85 % |
139
+
140
+
141
+ ### Accuracy with Cifar100 dataset
142
+
143
+ Dataset details: [link](https://www.cs.toronto.edu/~kriz/cifar.html) ,
144
+ License [CC0 4.0](https://creativecommons.org/licenses/by/4.0/), Quotation[[2]](#2) , Number of classes:100,
145
+ Number of images: 600 000
146
+
147
+ | Model | Format | Resolution | Top 1 Accuracy |
148
+ |----------------------------------------------------------------------------------------------------------------------|---------|------------|----------------|
149
+ | [ResNet v1 32 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar100/resnet_v1_32_32_tfs/resnet_v1_32_32_tfs.h5) | Float | 32x32x3 | 67.75 % |
150
+ | [ResNet v1 32 tfs](https://github.com/STMicroelectronics/stm32ai-modelzoo/image_classification/resnetv1/ST_pretrainedmodel_public_dataset/cifar100/resnet_v1_32_32_tfs/resnet_v1_32_32_tfs_int8.tflite) | Int8 | 32x32x3 | 66.58 % |
151
+
152
+ ## Retraining and Integration in a simple example:
153
+
154
+ Please refer to the stm32ai-modelzoo-services GitHub [here](https://github.com/STMicroelectronics/stm32ai-modelzoo-services)
155
+
156
+
157
+ # References
158
+
159
+ <a id="1">[1]</a>
160
+ "Tf_flowers : tensorflow datasets," TensorFlow. [Online]. Available: https://www.tensorflow.org/datasets/catalog/tf_flowers.
161
+
162
+ <a id="2">[2]</a>
163
+ J, ARUN PANDIAN; GOPAL, GEETHARAMANI (2019), "Data for: Identification of Plant Leaf Diseases Using a 9-layer Deep Convolutional Neural Network", Mendeley Data, V1, doi: 10.17632/tywbtsjrjv.1
164
+
165
+ <a id="3">[3]</a>
166
+ L. Bossard, M. Guillaumin, and L. Van Gool, "Food-101 -- Mining Discriminative Components with Random Forests." European Conference on Computer Vision, 2014.
167
+