Update Readme ST Model Zoo
Browse files
README.md
CHANGED
@@ -1,10 +1,3 @@
|
|
1 |
-
---
|
2 |
-
license: other
|
3 |
-
license_name: sla0044
|
4 |
-
license_link: >-
|
5 |
-
https://github.com/STMicroelectronics/stm32aimodelzoo/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/LICENSE.md
|
6 |
-
pipeline_tag: image-segmentation
|
7 |
-
---
|
8 |
# DeepLab v3
|
9 |
|
10 |
## **Use case** : `Semantic Segmentation`
|
@@ -70,9 +63,9 @@ Measures are done with default STM32Cube.AI configuration with enabled input / o
|
|
70 |
|
71 |
| Model | Dataset | Format | Resolution | Series | Internal RAM (KiB) | External RAM (KiB) | Weights Flash (KiB) | STM32Cube.AI version | STEdgeAI Core version |
|
72 |
|------------|---------------|----------|------------|-----------|--------------|--------------|---------------|----------------------|-----------------------|
|
73 |
-
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_256/deeplab_v3_mobilenetv2_05_16_256_asppv2_qdq_int8.onnx) | person COCO 2017 + PASCAL VOC 2012 | Int8 | 256x256x3 | STM32N6 |
|
74 |
-
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_320/deeplab_v3_mobilenetv2_05_16_320_asppv2_qdq_int8.onnx) |person COCO 2017 + PASCAL VOC 2012 | Int8 | 320x320x3 | STM32N6 |
|
75 |
-
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_416/deeplab_v3_mobilenetv2_05_16_416_asppv2_qdq_int8.onnx) | person COCO 2017 + PASCAL VOC 2012
|
76 |
|
77 |
|
78 |
|
@@ -81,18 +74,18 @@ Measures are done with default STM32Cube.AI configuration with enabled input / o
|
|
81 |
|
82 |
| Model | Dataset | Format | Resolution | Board | Execution Engine | Inference time (ms) | Inf / sec | STM32Cube.AI version | STEdgeAI Core version |
|
83 |
|------------|---------------|----------|------------|------------------|------------------|---------------------|-------------|----------------------|-------------------------|
|
84 |
-
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_256/deeplab_v3_mobilenetv2_05_16_256_asppv2_qdq_int8.onnx) | person COCO 2017 + PASCAL VOC 2012 | Int8 | 256x256x3 | STM32N6570-DK | NPU/MCU |
|
85 |
-
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_320/deeplab_v3_mobilenetv2_05_16_320_asppv2_qdq_int8.onnx) | person COCO 2017 + PASCAL VOC 2012 | Int8 | 320x320x3 | STM32N6570-DK | NPU/MCU |
|
86 |
-
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_416/deeplab_v3_mobilenetv2_05_16_416_asppv2_qdq_int8.onnx) | person COCO 2017 + PASCAL VOC 2012 | Int8 | 416x416x3 | STM32N6570-DK | NPU/MCU |
|
87 |
|
88 |
|
89 |
### Reference **MPU** inference time based on COCO 2017 + PASCAL VOC 2012 segmentation dataset 21 classes and a derivative person dataset from it (see Accuracy for details on dataset)
|
90 |
| Model | Dataset | Format | Resolution | Quantization | Board| Execution Engine | Frequency | Inference time (ms) | %NPU | %GPU | %CPU | X-LINUX-AI version |Framework |
|
91 |
|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------|--------|------------|----------------|-------------------|------------------|-----------|---------------------|-------|--------|------|--------------------|-----------------------|
|
92 |
-
| [DeepLabV3 per tensor (no ASPP)](https://www.st.com/en/embedded-software/x-linux-ai.html) | COCO 2017 + PASCAL VOC 2012 | Int8 | 257x257x3 | per-tensor | STM32MP257F-DK2 | NPU/GPU | 1500 MHz | 52.75 | 99.2 | 0.80 | 0 |
|
93 |
-
| [DeepLabV3 MobileNetv2 ASPPv1 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512/deeplab_v3_mobilenetv2_05_16_512_asppv1_int8.tflite) | COCO 2017 + PASCAL VOC 2012 | Int8 (tflite) | 512x512x3 | per-channel ** | STM32MP257F-DK2 | NPU/GPU | 1500 MHz |
|
94 |
-
| [DeepLabV3 MobileNetv2 ASPPv1 mixed precision](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512/deeplab_v3_mobilenetv2_05_16_512_asppv1_int8_f32.tflite) | COCO 2017 + PASCAL VOC 2012 | Int8 & float32 (tflite) | 512x512x3 | per-channel ** | STM32MP257F-DK2 | NPU/GPU | 1500 MHz |
|
95 |
-
| [DeepLabV3 MobileNetv2 ASPPv1 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512/deeplab_v3_mobilenetv2_05_16_512_asppv1_qdq_int8.onnx) | COCO 2017 + PASCAL VOC 2012 | Int8 (onnx) | 512x512x3 | per-channel ** | STM32MP257F-DK2 | NPU/GPU | 1500 MHz | 729.62 | 3.0 | 97.0 | 0 |
|
96 |
|
97 |
- **DeepLabV3 per tensor**:
|
98 |
This model, which does not include ASPP (Atrous Spatial Pyramid Pooling), was downloaded from the TensorFlow DeepLabV3 page on [Kaggle](https://www.kaggle.com/models/tensorflow/deeplabv3/).
|
@@ -111,19 +104,19 @@ Measures are done with default STM32Cube.AI configuration with enabled input / o
|
|
111 |
|
112 |
** **To get the most out of MP25 NPU hardware acceleration, please use per-tensor quantization**
|
113 |
|
114 |
-
### Accuracy with COCO 2017 + PASCAL VOC 2012
|
115 |
|
116 |
**Pascal VOC Dataset Details:**
|
117 |
|
118 |
-
- **Link:** [VOC 2012 Dataset](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/)
|
119 |
- **License:** [Database Contents License (DbCL) v1.0](https://opendatacommons.org/licenses/dbcl/1-0/)
|
120 |
- **Number of Classes:** 21
|
121 |
-
- **Contents:**
|
122 |
- 1464 training images and masks
|
123 |
- 1449 validation images and masks
|
124 |
|
125 |
|
126 |
-
**Please follow the [PASCAL VOC 2012 tutorial](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/
|
127 |
|
128 |
|
129 |
**COCO Dataset Details:**
|
@@ -134,7 +127,7 @@ Measures are done with default STM32Cube.AI configuration with enabled input / o
|
|
134 |
Please note, that the following accuracies are obtained after training the model with the augmented Pascal VOC + COCO data and evaluated on Pascal VOC 2012 validation set (val.txt), and with a preprocessing resize with interpolation method 'bilinear'.
|
135 |
Moreover, IoU are averaged on all classes including background.
|
136 |
|
137 |
-
**Please use the [COCO 2017 PASCAL VOC 2012 tutorial](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/
|
138 |
|
139 |
| Model Description | Resolution | Format | Accuracy | Averaed IoU |
|
140 |
|--------------------------------------------------------------------------------------------------------------------------------------------------------------|------------|------------|----------|--------------|
|
@@ -145,9 +138,9 @@ Moreover, IoU are averaged on all classes including background.
|
|
145 |
| [DeepLabv3 MobileNetv2 ASPPv1 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512/deeplab_v3_mobilenetv2_05_16_512_asppv1_qdq_int8.onnx) | 512x512x3 | Int8 (onnx) | 93.15%| 72.39% |
|
146 |
|
147 |
|
148 |
-
### Accuracy with Person COCO 2017 + PASCAL VOC 2012
|
149 |
|
150 |
-
**Please use the [Person COCO 2017 PASCAL VOC 2012 tutorial](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/
|
151 |
|
152 |
| Models Description | Resolution | Format | Accuracy (%) | average IoU |
|
153 |
|--------------------------------------------|-----------|---------------|--------------|-------------|
|
@@ -159,5 +152,4 @@ Moreover, IoU are averaged on all classes including background.
|
|
159 |
| [DeepLabv3 MobileNetv2 ASPPv2 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_416/deeplab_v3_mobilenetv2_05_16_416_asppv2_qdq_int8.onnx) | 416x416x3 | ONNX | 95.44 % | 80.36 % |
|
160 |
|
161 |
|
162 |
-
Please refer to the stm32ai-modelzoo-services GitHub [here](https://github.com/STMicroelectronics/stm32ai-modelzoo-services)
|
163 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
# DeepLab v3
|
2 |
|
3 |
## **Use case** : `Semantic Segmentation`
|
|
|
63 |
|
64 |
| Model | Dataset | Format | Resolution | Series | Internal RAM (KiB) | External RAM (KiB) | Weights Flash (KiB) | STM32Cube.AI version | STEdgeAI Core version |
|
65 |
|------------|---------------|----------|------------|-----------|--------------|--------------|---------------|----------------------|-----------------------|
|
66 |
+
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_256/deeplab_v3_mobilenetv2_05_16_256_asppv2_qdq_int8.onnx) | person COCO 2017 + PASCAL VOC 2012 | Int8 | 256x256x3 | STM32N6 | 2071.25 | 0.0 | 960.58 | 10.2.0 | 2.2.0 |
|
67 |
+
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_320/deeplab_v3_mobilenetv2_05_16_320_asppv2_qdq_int8.onnx) |person COCO 2017 + PASCAL VOC 2012 | Int8 | 320x320x3 | STM32N6 | 2583.5 | 0.0 | 959.74 | 10.2.0 | 2.2.0 |
|
68 |
+
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_416/deeplab_v3_mobilenetv2_05_16_416_asppv2_qdq_int8.onnx) | person COCO 2017 + PASCAL VOC 2012| Int8 | 416x416x3 | STM32N6 | 2727.12 | 2028.0 | 960.58 | 10.2.0 | 2.2.0 |
|
69 |
|
70 |
|
71 |
|
|
|
74 |
|
75 |
| Model | Dataset | Format | Resolution | Board | Execution Engine | Inference time (ms) | Inf / sec | STM32Cube.AI version | STEdgeAI Core version |
|
76 |
|------------|---------------|----------|------------|------------------|------------------|---------------------|-------------|----------------------|-------------------------|
|
77 |
+
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_256/deeplab_v3_mobilenetv2_05_16_256_asppv2_qdq_int8.onnx) | person COCO 2017 + PASCAL VOC 2012 | Int8 | 256x256x3 | STM32N6570-DK | NPU/MCU | 29.63 | 33.74 | 10.2.0 | 2.2.0 |
|
78 |
+
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_320/deeplab_v3_mobilenetv2_05_16_320_asppv2_qdq_int8.onnx) | person COCO 2017 + PASCAL VOC 2012 | Int8 | 320x320x3 | STM32N6570-DK | NPU/MCU | 45.34 | 22.05 | 10.2.0 | 2.2.0 |
|
79 |
+
| [DeepLabv3 MobileNetv2 ASPPv2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_416/deeplab_v3_mobilenetv2_05_16_416_asppv2_qdq_int8.onnx) | person COCO 2017 + PASCAL VOC 2012 | Int8 | 416x416x3 | STM32N6570-DK | NPU/MCU | 165.35 | 6.04 | 10.2.0 | 2.2.0 |
|
80 |
|
81 |
|
82 |
### Reference **MPU** inference time based on COCO 2017 + PASCAL VOC 2012 segmentation dataset 21 classes and a derivative person dataset from it (see Accuracy for details on dataset)
|
83 |
| Model | Dataset | Format | Resolution | Quantization | Board| Execution Engine | Frequency | Inference time (ms) | %NPU | %GPU | %CPU | X-LINUX-AI version |Framework |
|
84 |
|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------|--------|------------|----------------|-------------------|------------------|-----------|---------------------|-------|--------|------|--------------------|-----------------------|
|
85 |
+
| [DeepLabV3 per tensor (no ASPP)](https://www.st.com/en/embedded-software/x-linux-ai.html) | COCO 2017 + PASCAL VOC 2012 | Int8 | 257x257x3 | per-tensor | STM32MP257F-DK2 | NPU/GPU | 1500 MHz | 52.75 | 99.2 | 0.80 | 0 | v6.1.0 | OpenVX | | | | | v6.1.0
|
86 |
+
| [DeepLabV3 MobileNetv2 ASPPv1 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512/deeplab_v3_mobilenetv2_05_16_512_asppv1_int8.tflite) | COCO 2017 + PASCAL VOC 2012 | Int8 (tflite) | 512x512x3 | per-channel ** | STM32MP257F-DK2 | NPU/GPU | 1500 MHz | 830.50 | 7.38| 92.62 | 0 | v6.1.0 | OpenVX |
|
87 |
+
| [DeepLabV3 MobileNetv2 ASPPv1 mixed precision](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512/deeplab_v3_mobilenetv2_05_16_512_asppv1_int8_f32.tflite) | COCO 2017 + PASCAL VOC 2012 | Int8 & float32 (tflite) | 512x512x3 | per-channel ** | STM32MP257F-DK2 | NPU/GPU | 1500 MHz | 939.8 | 6.29 | 93.71 | 0 | v6.1.0 | OpenVX |
|
88 |
+
| [DeepLabV3 MobileNetv2 ASPPv1 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512/deeplab_v3_mobilenetv2_05_16_512_asppv1_qdq_int8.onnx) | COCO 2017 + PASCAL VOC 2012 | Int8 (onnx) | 512x512x3 | per-channel ** | STM32MP257F-DK2 | NPU/GPU | 1500 MHz | 729.62 | 3.0 | 97.0 | 0 | v6.1.0| OpenVX |
|
89 |
|
90 |
- **DeepLabV3 per tensor**:
|
91 |
This model, which does not include ASPP (Atrous Spatial Pyramid Pooling), was downloaded from the TensorFlow DeepLabV3 page on [Kaggle](https://www.kaggle.com/models/tensorflow/deeplabv3/).
|
|
|
104 |
|
105 |
** **To get the most out of MP25 NPU hardware acceleration, please use per-tensor quantization**
|
106 |
|
107 |
+
### Accuracy with COCO 2017 + PASCAL VOC 2012
|
108 |
|
109 |
**Pascal VOC Dataset Details:**
|
110 |
|
111 |
+
- **Link:** [VOC 2012 Dataset](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/)
|
112 |
- **License:** [Database Contents License (DbCL) v1.0](https://opendatacommons.org/licenses/dbcl/1-0/)
|
113 |
- **Number of Classes:** 21
|
114 |
+
- **Contents:**
|
115 |
- 1464 training images and masks
|
116 |
- 1449 validation images and masks
|
117 |
|
118 |
|
119 |
+
**Please follow the [PASCAL VOC 2012 tutorial](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/datasets) to have more training masks (about 10,582) and a `trainaug.txt` file containing the IDs of the new training masks.**
|
120 |
|
121 |
|
122 |
**COCO Dataset Details:**
|
|
|
127 |
Please note, that the following accuracies are obtained after training the model with the augmented Pascal VOC + COCO data and evaluated on Pascal VOC 2012 validation set (val.txt), and with a preprocessing resize with interpolation method 'bilinear'.
|
128 |
Moreover, IoU are averaged on all classes including background.
|
129 |
|
130 |
+
**Please use the [COCO 2017 PASCAL VOC 2012 tutorial](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/datasets/coco_2017_pascal_voc_2012) to create COCO 2017 + PASCAL VOC 2012 dataset to do the needed filtering. Only images containing one or more classes from the 21 Pascal VOC dataset classes should be used. Additionally, the masks need to be converted to the Pascal VOC masks format.**
|
131 |
|
132 |
| Model Description | Resolution | Format | Accuracy | Averaed IoU |
|
133 |
|--------------------------------------------------------------------------------------------------------------------------------------------------------------|------------|------------|----------|--------------|
|
|
|
138 |
| [DeepLabv3 MobileNetv2 ASPPv1 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_512/deeplab_v3_mobilenetv2_05_16_512_asppv1_qdq_int8.onnx) | 512x512x3 | Int8 (onnx) | 93.15%| 72.39% |
|
139 |
|
140 |
|
141 |
+
### Accuracy with Person COCO 2017 + PASCAL VOC 2012
|
142 |
|
143 |
+
**Please use the [Person COCO 2017 PASCAL VOC 2012 tutorial](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/datasets/n_class_coco_2017_pascal_voc_2012) to create Pesron COCO 2017 + PASCAL VOC 2012 dataset.**
|
144 |
|
145 |
| Models Description | Resolution | Format | Accuracy (%) | average IoU |
|
146 |
|--------------------------------------------|-----------|---------------|--------------|-------------|
|
|
|
152 |
| [DeepLabv3 MobileNetv2 ASPPv2 per channel](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/semantic_segmentation/deeplab_v3/ST_pretrainedmodel_public_dataset/person_coco_2017_pascal_voc_2012/deeplab_v3_mobilenetv2_05_16_416/deeplab_v3_mobilenetv2_05_16_416_asppv2_qdq_int8.onnx) | 416x416x3 | ONNX | 95.44 % | 80.36 % |
|
153 |
|
154 |
|
155 |
+
Please refer to the stm32ai-modelzoo-services GitHub [here](https://github.com/STMicroelectronics/stm32ai-modelzoo-services)
|
|