henry000 commited on
Commit
4a690bb
Β·
1 Parent(s): 2144126

πŸ“ [Update] tutorial, split cfg, data, model, task

Browse files
docs/1_tutorials/0_allIn1.rst ADDED
@@ -0,0 +1,204 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ All In 1
2
+ ========
3
+
4
+ :file:`yolo.lazy` is a packaged file that includes :guilabel:`training`, :guilabel:`validation`, and :guilabel:`inference` tasks.
5
+ For detailed function documentation, thercheck out the IPython notebooks to learn how to import and use these function
6
+ the following section will break down operation inside of lazy, also supporting directly import/call the function.
7
+
8
+ [TOC], setup, build, dataset, train, validation, inference
9
+ To train the model, you can run:
10
+
11
+ Train Model
12
+ ----------
13
+
14
+
15
+ - batch size check / cuda
16
+ - training time / check
17
+ - build model / check
18
+ - dataset / check
19
+
20
+ .. code-block:: bash
21
+
22
+ python yolo/lazy.py task=train
23
+
24
+ You can customize the training process by overriding the following common arguments:
25
+
26
+ - ``name``: :guilabel:`str`
27
+ The experiment name.
28
+
29
+ - ``model``: :guilabel:`str`
30
+ Model backbone, options include [model_zoo] v9-c, v7, v9-e, etc.
31
+
32
+ - ``cpu_num``: :guilabel:`int`
33
+ Number of CPU workers (num_workers).
34
+
35
+ - ``out_path``: :guilabel:`Path`
36
+ The output path for saving models and logs.
37
+
38
+ - ``weight``: :guilabel:`Path | bool | None`
39
+ The path to pre-trained weights, False for training from scratch, None for default weights.
40
+
41
+ - ``use_wandb``: :guilabel:`bool`
42
+ Whether to use Weights and Biases for experiment tracking.
43
+
44
+ - ``use_TensorBoard``: :guilabel:`bool`
45
+ Whether to use TensorBoard for logging.
46
+
47
+ - ``image_size``: :guilabel:`int | [int, int]`
48
+ The input image size.
49
+
50
+ - ``+quiet``: :guilabel:`bool`
51
+ Optional, disable all output.
52
+
53
+ - ``task.epoch``: :guilabel:`int`
54
+ Total number of training epochs.
55
+
56
+ - ``task.data.batch_size``: :guilabel:`int`
57
+ The size of each batch (auto-batch sizing [WIP]).
58
+
59
+ Examples
60
+ ~~~~~~~~
61
+
62
+ To train a model with a specific batch size and image size, you can run:
63
+
64
+ .. code-block:: bash
65
+
66
+ python yolo/lazy.py task=train task.data.batch_size=12 image_size=1280
67
+
68
+ Multi-GPU Training with DDP
69
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
70
+
71
+ For multi-GPU training, we use Distributed Data Parallel (DDP) for efficient and scalable training.
72
+ DDP enable training model with mutliple GPU, even the GPUs aren't on the same machine. For more details, you can refer to the `DDP tutorial <https://pytorch.org/tutorials/intermediate/ddp_tutorial.html>`_.
73
+
74
+ To train on multiple GPUs, replace the ``python`` command with ``torchrun --nproc_per_node=[GPU_NUM]``. The ``nproc_per_node`` argument specifies the number of GPUs to use.
75
+
76
+
77
+ .. tabs::
78
+
79
+ .. tab:: bash
80
+ .. code-block:: bash
81
+
82
+ torchrun --nproc_per_node=2 yolo/lazy.py task=train device=[0,1]
83
+
84
+ .. tab:: zsh
85
+ .. code-block:: bash
86
+
87
+ torchrun --nproc_per_node=2 yolo/lazy.py task=train device=\[0,1\]
88
+
89
+
90
+ Training on a Custom Dataset
91
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
92
+
93
+ To use the auto-download module, we suggest users construct the dataset config in the following format.
94
+ If the config files include `auto_download`, the model will automatically download the dataset when creating the dataloader.
95
+
96
+ Here is an example dataset config file:
97
+
98
+ .. literalinclude:: ../../yolo/config/dataset/dev.yaml
99
+ :language: YAML
100
+
101
+ Both of the following formats are acceptable:
102
+
103
+ - ``path``: :guilabel:`str`
104
+ The path to the dataset.
105
+
106
+ - ``train, validation``: :guilabel:`str`
107
+ The training and validation directory names under `/images`. If using txt as ground truth, these should also be the names under `/labels/`.
108
+
109
+ - ``class_num``: :guilabel:`int`
110
+ The number of dataset classes.
111
+
112
+ - ``class_list``: :guilabel:`List[str]`
113
+ Optional, the list of class names, used only for visualizing the bounding box classes.
114
+
115
+ - ``auto_download``: :guilabel:`dict`
116
+ Optional, whether to auto-download the dataset.
117
+
118
+ The dataset should include labels or annotations, preferably in JSON format for compatibility with pycocotools during inference:
119
+
120
+ .. code-block:: text
121
+
122
+ DataSetName/
123
+ β”œβ”€β”€ annotations
124
+ β”‚ β”œβ”€β”€ train_json_name.json
125
+ β”‚ └── val_json_name.json
126
+ β”œβ”€β”€ labels/
127
+ β”‚ β”œβ”€β”€ train/
128
+ β”‚ β”‚ β”œβ”€β”€ AnyLabelName.txt
129
+ β”‚ β”‚ └── ...
130
+ β”‚ └── validation/
131
+ β”‚ └── ...
132
+ └── images/
133
+ β”œβ”€β”€ train/
134
+ β”‚ β”œβ”€β”€ AnyImageNameN.{png,jpg,jpeg}
135
+ β”‚ └── ...
136
+ └── validation/
137
+ └── ...
138
+
139
+
140
+ Validation Model
141
+ ----------------
142
+
143
+ During training, this block will be auto-executed. You may also run this task manually to generate a JSON file representing the predictions for a given validation dataset. If the validation set includes JSON annotations, it will run pycocotools for evaluation.
144
+
145
+ We recommend setting ``task.data.shuffle`` to False and turning off ``task.data.data_augment``.
146
+
147
+ You can customize the validation process by overriding the following arguments:
148
+
149
+ - ``task.nms.min_confidence``: :guilabel:`str`
150
+ The minimum confidence of model prediction.
151
+
152
+ - ``task.nms.min_iou``: :guilabel:`str`
153
+ The minimum IoU threshold for NMS (Non-Maximum Suppression).
154
+
155
+ Examples
156
+ ~~~~~~~~
157
+
158
+ .. tabs::
159
+
160
+ .. tab:: git-cloned
161
+ .. code-block:: bash
162
+
163
+ python yolo/lazy.py task=validation task.nms.min_iou=0.9
164
+
165
+ .. tab:: PyPI
166
+ .. code-block:: bash
167
+
168
+ yolo task=validation task.nms.min_iou=0.9
169
+
170
+
171
+ Model Inference
172
+ ---------------
173
+
174
+ .. note::
175
+ The ``dataset`` parameter shouldn't be overridden because the model requires the ``class_num`` of the dataset. If the classes have names, please provide the ``class_list``.
176
+
177
+ You can customize the inference process by overriding the following arguments:
178
+
179
+ - ``task.fast_inference``: :guilabel:`str`
180
+ Optional. Values can be `onnx`, `trt`, `deploy`, or `None`. `deploy` will detach the model auxiliary head.
181
+
182
+ - ``task.data.source``: :guilabel:`str | Path | int`
183
+ This argument will be auto-resolved and could be a webcam ID, image folder path, video/image path.
184
+
185
+ - ``task.nms.min_confidence``: :guilabel:`str`
186
+ The minimum confidence of model prediction.
187
+
188
+ - ``task.nms.min_iou``: :guilabel:`str`
189
+ The minimum IoU threshold for NMS (Non-Maximum Suppression).
190
+
191
+ Examples
192
+ ~~~~~~~~
193
+
194
+ .. tabs::
195
+
196
+ .. tab:: git-cloned
197
+ .. code-block:: bash
198
+
199
+ python yolo/lazy.py model=v9-m task.nms.min_confidence=0.1 task.data.source=0 task.fast_inference=onnx
200
+
201
+ .. tab:: PyPI
202
+ .. code-block:: bash
203
+
204
+ yolo model=v9-m task.nms.min_confidence=0.1 task.data.source=0 task.fast_inference=onnx
docs/1_tutorials/0_train.rst DELETED
@@ -1,10 +0,0 @@
1
- .. _Train:
2
-
3
- Train
4
- =====
5
-
6
- Train on COCO2017
7
- -----------------
8
-
9
- Train on Cusom Dataset
10
- -----------------
 
 
 
 
 
 
 
 
 
 
 
docs/1_tutorials/1_setup.rst ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Setup Config
2
+ ============
3
+
4
+ To set up your configuration, you will need to generate a configuration class based on :class:`~yolo.config.config.Config`, which can be achieved using `hydra <https://hydra.cc/>`_.
5
+ The configuration will include all the necessary settings for your ``task``, including general configuration, ``dataset`` information, and task-specific information (``train``, ``inference``, ``validation``).
6
+
7
+ Next, create the progress logger to handle the output and progress bar. This class is based on `rich <https://github.com/Textualize/rich>`_'s progress bar and customizes the logger (print function) using `loguru <https://loguru.readthedocs.io/>`_.
8
+
9
+ .. tabs::
10
+
11
+ .. tab:: decorator
12
+ .. code-block:: python
13
+
14
+ import hydra
15
+ from yolo import ProgressLogger
16
+ from yolo.config.config import Config
17
+
18
+ @hydra.main(config_path="config", config_name="config", version_base=None)
19
+ def main(cfg: Config):
20
+ progress = ProgressLogger(cfg, exp_name=cfg.name)
21
+ pass
22
+
23
+ .. tab:: initialize & compose
24
+ .. code-block:: python
25
+
26
+ from hydra import compose, initialize
27
+ from yolo import ProgressLogger
28
+ from yolo.config.config import Config
29
+
30
+ with initialize(config_path="config", version_base=None):
31
+ cfg = compose(config_name="config", overrides=["task=train", "model=v9-c"])
32
+
33
+ progress = ProgressLogger(cfg, exp_name=cfg.name)
34
+
35
+ TODO: add a config over view
docs/1_tutorials/1_validation.rst DELETED
@@ -1,9 +0,0 @@
1
- Validation
2
- ==========
3
-
4
-
5
- Validation on COCO2017
6
- ----------------------
7
-
8
- Validation on Custom Dataset
9
- ----------------------------
 
 
 
 
 
 
 
 
 
 
docs/1_tutorials/2_buildmodel.rst ADDED
@@ -0,0 +1,62 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Build Model
2
+ ===========
3
+
4
+ In YOLOv7, the prediction will be ``Anchor``, and in YOLOv9, it will predict ``Vector``. The converter will turn the bounding box to the vector.
5
+
6
+ The overall model flowchart is as follows:
7
+
8
+ .. mermaid::
9
+
10
+ flowchart LR
11
+ Input-->Model;
12
+ Model--Class-->NMS;
13
+ Model--Anc/Vec-->Converter;
14
+ Converter--Box-->NMS;
15
+ NMS-->Output;
16
+
17
+ Load Model
18
+ ~~~~~~~~~~
19
+
20
+ Using `create_model`, it will automatically create the :class:`~yolo.model.yolo.YOLO` model and load the provided weights.
21
+
22
+ Arguments:
23
+
24
+ - **model**: :class:`~yolo.config.config.ModelConfig`
25
+ The model configuration.
26
+ - **class_num**: :guilabel:`int`
27
+ The number of classes in the dataset, used for the YOLO's prediction head.
28
+ - **weight_path**: :guilabel:`Path | bool`
29
+ The path to the model weights.
30
+ - If `False`, weights are not loaded.
31
+ - If :guilabel:`True | None`, default weights are loaded.
32
+ - If a `Path`, the model weights are loaded from the specified path.
33
+
34
+ .. code-block:: python
35
+
36
+ model = create_model(cfg.model, class_num=cfg.dataset.class_num, weight_path=cfg.weight)
37
+ model = model.to(device)
38
+
39
+ Deploy Model
40
+ ~~~~~~~~~~~~
41
+
42
+ In the deployment version, we will remove the auxiliary branch of the model for fast inference. If the config includes ONNX and TensorRT, it will load/compile the model to ONNX or TensorRT format after removing the auxiliary branch.
43
+
44
+ .. code-block:: python
45
+
46
+ model = FastModelLoader(cfg).load_model(device)
47
+
48
+ Autoload Converter
49
+ ~~~~~~~~~~~~~~~~~~
50
+
51
+ Autoload the converter based on the model type (v7 or v9).
52
+
53
+ Arguments:
54
+
55
+ - **Model Name**: :guilabel:`str`
56
+ Used for choosing ``Vec2Box`` or ``Anc2Box``.
57
+ - **Anchor Config**: The anchor configuration, used to generate the anchor grid.
58
+ - **model**, **image_size**: Used for auto-detecting the anchor grid.
59
+
60
+ .. code-block:: python
61
+
62
+ converter = create_converter(cfg.model.name, model, cfg.model.anchor, cfg.image_size, device)
docs/1_tutorials/2_inference.rst DELETED
@@ -1,9 +0,0 @@
1
- Inference
2
- ==========
3
-
4
-
5
- Inference Video
6
- ---------------
7
-
8
- Inference Image
9
- ---------------
 
 
 
 
 
 
 
 
 
 
docs/1_tutorials/3_dataset.rst ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Create Dataset
2
+ ==============
3
+
4
+ In this section, we will prepare the dataset and create a dataloader.
5
+
6
+ Overall, the dataloader can be created by:
7
+
8
+ .. code-block:: python
9
+
10
+ from yolo import create_dataloader
11
+ dataloader = create_dataloader(cfg.task.data, cfg.dataset, cfg.task.task, use_ddp)
12
+
13
+ For inference, the dataset will be handled by :class:`~yolo.tools.data_loader.StreamDataLoader`, while for training and validation, it will be handled by :class:`~yolo.tools.data_loader.YoloDataLoader`.
14
+
15
+ The input arguments are:
16
+
17
+ - **DataConfig**: :class:`~yolo.config.config.DataConfig`, the relevant configuration for the dataloader.
18
+ - **DatasetConfig**: :class:`~yolo.config.config.DatasetConfig`, the relevant configuration for the dataset.
19
+ - **task_name**: :guilabel:`str`, the task name, which can be `inference`, `validation`, or `train`.
20
+ - **use_ddp**: :guilabel:`bool`, whether to use DDP (Distributed Data Parallel). Default is `False`.
21
+
22
+ Train and Validation
23
+ ----------------------------
24
+
25
+ Dataloader Return Type
26
+ ~~~~~~~~~~~~~~~~~~~~~
27
+
28
+ For each iteration, the return type includes:
29
+
30
+ - **batch_size**: the size of each batch, used to calculate batch average loss.
31
+ - **images**: the input images.
32
+ - **targets**: the ground truth of the images according to the task.
33
+
34
+ Auto Download Dataset
35
+ ~~~~~~~~~~~~~~~~~~~~~
36
+
37
+ The dataset will be auto-downloaded if the user provides the `auto_download` configuration. For example, if the configuration is as follows:
38
+
39
+
40
+ .. literalinclude:: ../../yolo/config/dataset/mock.yaml
41
+ :language: YAML
42
+
43
+
44
+ First, it will download and unzip the dataset from `{prefix}/{postfix}`, and verify that the dataset has `{file_num}` files.
45
+
46
+ Once the dataset is verified, it will generate `{train, validation}.cache` in Tensor format, which accelerates the dataset preparation speed.
47
+
48
+ Inference
49
+ -----------------
50
+
51
+ In streaming mode, the model will infer the most recent frame and draw the bounding boxes by default, given the save flag to save the image. In other modes, it will save the predictions to `runs/inference/{exp_name}/outputs/` by default.
52
+
53
+ Dataloader Return Type
54
+ ~~~~~~~~~~~~~~~~~~~~~
55
+
56
+ For each iteration, the return type of `StreamDataLoader` includes:
57
+
58
+ - **images**: tensor, the size of each batch, used to calculate batch average loss.
59
+ - **rev_tensor**: tensor, reverse tensor for reverting the bounding boxes and images to the input shape.
60
+ - **origin_frame**: tensor, the original input image.
61
+
62
+ Input Type
63
+ ~~~~~~~~~~
64
+
65
+ - **Stream Input**:
66
+
67
+ - **webcam**: :guilabel:`int`, ID of the webcam, for example, 0, 1.
68
+ - **rtmp**: :guilabel:`str`, RTMP address.
69
+
70
+ - **Single Source**:
71
+
72
+ - **image**: :guilabel:`Path`, path to image files (`jpeg`, `jpg`, `png`, `tiff`).
73
+ - **video**: :guilabel:`Path`, path to video files (`mp4`).
74
+
75
+ - **Folder**:
76
+
77
+ - **folder of images**: :guilabel:`Path`, the relative or absolute path to the folder containing images.
docs/1_tutorials/4_train.rst ADDED
@@ -0,0 +1,55 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Train & Validation
2
+ ==================
3
+
4
+ Training Model
5
+ ----------------
6
+
7
+ To train a model, the :class:`~yolo.tools.solver.ModelTrainer` can help manage the training process. Initialize the :class:`~yolo.tools.solver.ModelTrainer` and use the :func:`~yolo.tools.solver.ModelTrainer.solve` function to start the training.
8
+
9
+ Before starting the training, don't forget to start the progress logger to enable logging the process status. This will also enable `Weights & Biases (wandb) <https://wandb.ai/site>`_ or TensorBoard if configured.
10
+
11
+ .. code-block:: python
12
+
13
+ from yolo import ModelTrainer
14
+ solver = ModelTrainer(cfg, model, converter, progress, device, use_ddp)
15
+ progress.start()
16
+ solver.solve(dataloader)
17
+
18
+ Training Diagram
19
+ ~~~~~~~~~~~~~~~~
20
+
21
+ The following diagram illustrates the training process:
22
+
23
+ .. mermaid::
24
+
25
+ flowchart LR
26
+ subgraph TS["trainer.solve"]
27
+ subgraph TE["train one epoch"]
28
+ subgraph "train one batch"
29
+ backpropagation-->TF[forward]
30
+ TF-->backpropagation
31
+ end
32
+ end
33
+ subgraph validator.solve
34
+ VC["calculate mAP"]-->VF[forward]
35
+ VF[forward]-->VC
36
+ end
37
+ end
38
+ TE-->validator.solve
39
+ validator.solve-->TE
40
+
41
+ Validation Model
42
+ ----------------
43
+
44
+ To validate the model performance, we follow a similar approach as the training process using :class:`~yolo.tools.solver.ModelValidator`.
45
+
46
+ .. code-block:: python
47
+
48
+ from yolo import ModelValidator
49
+ solver = ModelValidator(cfg, model, converter, progress, device, use_ddp)
50
+ progress.start()
51
+ solver.solve(dataloader)
52
+
53
+ The :class:`~yolo.tools.solver.ModelValidator` class helps manage the validation process, ensuring that the model's performance is evaluated accurately.
54
+
55
+ .. note:: The original training process already includes the validation phase. Call this separately if you want to run the validation again after the training is completed.
docs/1_tutorials/5_inference.rst ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Inference
2
+ ==========
3
+
4
+
5
+ Inference Video
6
+ ---------------
7
+
8
+ Inference Image
9
+ ---------------
10
+ task: inference
11
+
12
+ fast_inference: # onnx, trt, deploy or Empty
13
+ data:
14
+ source: demo/images/inference/image.png
15
+ image_size: ${image_size}
16
+ data_augment: {}
17
+ nms:
18
+ min_confidence: 0.5
19
+ min_iou: 0.5
20
+ # save_predict: True
docs/conf.py CHANGED
@@ -20,6 +20,7 @@ sys.path.insert(0, os.path.abspath(".."))
20
 
21
  extensions = [
22
  "sphinx_rtd_theme",
 
23
  "sphinxcontrib.mermaid",
24
  "sphinx.ext.autodoc",
25
  "sphinx.ext.autosectionlabel",
@@ -35,7 +36,9 @@ myst_enable_extensions = [
35
  "deflist",
36
  ]
37
  html_theme = "sphinx_rtd_theme"
38
-
 
 
39
 
40
  templates_path = ["_templates"]
41
  exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
 
20
 
21
  extensions = [
22
  "sphinx_rtd_theme",
23
+ "sphinx_tabs.tabs",
24
  "sphinxcontrib.mermaid",
25
  "sphinx.ext.autodoc",
26
  "sphinx.ext.autosectionlabel",
 
36
  "deflist",
37
  ]
38
  html_theme = "sphinx_rtd_theme"
39
+ html_theme_options = {
40
+ "sticky_navigation": False,
41
+ }
42
 
43
  templates_path = ["_templates"]
44
  exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
docs/index.rst CHANGED
@@ -33,9 +33,12 @@ Explore our documentation:
33
  :maxdepth: 1
34
  :caption: Tutorials
35
 
36
- 1_tutorials/0_train
37
- 1_tutorials/1_validation
38
- 1_tutorials/2_inference
 
 
 
39
 
40
 
41
  .. toctree::
 
33
  :maxdepth: 1
34
  :caption: Tutorials
35
 
36
+ 1_tutorials/0_allIn1
37
+ 1_tutorials/1_setup
38
+ 1_tutorials/2_buildmodel
39
+ 1_tutorials/3_dataset
40
+ 1_tutorials/4_train
41
+ 1_tutorials/5_inference
42
 
43
 
44
  .. toctree::