🔀 [Merge] branch 'SETUP'

.readthedocs.yaml

@@ -1,5 +1,10 @@
 version: 2
 
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.12"
+
 sphinx:
   configuration: docs/conf.py
 

docs/0_get_start/0_quick_start.md

@@ -1 +0,0 @@
-# Quick Start

docs/0_get_start/0_quick_start.rst

@@ -0,0 +1,71 @@
+Quick Start
+===========
+
+.. note::
+   We expect all customizations to be done primarily by passing arguments or modifying the YAML config files.
+   If more detailed modifications are needed, custom content should be modularized as much as possible to avoid extensive code modifications.
+
+.. _QuickInstallYOLO:
+
+Install YOLO
+------------
+
+Clone the repository and install the dependencies:
+
+.. code-block:: bash
+
+   git clone https://github.com/WongKinYiu/YOLO.git
+   cd YOLO
+   pip install -r requirements-dev.txt
+   # Make sure to work inside the cloned folder.
+
+Alternatively, If you are planning to make a simple change:
+
+**Note**: In the following examples, you should replace ``python yolo/lazy.py`` with ``yolo`` .
+
+.. code-block:: bash
+
+   pip install git+https://github.com/WongKinYiu/YOLO.git
+
+**Note**: Most tasks already include at yolo/lazy.py, so you can run with this prefix and follow arguments: ``python yolo/lazy.py``
+
+
+Train Model
+-----------
+
+To train the model, use the following command:
+
+.. code-block:: bash
+
+   python yolo/lazy.py task=train
+
+   yolo task=train # if installed via pip
+
+- Overriding the ``dataset`` parameter, you can customize your dataset via a dataset config.
+- Overriding YOLO model by setting the ``model`` parameter to ``{v9-c, v9-m, ...}``.
+- More details can be found at :ref:`Train Tutorials<Train>`.
+
+For example:
+
+.. code-block:: bash
+
+   python yolo/lazy.py task=train dataset=AYamlFilePath model=v9-m
+
+   yolo task=train dataset=AYamlFilePath model=v9-m # if installed via pip
+
+Inference & Deployment
+------------------------
+
+Inference is the default task of ``yolo/lazy.py``. To run inference and deploy the model, use:
+More details can be found at :ref:`Inference Tutorials <Inference>`.
+
+.. code-block:: bash
+
+   python yolo/lazy.py task.data.source=AnySource
+
+   yolo task.data.source=AnySource # if installed via pip
+
+You can enable fast inference modes by adding the parameter ``task.fast_inference={onnx, trt, deploy}``.
+
+- Theoretical acceleration following :ref:`YOLOv9 <Deploy>`.
+- Hardware acceleration like :ref:`ONNX <ONNX>` and :ref:`TensorRT <TensorRT>`. for optimized deployment.

docs/0_get_start/1_installations.md

@@ -1,4 +0,0 @@
-# Install YOLO
-
-## GPU (Cuda / MPS)
-## CPU only

docs/0_get_start/1_introduction.rst

@@ -0,0 +1,66 @@
+What is YOLO
+============
+
+``YOLO`` (You Only Look Once) is a state-of-the-art, real-time object detection system. It is designed to predict bounding boxes and class probabilities for objects in an image with high accuracy and speed. YOLO models, including the latest YOLOv9, are known for their efficiency in detecting objects in a single forward pass through the network, making them highly suitable for real-time applications.
+
+YOLOv9 introduces improvements in both architecture and loss functions to enhance prediction accuracy and inference speed.
+
+Forward Process
+---------------
+
+The forward process of YOLOv9 can be visualized as follows:
+
+.. mermaid::
+
+    graph LR
+    subgraph YOLOv9
+        Auxiliary
+        AP["Auxiliary Prediction"]
+    end
+    BackBone-->FPN;
+    FPN-->PAN;
+    PAN-->MP["Main Prediction"];
+    BackBone-->Auxiliary;
+    Auxiliary-->AP;
+
+- **BackBone**: Extracts features from the input image.
+- **FPN (Feature Pyramid Network)**: Aggregates features at different scales.
+- **PAN (Region Proposal Network)**: Proposes regions of interest.
+- **Main Prediction**: The primary detection output.
+- **Auxiliary Prediction**: Additional predictions to assist the main prediction.
+
+Loss Function
+-------------
+
+The loss function of YOLOv9 combines several components to optimize the model's performance:
+
+.. mermaid::
+
+    flowchart LR
+        gtb-->cls
+        gtb["Ground Truth"]-->iou
+        pdm-.->cls["Max Class"]
+        pdm["Main Prediction"]-.->iou["Closest IoU"]
+        pdm-.->anc["box in anchor"]
+        cls-->gt
+        iou-->gt["Matched GT Box"]
+        anc-.->gt
+
+        gt-->Liou["IoU Loss"]
+        pdm-->Liou
+        pdm-->Lbce
+        gt-->Lbce["BCE Loss"]
+        gt-->Ldfl["DFL Loss"]
+        pdm-->Ldfl
+
+        Lbce-->ML
+        Liou-->ML
+        Ldfl-->ML["Total Loss"]
+
+- **Ground Truth**: The actual labels and bounding boxes in the dataset.
+- **Main Prediction**: The model's predicted bounding boxes and class scores.
+- **IoU (Intersection over Union)**: Measures the overlap between the predicted and ground truth boxes.
+- **BCE (Binary Cross-Entropy) Loss**: Used for class prediction.
+- **DFL (Distribution Focal Loss)**: Used for improving the precision of bounding box regression.
+
+By optimizing these components, YOLOv9 aims to achieve high accuracy and robustness in object detection tasks.

docs/0_get_start/2_git.md

@@ -1,3 +0,0 @@
-# Install via Git& GitHub
-
-## [WIP]

docs/0_get_start/2_installations.rst

@@ -0,0 +1,101 @@
+Install YOLO
+============
+
+This guide will help you set up YOLO on your machine.
+We recommend starting with `GitHub Settings <#git-github>`_ for more flexible customization.
+If you are planning to perform inference only or require a simple customization, you can choose to install via `PyPI <#pypi-pip-install>`_.
+
+Torch Requirements
+-------------------
+
+The following table summarizes the torch requirements for different operating systems and hardware configurations:
+
+
+.. tabs::
+
+   .. tab:: Linux
+
+      .. tabs::
+
+         .. tab:: CUDA
+
+            PyTorch: 1.12+
+
+         .. tab:: CPU
+
+            PyTorch: 1.12+
+
+   .. tab:: MacOS
+
+      .. tabs::
+
+         .. tab:: MPS
+
+            PyTorch: 2.2+
+         .. tab:: CPU
+            PyTorch: 2.2+
+   .. tab:: Windows
+
+      .. tabs::
+
+         .. tab:: CUDA
+
+            [WIP]
+
+         .. tab:: CPU
+
+            [WIP]
+
+
+Git & GitHub
+------------
+
+First, Clone the repository:
+
+.. code-block:: bash
+
+   git clone https://github.com/WongKinYiu/YOLO.git
+
+Alternatively, you can directly download the repository via this `link <https://github.com/WongKinYiu/YOLO/archive/refs/heads/main.zip>`_.
+
+Next, install the required packages:
+
+.. code-block:: bash
+
+    # For the minimal requirements, use:
+    pip install -r requirements.txt
+    # For a full installation, use:
+    pip install -r requirements-dev.txt
+
+Moreover, if you plan to utilize ONNX or TensorRT, please follow :ref:`ONNX`, :ref:`TensorRT` for more installation details.
+
+PyPI (pip install)
+------------------
+
+.. note::
+    Due to the :guilabel:`yolo` this name already being occupied in the PyPI library, we are still determining the package name.
+    Currently, we provide an alternative way to install via the GitHub repository. Ensure your shell has `git` and `pip3` (or `pip`).
+
+To install YOLO via GitHub:
+
+.. code-block:: bash
+
+   pip install git+https://github.com/WongKinYiu/YOLO.git
+
+Docker
+------
+
+To run YOLO using NVIDIA Docker, you can pull the Docker image and run it with GPU support:
+
+.. code-block:: bash
+
+   docker pull henrytsui000/yolo
+   docker run --gpus all -it henrytsui000/yolo
+
+Make sure you have the NVIDIA Docker toolkit installed. For more details on setting up NVIDIA Docker, refer to the `NVIDIA Docker documentation <https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html>`_.
+
+
+Conda
+-----
+
+We will publish it in the near future!

docs/0_get_start/3_pypi.md

@@ -1,3 +0,0 @@
-# Install via PyPi(pip install)
-
-## [WIP]

docs/0_get_start/4_docker.md

@@ -1,3 +0,0 @@
-# Install in Docker
-
-## [WIP]

docs/0_get_start/5_conda.md

@@ -1,3 +0,0 @@
-# Install in Conda env
-
-## [WIP]

docs/1_tutorials/0_allIn1.rst

@@ -0,0 +1,204 @@
+All In 1
+========
+
+:file:`yolo.lazy` is a packaged file that includes :guilabel:`training`, :guilabel:`validation`, and :guilabel:`inference` tasks.
+For detailed function documentation, thercheck out the IPython notebooks to learn how to import and use these function
+the following section will break down operation inside of lazy, also supporting directly import/call the function.
+
+[TOC], setup, build, dataset, train, validation, inference
+To train the model, you can run:
+
+Train Model
+----------
+
+
+- batch size check / cuda
+- training time / check
+- build model / check
+- dataset / check
+
+.. code-block:: bash
+
+    python yolo/lazy.py task=train
+
+You can customize the training process by overriding the following common arguments:
+
+- ``name``: :guilabel:`str`
+  The experiment name.
+
+- ``model``: :guilabel:`str`
+  Model backbone, options include [model_zoo] v9-c, v7, v9-e, etc.
+
+- ``cpu_num``: :guilabel:`int`
+  Number of CPU workers (num_workers).
+
+- ``out_path``: :guilabel:`Path`
+  The output path for saving models and logs.
+
+- ``weight``: :guilabel:`Path | bool | None`
+  The path to pre-trained weights, False for training from scratch, None for default weights.
+
+- ``use_wandb``: :guilabel:`bool`
+  Whether to use Weights and Biases for experiment tracking.
+
+- ``use_TensorBoard``: :guilabel:`bool`
+  Whether to use TensorBoard for logging.
+
+- ``image_size``: :guilabel:`int | [int, int]`
+  The input image size.
+
+- ``+quiet``: :guilabel:`bool`
+  Optional, disable all output.
+
+- ``task.epoch``: :guilabel:`int`
+  Total number of training epochs.
+
+- ``task.data.batch_size``: :guilabel:`int`
+  The size of each batch (auto-batch sizing [WIP]).
+
+Examples
+~~~~~~~~
+
+To train a model with a specific batch size and image size, you can run:
+
+.. code-block:: bash
+
+    python yolo/lazy.py task=train task.data.batch_size=12 image_size=1280
+
+Multi-GPU Training with DDP
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For multi-GPU training, we use Distributed Data Parallel (DDP) for efficient and scalable training.
+DDP enable training model with mutliple GPU, even the GPUs aren't on the same machine. For more details, you can refer to the `DDP tutorial <https://pytorch.org/tutorials/intermediate/ddp_tutorial.html>`_.
+
+To train on multiple GPUs, replace the ``python`` command with ``torchrun --nproc_per_node=[GPU_NUM]``. The ``nproc_per_node`` argument specifies the number of GPUs to use.
+
+
+.. tabs::
+
+   .. tab:: bash
+    .. code-block:: bash
+
+        torchrun --nproc_per_node=2 yolo/lazy.py task=train device=[0,1]
+
+   .. tab:: zsh
+    .. code-block:: bash
+
+        torchrun --nproc_per_node=2 yolo/lazy.py task=train device=\[0,1\]
+
+
+Training on a Custom Dataset
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To use the auto-download module, we suggest users construct the dataset config in the following format.
+If the config files include `auto_download`, the model will automatically download the dataset when creating the dataloader.
+
+Here is an example dataset config file:
+
+.. literalinclude:: ../../yolo/config/dataset/dev.yaml
+  :language: YAML
+
+Both of the following formats are acceptable:
+
+- ``path``: :guilabel:`str`
+  The path to the dataset.
+
+- ``train, validation``: :guilabel:`str`
+  The training and validation directory names under `/images`. If using txt as ground truth, these should also be the names under `/labels/`.
+
+- ``class_num``: :guilabel:`int`
+  The number of dataset classes.
+
+- ``class_list``: :guilabel:`List[str]`
+  Optional, the list of class names, used only for visualizing the bounding box classes.
+
+- ``auto_download``: :guilabel:`dict`
+  Optional, whether to auto-download the dataset.
+
+The dataset should include labels or annotations, preferably in JSON format for compatibility with pycocotools during inference:
+
+.. code-block:: text
+
+    DataSetName/
+    ├── annotations
+    │   ├── train_json_name.json
+    │   └── val_json_name.json
+    ├── labels/
+    │   ├── train/
+    │   │   ├── AnyLabelName.txt
+    │   │   └── ...
+    │   └── validation/
+    │       └── ...
+    └── images/
+        ├── train/
+        │   ├── AnyImageNameN.{png,jpg,jpeg}
+        │   └── ...
+        └── validation/
+            └── ...
+
+
+Validation Model
+----------------
+
+During training, this block will be auto-executed. You may also run this task manually to generate a JSON file representing the predictions for a given validation dataset. If the validation set includes JSON annotations, it will run pycocotools for evaluation.
+
+We recommend setting ``task.data.shuffle`` to False and turning off ``task.data.data_augment``.
+
+You can customize the validation process by overriding the following arguments:
+
+- ``task.nms.min_confidence``: :guilabel:`str`
+  The minimum confidence of model prediction.
+
+- ``task.nms.min_iou``: :guilabel:`str`
+  The minimum IoU threshold for NMS (Non-Maximum Suppression).
+
+Examples
+~~~~~~~~
+
+.. tabs::
+
+   .. tab:: git-cloned
+      .. code-block:: bash
+
+         python yolo/lazy.py task=validation task.nms.min_iou=0.9
+
+   .. tab:: PyPI
+      .. code-block:: bash
+
+         yolo task=validation task.nms.min_iou=0.9
+
+
+Model Inference
+---------------
+
+.. note::
+   The ``dataset`` parameter shouldn't be overridden because the model requires the ``class_num`` of the dataset. If the classes have names, please provide the ``class_list``.
+
+You can customize the inference process by overriding the following arguments:
+
+- ``task.fast_inference``: :guilabel:`str`
+  Optional. Values can be `onnx`, `trt`, `deploy`, or `None`. `deploy` will detach the model auxiliary head.
+
+- ``task.data.source``: :guilabel:`str | Path | int`
+  This argument will be auto-resolved and could be a webcam ID, image folder path, video/image path.
+
+- ``task.nms.min_confidence``: :guilabel:`str`
+  The minimum confidence of model prediction.
+
+- ``task.nms.min_iou``: :guilabel:`str`
+  The minimum IoU threshold for NMS (Non-Maximum Suppression).
+
+Examples
+~~~~~~~~
+
+.. tabs::
+
+   .. tab:: git-cloned
+      .. code-block:: bash
+
+         python yolo/lazy.py model=v9-m task.nms.min_confidence=0.1 task.data.source=0 task.fast_inference=onnx
+
+   .. tab:: PyPI
+      .. code-block:: bash
+
+         yolo model=v9-m task.nms.min_confidence=0.1 task.data.source=0 task.fast_inference=onnx

docs/1_tutorials/0_train.md

@@ -1,5 +0,0 @@
-# Train
-
-## Train on COCO2017
-
-## Train on Cusom Dataset

docs/1_tutorials/1_setup.rst

@@ -0,0 +1,35 @@
+Setup Config
+============
+
+To set up your configuration, you will need to generate a configuration class based on :class:`~yolo.config.config.Config`, which can be achieved using `hydra <https://hydra.cc/>`_.
+The configuration will include all the necessary settings for your ``task``, including general configuration, ``dataset`` information, and task-specific information (``train``, ``inference``, ``validation``).
+
+Next, create the progress logger to handle the output and progress bar. This class is based on `rich <https://github.com/Textualize/rich>`_'s progress bar and customizes the logger (print function) using `loguru <https://loguru.readthedocs.io/>`_.
+
+.. tabs::
+
+   .. tab:: decorator
+      .. code-block:: python
+
+         import hydra
+         from yolo import ProgressLogger
+         from yolo.config.config import Config
+
+         @hydra.main(config_path="config", config_name="config", version_base=None)
+         def main(cfg: Config):
+             progress = ProgressLogger(cfg, exp_name=cfg.name)
+             pass
+
+   .. tab:: initialize & compose
+      .. code-block:: python
+
+         from hydra import compose, initialize
+         from yolo import ProgressLogger
+         from yolo.config.config import Config
+
+         with initialize(config_path="config", version_base=None):
+             cfg = compose(config_name="config", overrides=["task=train", "model=v9-c"])
+
+         progress = ProgressLogger(cfg, exp_name=cfg.name)
+
+TODO: add a config over view

docs/1_tutorials/1_validation.md

@@ -1,5 +0,0 @@
-# Validation
-
-## Validation on COCO2017
-
-## Validation on Custom Dataset

docs/1_tutorials/2_buildmodel.rst

@@ -0,0 +1,62 @@
+Build Model
+===========
+
+In YOLOv7, the prediction will be ``Anchor``, and in YOLOv9, it will predict ``Vector``. The converter will turn the bounding box to the vector.
+
+The overall model flowchart is as follows:
+
+.. mermaid::
+
+    flowchart LR
+    Input-->Model;
+    Model--Class-->NMS;
+    Model--Anc/Vec-->Converter;
+    Converter--Box-->NMS;
+    NMS-->Output;
+
+Load Model
+~~~~~~~~~~
+
+Using `create_model`, it will automatically create the :class:`~yolo.model.yolo.YOLO` model and load the provided weights.
+
+Arguments:
+
+- **model**: :class:`~yolo.config.config.ModelConfig`
+  The model configuration.
+- **class_num**: :guilabel:`int`
+  The number of classes in the dataset, used for the YOLO's prediction head.
+- **weight_path**: :guilabel:`Path | bool`
+  The path to the model weights.
+    - If `False`, weights are not loaded.
+    - If :guilabel:`True | None`, default weights are loaded.
+    - If a `Path`, the model weights are loaded from the specified path.
+
+.. code-block:: python
+
+    model = create_model(cfg.model, class_num=cfg.dataset.class_num, weight_path=cfg.weight)
+    model = model.to(device)
+
+Deploy Model
+~~~~~~~~~~~~
+
+In the deployment version, we will remove the auxiliary branch of the model for fast inference. If the config includes ONNX and TensorRT, it will load/compile the model to ONNX or TensorRT format after removing the auxiliary branch.
+
+.. code-block:: python
+
+    model = FastModelLoader(cfg).load_model(device)
+
+Autoload Converter
+~~~~~~~~~~~~~~~~~~
+
+Autoload the converter based on the model type (v7 or v9).
+
+Arguments:
+
+- **Model Name**: :guilabel:`str`
+  Used for choosing ``Vec2Box`` or ``Anc2Box``.
+- **Anchor Config**: The anchor configuration, used to generate the anchor grid.
+- **model**, **image_size**: Used for auto-detecting the anchor grid.
+
+.. code-block:: python
+
+    converter = create_converter(cfg.model.name, model, cfg.model.anchor, cfg.image_size, device)

docs/1_tutorials/3_dataset.rst

@@ -0,0 +1,77 @@
+Create Dataset
+==============
+
+In this section, we will prepare the dataset and create a dataloader.
+
+Overall, the dataloader can be created by:
+
+.. code-block:: python
+
+   from yolo import create_dataloader
+   dataloader = create_dataloader(cfg.task.data, cfg.dataset, cfg.task.task, use_ddp)
+
+For inference, the dataset will be handled by :class:`~yolo.tools.data_loader.StreamDataLoader`, while for training and validation, it will be handled by :class:`~yolo.tools.data_loader.YoloDataLoader`.
+
+The input arguments are:
+
+- **DataConfig**: :class:`~yolo.config.config.DataConfig`, the relevant configuration for the dataloader.
+- **DatasetConfig**: :class:`~yolo.config.config.DatasetConfig`, the relevant configuration for the dataset.
+- **task_name**: :guilabel:`str`, the task name, which can be `inference`, `validation`, or `train`.
+- **use_ddp**: :guilabel:`bool`, whether to use DDP (Distributed Data Parallel). Default is `False`.
+
+Train and Validation
+----------------------------
+
+Dataloader Return Type
+~~~~~~~~~~~~~~~~~~~~~
+
+For each iteration, the return type includes:
+
+- **batch_size**: the size of each batch, used to calculate batch average loss.
+- **images**: the input images.
+- **targets**: the ground truth of the images according to the task.
+
+Auto Download Dataset
+~~~~~~~~~~~~~~~~~~~~~
+
+The dataset will be auto-downloaded if the user provides the `auto_download` configuration. For example, if the configuration is as follows:
+
+
+.. literalinclude:: ../../yolo/config/dataset/mock.yaml
+  :language: YAML
+
+
+First, it will download and unzip the dataset from `{prefix}/{postfix}`, and verify that the dataset has `{file_num}` files.
+
+Once the dataset is verified, it will generate `{train, validation}.cache` in Tensor format, which accelerates the dataset preparation speed.
+
+Inference
+-----------------
+
+In streaming mode, the model will infer the most recent frame and draw the bounding boxes by default, given the save flag to save the image. In other modes, it will save the predictions to `runs/inference/{exp_name}/outputs/` by default.
+
+Dataloader Return Type
+~~~~~~~~~~~~~~~~~~~~~
+
+For each iteration, the return type of `StreamDataLoader` includes:
+
+- **images**: tensor, the size of each batch, used to calculate batch average loss.
+- **rev_tensor**: tensor, reverse tensor for reverting the bounding boxes and images to the input shape.
+- **origin_frame**: tensor, the original input image.
+
+Input Type
+~~~~~~~~~~
+
+- **Stream Input**:
+
+  - **webcam**: :guilabel:`int`, ID of the webcam, for example, 0, 1.
+  - **rtmp**: :guilabel:`str`, RTMP address.
+
+- **Single Source**:
+
+  - **image**: :guilabel:`Path`, path to image files (`jpeg`, `jpg`, `png`, `tiff`).
+  - **video**: :guilabel:`Path`, path to video files (`mp4`).
+
+- **Folder**:
+
+  - **folder of images**: :guilabel:`Path`, the relative or absolute path to the folder containing images.

docs/1_tutorials/4_train.rst

@@ -0,0 +1,55 @@
+Train & Validation
+==================
+
+Training Model
+----------------
+
+To train a model, the :class:`~yolo.tools.solver.ModelTrainer` can help manage the training process. Initialize the :class:`~yolo.tools.solver.ModelTrainer` and use the :func:`~yolo.tools.solver.ModelTrainer.solve` function to start the training.
+
+Before starting the training, don't forget to start the progress logger to enable logging the process status. This will also enable `Weights & Biases (wandb) <https://wandb.ai/site>`_ or TensorBoard if configured.
+
+.. code-block:: python
+
+  from yolo import ModelTrainer
+  solver = ModelTrainer(cfg, model, converter, progress, device, use_ddp)
+  progress.start()
+  solver.solve(dataloader)
+
+Training Diagram
+~~~~~~~~~~~~~~~~
+
+The following diagram illustrates the training process:
+
+.. mermaid::
+
+  flowchart LR
+    subgraph TS["trainer.solve"]
+      subgraph TE["train one epoch"]
+        subgraph "train one batch"
+          backpropagation-->TF[forward]
+          TF-->backpropagation
+        end
+      end
+      subgraph validator.solve
+          VC["calculate mAP"]-->VF[forward]
+          VF[forward]-->VC
+      end
+    end
+    TE-->validator.solve
+    validator.solve-->TE
+
+Validation Model
+----------------
+
+To validate the model performance, we follow a similar approach as the training process using :class:`~yolo.tools.solver.ModelValidator`.
+
+.. code-block:: python
+
+   from yolo import ModelValidator
+   solver = ModelValidator(cfg, model, converter, progress, device, use_ddp)
+   progress.start()
+   solver.solve(dataloader)
+
+The :class:`~yolo.tools.solver.ModelValidator` class helps manage the validation process, ensuring that the model's performance is evaluated accurately.
+
+.. note:: The original training process already includes the validation phase. Call this separately if you want to run the validation again after the training is completed.

docs/1_tutorials/5_inference.rst

@@ -0,0 +1,20 @@
+Inference
+==========
+
+
+Inference Video
+---------------
+
+Inference Image
+---------------
+task: inference
+
+fast_inference:  # onnx, trt, deploy or Empty
+data:
+    source: demo/images/inference/image.png
+    image_size: ${image_size}
+    data_augment: {}
+nms:
+  min_confidence: 0.5
+  min_iou: 0.5
+# save_predict: True

docs/2_model_zoo/0_object_detection.md

@@ -1,5 +0,0 @@
-# Object Detection
-
-## YOLOv7
-
-## YOLOv9

docs/2_model_zoo/0_object_detection.rst

@@ -0,0 +1,169 @@
+Object Detection
+================
+
+YOLOv7
+~~~~~~
+
+
+.. list-table::
+   :header-rows: 1
+
+   * - Model
+     - State
+     - Test Size
+     - :math:`AP^{val}`
+     - :math:`AP_{50}^{val}`
+     - :math:`AP_{75}^{val}`
+     - Param.
+     - FLOPs
+   * - `YOLOv7 <https://github.com/WongKinYiu/YOLO/releases/download/v1.0-alpha/v7.pt>`_
+     - 🔧
+     - 640
+     - **51.4%**
+     - **69.7%**
+     - **55.9%**
+     -
+     -
+   * - `YOLOv7-X <URL>`_
+     - 🔧
+     - 640
+     - **53.1%**
+     - **71.2%**
+     - **57.8%**
+     -
+     -
+   * - `YOLOv7-W6 <URL>`_
+     - 🔧
+     - 1280
+     - **54.9%**
+     - **72.6%**
+     - **60.1%**
+     -
+     -
+   * - `YOLOv7-E6 <URL>`_
+     - 🔧
+     - 1280
+     - **56.0%**
+     - **73.5%**
+     - **61.2%**
+     -
+     -
+   * - `YOLOv7-D6 <URL>`_
+     - 🔧
+     - 1280
+     - **56.6%**
+     - **74.0%**
+     - **61.8%**
+     -
+     -
+   * - `YOLOv7-E6E <URL>`_
+     - 🔧
+     - 1280
+     - **56.8%**
+     - **74.4%**
+     - **62.1%**
+     -
+     -
+
+YOLOv9
+~~~~~~
+
+.. list-table::
+   :header-rows: 1
+
+   * - Model
+     - State
+     - Test Size
+     - :math:`AP^{val}`
+     - :math:`AP_{50}^{val}`
+     - :math:`AP_{75}^{val}`
+     - Param.
+     - FLOPs
+   * - `YOLOv9-T <https://github.com/WongKinYiu/YOLO/releases/download/v1.0-alpha/v9-t.pt>`_
+     - 🔧
+     - 640
+     -
+     -
+     -
+     -
+     -
+   * - `YOLOv9-S <https://github.com/WongKinYiu/YOLO/releases/download/v1.0-alpha/v9-s.pt>`_
+     - ✅
+     - 640
+     - **46.8%**
+     - **63.4%**
+     - **50.7%**
+     - **7.1M**
+     - **26.4G**
+   * - `YOLOv9-M <https://github.com/WongKinYiu/YOLO/releases/download/v1.0-alpha/v9-m.pt>`_
+     - ✅
+     - 640
+     - **51.4%**
+     - **68.1%**
+     - **56.1%**
+     - **20.0M**
+     - **76.3G**
+   * - `YOLOv9-C <https://github.com/WongKinYiu/YOLO/releases/download/v1.0-alpha/v9-c.pt>`_
+     - ✅
+     - 640
+     - **53.0%**
+     - **70.2%**
+     - **57.8%**
+     - **25.3M**
+     - **102.1G**
+   * - `YOLOv9-E <https://github.com/WongKinYiu/YOLO/releases/download/v1.0-alpha/v9-e.pt>`_
+     - 🔧
+     - 640
+     - **55.6%**
+     - **72.8%**
+     - **60.6%**
+     - **57.3M**
+     - **189.0G**
+
+
+
+
+.. mermaid::
+
+    graph LR
+    subgraph BackBone
+    B1-->B2;
+    B2-->B3;
+    B3-->B4;
+    B4-->B5;
+    end
+
+    subgraph FPN
+    B3-->N3;
+    B4-->N4;
+    B5-->N5;
+    N5-->N4;
+    N4-->N3;
+    end
+
+    subgraph PAN
+    P3-->P4;
+    P4-->P5;
+    N3-->P3;
+    N4-->P4;
+    N5-->P5;
+    end
+
+    P3-->Main_Head;
+    P4-->Main_Head;
+    P5-->Main_Head;
+
+    subgraph Aux
+    B3-->R3;
+    B4-->R4;
+    B5-->R5;
+    R3-->A3;
+    R4-->A3;
+    R4-->A4;
+    R5-->A3;
+    R5-->A4;
+    R5-->A5;
+    end
+    A3-->Auxiliary_Head;
+    A4-->Auxiliary_Head;
+    A5-->Auxiliary_Head;

docs/2_model_zoo/1_segmentation.md

@@ -1,5 +0,0 @@
-# Segmentations
-
-## YOLOv7
-
-## YOLOv9

docs/2_model_zoo/1_segmentation.rst

@@ -0,0 +1,11 @@
+Segmentations
+=============
+.. _YOLOv7-seg:
+
+YOLOv7
+------
+
+.. _YOLOv9-seg:
+
+YOLOv9
+------

docs/2_model_zoo/2_classification.md

@@ -1,5 +0,0 @@
-# Classification
-
-## YOLOv7
-
-## YOLOv9

docs/2_model_zoo/2_classification.rst

@@ -0,0 +1,4 @@
+Classification
+==============
+
+[WIP]

docs/3_custom/0_model.md

@@ -1 +0,0 @@
-# Model

docs/3_custom/0_model.rst

@@ -0,0 +1,12 @@
+Model
+=====
+
+Modified Architecture
+---------------------
+
+
+
+
+
+Modified Model Module
+---------------------

docs/3_custom/1_data_augment.md

@@ -1 +0,0 @@
-# Data Augment

docs/3_custom/1_data_augment.rst

@@ -0,0 +1,4 @@
+.. _DataAugment:
+
+Data Augment
+============

docs/3_custom/2_loss.md

@@ -1 +0,0 @@
-# Loss

docs/3_custom/2_loss.rst

@@ -0,0 +1,2 @@
+Loss Function
+=============

docs/3_custom/3_task.md

@@ -1 +0,0 @@
-# Task

docs/3_custom/3_task.rst

@@ -0,0 +1,2 @@
+Custom Task
+===========

docs/4_deploy/1_deploy.md

@@ -1,3 +0,0 @@
-# Deploy YOLOv9
-
-# Deploy YOLOv7

docs/4_deploy/1_deploy.rst

@@ -0,0 +1,10 @@
+.. _Deploy:
+
+Deploy Model
+============
+
+Deploy YOLOv9
+-------------
+
+Deploy YOLOv7
+-------------

docs/4_deploy/2_onnx.md

@@ -1 +0,0 @@
-# Compile to ONNX

docs/4_deploy/2_onnx.rst

@@ -0,0 +1,4 @@
+.. _ONNX:
+
+Compile to ONNX
+===============

docs/4_deploy/3_tensorrt.md

@@ -1 +0,0 @@
-# Compile to TensorRT

docs/4_deploy/3_tensorrt.rst

@@ -0,0 +1,5 @@
+.. _TensorRT:
+
+
+Compile to TensorRT
+===================

docs/5_features/0_small_object.rst

@@ -0,0 +1,2 @@
+Small Object
+============

docs/5_features/1_version_convert.rst

@@ -0,0 +1,2 @@
+Version Convert
+===============

docs/5_features/2_IPython.rst

@@ -0,0 +1,2 @@
+IPython
+=======

docs/6_function_docs/0_solver.rst

@@ -0,0 +1,12 @@
+Solver
+======
+
+.. automodule:: yolo.tools.solver
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+.. automodule:: yolo.utils.bounding_box_utils
+    :members:
+    :undoc-members:
+    :show-inheritance:

docs/6_function_docs/1_tools.rst

@@ -0,0 +1,4 @@
+.. _Tools:
+
+Useful Tools
+============

docs/6_function_docs/2_module.rst

@@ -0,0 +1,4 @@
+.. _Module:
+
+Model Module
+============

docs/6_function_docs/3_config.rst

@@ -0,0 +1,188 @@
+Config
+======
+
+
+
+.. autoclass:: yolo.config.config.Config
+    :members:
+    :undoc-members:
+
+.. automodule:: yolo.config.config
+    :members:
+    :undoc-members:
+
+
+
+.. mermaid::
+
+    classDiagram
+    class AnchorConfig {
+        List~int~ strides
+        Optional~int~ reg_max
+        Optional~int~ anchor_num
+        List~List~int~~ anchor
+    }
+
+    class LayerConfig {
+        Dict args
+        Union~List~int~~ source
+        str tags
+    }
+
+    class BlockConfig {
+        List~Dict~LayerConfig~~ block
+    }
+
+    class ModelConfig {
+        Optional~str~ name
+        AnchorConfig anchor
+        Dict~BlockConfig~ model
+    }
+
+    AnchorConfig --> ModelConfig
+    LayerConfig --> BlockConfig
+    BlockConfig --> ModelConfig
+
+.. mermaid::
+
+    classDiagram
+    class DownloadDetail {
+        str url
+        int file_size
+    }
+
+    class DownloadOptions {
+        Dict~DownloadDetail~ details
+    }
+
+    class DatasetConfig {
+        str path
+        int class_num
+        List~str~ class_list
+        Optional~DownloadOptions~ auto_download
+    }
+
+    class DataConfig {
+        bool shuffle
+        int batch_size
+        bool pin_memory
+        int cpu_num
+        List~int~ image_size
+        Dict~int~ data_augment
+        Optional~Union~str~~ source
+    }
+
+    DownloadDetail --> DownloadOptions
+    DownloadOptions --> DatasetConfig
+
+.. mermaid::
+
+    classDiagram
+    class OptimizerArgs {
+        float lr
+        float weight_decay
+    }
+
+    class OptimizerConfig {
+        str type
+        OptimizerArgs args
+    }
+
+    class MatcherConfig {
+        str iou
+        int topk
+        Dict~str~ factor
+    }
+
+    class LossConfig {
+        Dict~str~ objective
+        Union~bool~ aux
+        MatcherConfig matcher
+    }
+
+    class SchedulerConfig {
+        str type
+        Dict~str~ warmup
+        Dict~str~ args
+    }
+
+    class EMAConfig {
+        bool enabled
+        float decay
+    }
+
+    class TrainConfig {
+        str task
+        int epoch
+        DataConfig data
+        OptimizerConfig optimizer
+        LossConfig loss
+        SchedulerConfig scheduler
+        EMAConfig ema
+        ValidationConfig validation
+    }
+
+    class NMSConfig {
+        int min_confidence
+        int min_iou
+    }
+
+    class InferenceConfig {
+        str task
+        NMSConfig nms
+        DataConfig data
+        Optional~None~ fast_inference
+        bool save_predict
+    }
+
+    class ValidationConfig {
+        str task
+        NMSConfig nms
+        DataConfig data
+    }
+
+    OptimizerArgs --> OptimizerConfig
+    OptimizerConfig --> TrainConfig
+    MatcherConfig --> LossConfig
+    LossConfig --> TrainConfig
+    SchedulerConfig --> TrainConfig
+    EMAConfig --> TrainConfig
+    NMSConfig --> InferenceConfig
+    NMSConfig --> ValidationConfig
+
+
+.. mermaid::
+
+    classDiagram
+    class GeneralConfig {
+        str name
+        Union~str~ device
+        int cpu_num
+        List~int~ class_idx_id
+        List~int~ image_size
+        str out_path
+        bool exist_ok
+        int lucky_number
+        bool use_wandb
+        bool use_TensorBoard
+        Optional~str~ weight
+    }
+
+.. mermaid::
+
+    classDiagram
+    class Config {
+        Union~ValidationConfig~ task
+        DatasetConfig dataset
+        ModelConfig model
+        GeneralConfig model
+    }
+
+    DatasetConfig --> Config
+    DataConfig --> TrainConfig
+    DataConfig --> InferenceConfig
+    DataConfig --> ValidationConfig
+    InferenceConfig --> Config
+    ValidationConfig --> Config
+    TrainConfig --> Config
+    GeneralConfig --> Config

docs/6_function_docs/4_dataloader.rst

@@ -0,0 +1,8 @@
+Dataloader
+==========
+
+
+
+.. automodule:: yolo.tools.data_loader
+    :members:
+    :undoc-members:

docs/MODELS.md

@@ -1,30 +0,0 @@
-# YOLO Model Zoo
-
-Welcome to the YOLOv9 Model Zoo! Here, you will find a variety of pre-trained models tailored to different use cases and performance needs. Each model comes with detailed information about its training regime, performance metrics, and usage instructions.
-
-## Standard Models
-
-These models are trained on common datasets like COCO and provide a balance between speed and accuracy.
-
-
-| Model | Support? |Test Size | AP<sup>val</sup> | AP<sub>50</sub><sup>val</sup> | AP<sub>75</sub><sup>val</sup> | Param. | FLOPs |
-| :-- | :-: | :-: | :-: | :-: | :-: | :-: | :-: |
-| [**YOLOv9-S**]() |✅ | 640  | **46.8%** | **63.4%** | **50.7%** | **7.1M** | **26.4G** |
-| [**YOLOv9-M**]() |✅ | 640  | **51.4%** | **68.1%** | **56.1%** | **20.0M** | **76.3G** |
-| [**YOLOv9-C**]() |✅ | 640  | **53.0%** | **70.2%** | **57.8%** | **25.3M** | **102.1G** |
-| [**YOLOv9-E**]() | 🔧 | 640  | **55.6%** | **72.8%** | **60.6%** | **57.3M** | **189.0G** |
-|  |  |  |  |  |  |  |
-| [**YOLOv7**]() |🔧 | 640  | **51.4%** | **69.7%** | **55.9%** |
-| [**YOLOv7-X**]() |🔧 | 640  | **53.1%** | **71.2%** | **57.8%** |
-| [**YOLOv7-W6**]() | 🔧 | 1280 | **54.9%** | **72.6%** | **60.1%** |
-| [**YOLOv7-E6**]() | 🔧 | 1280 | **56.0%** | **73.5%** | **61.2%** |
-| [**YOLOv7-D6**]() | 🔧 | 1280 | **56.6%** | **74.0%** | **61.8%** |
-| [**YOLOv7-E6E**]() | 🔧 | 1280 | **56.8%** | **74.4%** | **62.1%** |
-
-## Download and Usage Instructions
-
-To use these models, download them from the links provided and use the following command to run detection:
-
-```bash
-$yolo detect weights=path/to/model.pt img=640 conf=0.25 source=your_image.jpg
-```

docs/conf.py

@@ -13,9 +13,20 @@ author = "Kin-Yiu, Wong and Hao-Tang, Tsui"
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
 
+import os
+import sys
+
+sys.path.insert(0, os.path.abspath(".."))
+
 extensions = [
-    "sphinx.ext.autodoc",
     "sphinx_rtd_theme",
+    "sphinx_tabs.tabs",
+    "sphinxcontrib.mermaid",
+    "sphinx.ext.autodoc",
+    "sphinx.ext.autosectionlabel",
+    "sphinx.ext.viewcode",
+    "sphinx.ext.napoleon",
+    "linuxdoc.rstFlatTable",
     "myst_parser",
 ]
 
@@ -25,7 +36,9 @@ myst_enable_extensions = [
     "deflist",
 ]
 html_theme = "sphinx_rtd_theme"
-
+html_theme_options = {
+    "sticky_navigation": False,
+}
 
 templates_path = ["_templates"]
 exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]

docs/index.rst

@@ -22,58 +22,67 @@ Explore our documentation:
 
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
    :caption: Get Started
 
-   0_get_start/0_quick_start.md
-   0_get_start/1_installations.md
-   0_get_start/2_git.md
-   0_get_start/3_pypi.md
-   0_get_start/4_docker.md
-   0_get_start/5_conda.md
+   0_get_start/0_quick_start
+   0_get_start/1_introduction
+   0_get_start/2_installations
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
    :caption: Tutorials
 
-   1_tutorials/0_train.md
-   1_tutorials/1_validation.md
+   1_tutorials/0_allIn1
+   1_tutorials/1_setup
+   1_tutorials/2_buildmodel
+   1_tutorials/3_dataset
+   1_tutorials/4_train
+   1_tutorials/5_inference
 
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
    :caption: Model Zoo
 
-   2_model_zoo/0_object_detection.md
-   2_model_zoo/1_segmentation.md
-   2_model_zoo/2_classification.md
+   2_model_zoo/0_object_detection
+   2_model_zoo/1_segmentation
+   2_model_zoo/2_classification
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
    :caption: Custom YOLO
 
-   3_custom/0_model.md
-   3_custom/1_data_augment.md
-   3_custom/2_loss.md
-   3_custom/3_task.md
+   3_custom/0_model
+   3_custom/1_data_augment
+   3_custom/2_loss
+   3_custom/3_task
 
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
    :caption: Deploy
 
-   4_deploy/1_deploy.md
-   4_deploy/2_onnx.md
-   4_deploy/3_tensorrt.md
+   4_deploy/1_deploy
+   4_deploy/2_onnx
+   4_deploy/3_tensorrt
 
 
 .. toctree::
-   :maxdepth: 2
-   :caption: Deploy
+   :maxdepth: 1
+   :caption: Features
+
+   5_features/0_small_object
+   5_features/1_version_convert
+   5_features/2_IPython
+
+.. toctree::
+   :maxdepth: 1
+   :caption: Function Docs
 
-   4_deploy/1_deploy.md
-   4_deploy/2_onnx.md
-   4_deploy/3_tensorrt.md
+   6_function_docs/0_solver
+   6_function_docs/1_tools
+   6_function_docs/2_module
 
 License
 -------

docs/requirements.txt

@@ -1,3 +1,6 @@
+myst-parser
+linuxdoc
 sphinx
+sphinx-tabs
 sphinx_rtd_theme
-myst-parser
+sphinxcontrib-mermaid