changed readme to match standard payload

README.md

@@ -1,22 +1,24 @@
 ---
-title: Mot Metrics
-emoji: 📚
+app_file: app.py
 colorFrom: gray
 colorTo: green
-tags:
-- evaluate
-- metric
-description: "TODO: add a description here"
+description: 'TODO: add a description here'
+emoji: "\U0001F4DA"
+pinned: false
+runme:
+  id: 01HPS3ASFJXVQR88985QNSXVN1
+  version: v3
 sdk: gradio
 sdk_version: 3.19.1
-app_file: app.py
-pinned: false
+tags:
+  - evaluate
+  - metric
+title: Mot Metrics
 ---
 
 # How to Use
 
-The MOT metrics takes two numeric arrays as input corresponding to the predictions and references bounding boxes:
-```python
+```python {"id":"01HPS3ASFHPCECERTYN7Z4Z7MN"}
 >>> import numpy as np
 >>> module = evaluate.load("SEA-AI/mot-metrics")
 >>> predicted =[[1,1,10,20,30,40,0.85],[2,1,15,25,35,45,0.78],[2,2,55,65,75,85,0.95]]
@@ -32,22 +34,46 @@ The MOT metrics takes two numeric arrays as input corresponding to the predictio
 'mota': 0.7, 'motp': 0.02981870229007634,
 'num_transfer': 0, 'num_ascend': 0,
 'num_migrate': 0}
-```
 
-## Input
-Each line of the **predictions** array is a list with the following format:
-```
-[frame ID, object ID, x, y, width, height, confidence]
-```
 
-Each line of the **references** array is a list with the following format:
-```
-[frame ID, object ID, x, y, width, height]
+>>> import evaluate
+>>> from seametrics.fo_to_payload.utils import fo_to_payload
+>>> b = fo_to_payload(
+>>>         dataset="SENTRY_VIDEOS_DATASET_QA",
+>>>         gt_field="ground_truth_det",
+>>>         models=['volcanic-sweep-3_02_2023_N_LN1_ep288_TRACKER'],
+>>>         sequence_list=["Sentry_2022_11_PROACT_CELADON_7.5M_MOB_2022_11_25_12_12_39"],
+>>>         tracking_mode=True
+>>>    )
+>>> module = evaluate.load("SEA-AI/mot-metrics")
+>>> res = module._calculate(b, max_iou=0.99)
+>>> print(res)
+{'Sentry_2022_11_PROACT_CELADON_7.5M_MOB_2022_11_25_12_12_39': {'volcanic-sweep-3_02_2023_N_LN1_ep288_TRACKER': {'idf1': 0.9543031226199543,
+   'idp': 0.9804381846635368,
+   'idr': 0.9295252225519288,
+   'recall': 0.9436201780415431,
+   'precision': 0.9953051643192489,
+   'num_unique_objects': 2,
+   'mostly_tracked': 1,
+   'partially_tracked': 0,
+   'mostly_lost': 1,
+   'num_false_positives': 6,
+   'num_misses': 76,
+   'num_switches': 1,
+   'num_fragmentations': 4,
+   'mota': 0.9384272997032641,
+   'motp': 0.5235835810268012,
+   'num_transfer': 0,
+   'num_ascend': 1,
+   'num_migrate': 0}}}
 ```
 
-The `max_iou` parameter is used to filter out the bounding boxes with IOU less than the threshold. The default value is 0.5. This means that if a ground truth and a predicted bounding boxes IoU value is less than 0.5, then the predicted bounding box is not considered for association.
+## Metric Settings
+
+The `max_iou` parameter is used to filter out the bounding boxes with IOU less than the threshold. The default value is 0.5. This means that if a ground truth and a predicted bounding boxes IoU value is less than 0.5, then the predicted bounding box is not considered for association. So, the higher the `max_iou` value, the more the predicted bounding boxes are considered for association.
 
 ## Output
+
 The output is a dictionary containing the following metrics:
 
 | Name                 | Description                                                                        |
@@ -68,16 +94,16 @@ The output is a dictionary containing the following metrics:
 | mota                 | Multiple object tracker accuracy.                                                  |
 | motp                 | Multiple object tracker precision.                                                 |
 
-
 ## Citations
-```bibtex
+
+```bibtex {"id":"01HPS3ASFJXVQR88985GKHAQRE"}
 @InProceedings{huggingface:module,
 title = {A great new module},
 authors={huggingface, Inc.},
 year={2020}}
 ```
-    
-```bibtex
+
+```bibtex {"id":"01HPS3ASFJXVQR88985KRT478N"}
 @article{milan2016mot16,
 title={MOT16: A benchmark for multi-object tracking},
 author={Milan, Anton and Leal-Taix{\'e}, Laura and Reid, Ian and Roth, Stefan and Schindler, Konrad},
@@ -86,4 +112,5 @@ year={2016}}
 ```
 
 ## Further References
+
 - [Github Repository - py-motmetrics](https://github.com/cheind/py-motmetrics/tree/develop)

mot-metrics.py

@@ -49,64 +49,6 @@ Args:
     max_iou (`float`, *optional*):
         If specified, this is the minimum Intersection over Union (IoU) threshold to consider a detection as a true positive.
         Default is 0.5.
-Returns:
-    summary: pandas.DataFrame with the following columns:
-        - idf1 (IDF1 Score): The F1 score for the identity assignment, computed as 2 * (IDP * IDR) / (IDP + IDR).
-        - idp (ID Precision): Identity Precision, representing the ratio of correctly assigned identities to the total number of predicted identities.
-        - idr (ID Recall): Identity Recall, representing the ratio of correctly assigned identities to the total number of ground truth identities.
-        - recall: Recall, computed as the ratio of the number of correctly tracked objects to the total number of ground truth objects.
-        - precision: Precision, computed as the ratio of the number of correctly tracked objects to the total number of predicted objects.
-        - num_unique_objects: Total number of unique objects in the ground truth.
-        - mostly_tracked: Number of objects that are mostly tracked throughout the sequence.
-        - partially_tracked: Number of objects that are partially tracked but not mostly tracked.
-        - mostly_lost: Number of objects that are mostly lost throughout the sequence.
-        - num_false_positives: Number of false positive detections (predicted objects not present in the ground truth).
-        - num_misses: Number of missed detections (ground truth objects not detected in the predictions).
-        - num_switches: Number of identity switches.
-        - num_fragmentations: Number of fragmented objects (objects that are broken into multiple tracks).
-        - mota (MOTA - Multiple Object Tracking Accuracy): Overall tracking accuracy, computed as 1 - ((num_false_positives + num_misses + num_switches) / num_unique_objects).
-        - motp (MOTP - Multiple Object Tracking Precision): Average precision of the object localization, computed as the mean of the localization errors of correctly detected objects.
-        - num_transfer: Number of track transfers.
-        - num_ascend: Number of ascended track IDs.
-        - num_migrate: Number of track ID migrations.
-
-Examples:
-    >>> import numpy as np
-    >>> module = evaluate.load("bascobasculino/mot-metrics")
-
-    >>> predicted =[
-            [1,1,10,20,30,40,0.85],
-            [1,2,50,60,70,80,0.92],
-            [1,3,80,90,100,110,0.75],
-            [2,1,15,25,35,45,0.78],
-            [2,2,55,65,75,85,0.95],
-            [3,1,20,30,40,50,0.88],
-            [3,2,60,70,80,90,0.82],
-            [4,1,25,35,45,55,0.91],
-            [4,2,65,75,85,95,0.89]
-        ]
-
-    >>> ground_truth = [
-            [1, 1, 10, 20, 30, 40],
-            [1, 2, 50, 60, 70, 80],
-            [1, 3, 85, 95, 105, 115],
-            [2, 1, 15, 25, 35, 45],
-            [2, 2, 55, 65, 75, 85],
-            [3, 1, 20, 30, 40, 50],
-            [3, 2, 60, 70, 80, 90],
-            [4, 1, 25, 35, 45, 55],
-            [5, 1, 30, 40, 50, 60],
-            [5, 2, 70, 80, 90, 100]
-        ]
-    >>> predicted = [np.array(a) for a in predicted]
-    >>> ground_truth = [np.array(a) for a in ground_truth]
-
-    >>> results = module._compute(predictions=predicted, references=ground_truth, max_iou=0.5)
-    >>> print(results)
-    {'idf1': 0.8421052631578947, 'idp': 0.8888888888888888, 'idr': 0.8, 'recall': 0.8, 'precision': 0.8888888888888888,
-    'num_unique_objects': 3,'mostly_tracked': 2, 'partially_tracked': 1, 'mostly_lost': 0, 'num_false_positives': 1, 
-    'num_misses': 2, 'num_switches': 0, 'num_fragmentations': 0, 'mota': 0.7, 'motp': 0.02981870229007634,
-    'num_transfer': 0, 'num_ascend': 0, 'num_migrate': 0}
 """
 
 
@@ -202,12 +144,12 @@ def calculate_from_payload(payload: dict, max_iou: float = 0.5, debug: bool = Fa
     for sequence in sequence_list:
         output[sequence] = {}
         frames = payload['sequences'][sequence][gt_field_name]
-        formatted_references = []
+        formated_references = []
         for frame_id, frame in enumerate(frames):
             for detection in frame:
                 id = detection['index']
                 x, y, w, h = detection['bounding_box']
-                formatted_references.append([frame_id+1, id, x, y, w, h])  
+                formated_references.append([frame_id+1, id, x, y, w, h])  
 
         for model in models:
             frames = payload['sequences'][sequence][model]
@@ -223,13 +165,13 @@ def calculate_from_payload(payload: dict, max_iou: float = 0.5, debug: bool = Fa
             if debug:
                 print("sequence/model: ", sequence, model)
                 print("formated_predictions: ", formated_predictions)
-                print("formated_references: ", formatted_references)
+                print("formated_references: ", formated_references)
             if len(formated_predictions) == 0:
                 output[sequence][model] = "Model had no predictions."
-            elif len(formatted_references) == 0:
+            elif len(formated_references) == 0:
                 output[sequence][model] = "No ground truth."
             else:
-                output[sequence][model] = calculate(formated_predictions, formatted_references, max_iou=max_iou)
+                output[sequence][model] = calculate(formated_predictions, formated_references, max_iou=max_iou)
     return output