|
--- |
|
title: README |
|
emoji: 👀 |
|
colorFrom: gray |
|
colorTo: gray |
|
sdk: static |
|
pinned: false |
|
--- |
|
|
|
|
|
<div align="center"> |
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/6567e9af88bfbc261a34673d/riHfcszflP6BzNls2Phrf.png" |
|
width=400px |
|
style="border-radius: 20px;" |
|
/> |
|
|
|
</div> |
|
|
|
<div align="center"> |
|
<h1> Want to contribute? </h1> |
|
</div> |
|
|
|
- 🥙 For naming models, spaces, datasets, and metrics, employ [kebab-case](https://en.wiktionary.org/wiki/kebab_case). Use lowercase letters, except for acronyms, which may be capitalized. |
|
- 🔒 Please mind about making the models, spaces and datasets public or private. Metrics can (and have to) be public. |
|
- 🤐 Do not expose passwords or tokens, use [secrets](https://huggingface.co/docs/hub/spaces-overview#managing-secrets). |
|
|
|
<div align="center"> |
|
<h1> Looking for metrics? </h1> |
|
</div> |
|
|
|
- https://huggingface.co/spaces/SEA-AI/det-metrics |
|
- Object detection metrics based on [`pycocotools`](https://github.com/cocodataset/cocoapi) and [torchmetrics' Mean Avergae Precision](https://lightning.ai/docs/torchmetrics/stable/detection/mean_average_precision.html). |
|
- https://huggingface.co/spaces/SEA-AI/box-metrics |
|
- Bounding box statistics, including IOU, BEP (bottom edge proximity), and others. |
|
- https://huggingface.co/spaces/SEA-AI/horizon-metrics |
|
- Comparing horizons in an image w.r.t their midpoint and slope errors |
|
- https://huggingface.co/spaces/SEA-AI/mot-metrics |
|
- Multi-object-tracking metrics using [`py-motmetrics`](https://github.com/cheind/py-motmetrics) |
|
- https://huggingface.co/spaces/SEA-AI/panoptic-quality |
|
- Evaluating panoptic models |
|
|
|
<details> |
|
<summary>Proposed Metric Output Structure</summary> |
|
|
|
Your metric should have the following structure: |
|
- `_compute(references, predictions)` |
|
- Calls a metric engine, defined for example in the `seametrics` package, or other |
|
- `compute_from_payload(paylaod)` |
|
- Call the `module.compute` method internally after converting payload -> references, predictions |
|
- All the metric's parameters, such as `iou_threshold`, `area_ranges`, etc.. should be moved to the `__init__` method. |
|
- Output should look like this: |
|
```json |
|
{ |
|
"ahoy_IR_b2_engine_3_6_0_49_gd81d3b63_oversea": { |
|
"overall": { |
|
"all": { |
|
"f1": 0.15967351103175614, |
|
"fn": 2923.0, |
|
"fp": 3666.0, |
|
"num_gt_ids": 10, |
|
"precision": 0.14585274930102515, |
|
"recall": 0.1763877148492533, |
|
"recognition_0.3": 0.1, |
|
"recognition_0.5": 0.1, |
|
"recognition_0.8": 0.1, |
|
"recognized_0.3": 1, |
|
"recognized_0.5": 1, |
|
"recognized_0.8": 1, |
|
"tp": 626.0 |
|
} |
|
}, |
|
"per_sequence": { |
|
"Sentry_2023_02_08_PROACT_CELADON_@6m_MOB_2023_02_08_12_51_49": { |
|
"all": { |
|
"f1": 0.15967351103175614, |
|
"fn": 2923.0, |
|
"fp": 3666.0, |
|
"num_gt_ids": 10, |
|
"precision": 0.14585274930102515, |
|
"recall": 0.1763877148492533, |
|
"recognition_0.3": 0.1, |
|
"recognition_0.5": 0.1, |
|
"recognition_0.8": 0.1, |
|
"recognized_0.3": 1, |
|
"recognized_0.5": 1, |
|
"recognized_0.8": 1, |
|
"tp": 626.0 |
|
} |
|
} |
|
} |
|
} |
|
} |
|
``` |
|
|
|
</details> |
