README.md

metadata

title: README
emoji: 👀
colorFrom: gray
colorTo: gray
sdk: static
pinned: false

Want to contribute?

• 🥙 For naming models, spaces, datasets, and metrics, employ kebab-case. Use lowercase letters, except for acronyms, which may be capitalized.
• 🔒 Please mind about making the models, spaces and datasets public or private. Metrics can (and have to) be public.
• 🤐 Do not expose passwords or tokens, use secrets.

Looking for metrics?

• https://huggingface.co/spaces/SEA-AI/det-metrics
  • Object detection metrics based on pycocotools and torchmetrics' Mean Avergae Precision.
• https://huggingface.co/spaces/SEA-AI/box-metrics
  • Bounding box statistics, including IOU, BEP (bottom edge proximity), and others.
• https://huggingface.co/spaces/SEA-AI/horizon-metrics
  • Comparing horizons in an image w.r.t their midpoint and slope errors
• https://huggingface.co/spaces/SEA-AI/mot-metrics
  • Multi-object-tracking metrics using py-motmetrics
• https://huggingface.co/spaces/SEA-AI/panoptic-quality
  • Evaluating panoptic models

Proposed Metric Output Structure

Your metric should have the following structure:

• _compute(references, predictions)
  • Calls a metric engine, defined for example in the seametrics package, or other
• compute_from_payload(paylaod)
  • Call the module.compute method internally after converting payload -> references, predictions
• All the metric's parameters, such as iou_threshold, area_ranges, etc.. should be moved to the __init__ method.
• Output should look like this:

{
    "ahoy_IR_b2_engine_3_6_0_49_gd81d3b63_oversea": {
        "overall": {
            "all": {
                "f1": 0.15967351103175614,
                "fn": 2923.0,
                "fp": 3666.0,
                "num_gt_ids": 10,
                "precision": 0.14585274930102515,
                "recall": 0.1763877148492533,
                "recognition_0.3": 0.1,
                "recognition_0.5": 0.1,
                "recognition_0.8": 0.1,
                "recognized_0.3": 1,
                "recognized_0.5": 1,
                "recognized_0.8": 1,
                "tp": 626.0
            }
        },
        "per_sequence": {
            "Sentry_2023_02_08_PROACT_CELADON_@6m_MOB_2023_02_08_12_51_49": {
                "all": {
                    "f1": 0.15967351103175614,
                    "fn": 2923.0,
                    "fp": 3666.0,
                    "num_gt_ids": 10,
                    "precision": 0.14585274930102515,
                    "recall": 0.1763877148492533,
                    "recognition_0.3": 0.1,
                    "recognition_0.5": 0.1,
                    "recognition_0.8": 0.1,
                    "recognized_0.3": 1,
                    "recognized_0.5": 1,
                    "recognized_0.8": 1,
                    "tp": 626.0
                }
            }
        }
    }
}