|
--- |
|
title: Systematic Error Analysis and Labeling |
|
emoji: 🦭 |
|
colorFrom: yellow |
|
colorTo: pink |
|
sdk: streamlit |
|
sdk_version: 1.10.0 |
|
app_file: app.py |
|
pinned: false |
|
license: apache-2.0 |
|
--- |
|
# SEAL |
|
Systematic Error Analysis and Labeling (SEAL) is an interactive tool for discovering systematic errors in NLP models via clustering on high-loss example groups and semantic labeling for interpretability of those error-groups. It supports fine-grained analytical visualization for interactively zooming into potential systematic bugs and features for crafting prompts to label those bugs semantically. |
|
|
|
🎥 [Demo screencast](https://vimeo.com/736659216) |
|
|
|
<p> |
|
<img src="./assets/website/seal.gif" alt="Demo gif"/> |
|
</p> |
|
|
|
## Table of Contents |
|
- [Installation](#installation) |
|
- [Quickstart](#quickstart) |
|
- [Running Locally](#running-locally) |
|
- [Citation](#citation) |
|
|
|
## Installation |
|
Please use python>=3.8 since some dependencies require that for installation. |
|
```shell |
|
git clone https://huggingface.co/spaces/nazneen/seal |
|
cd seal |
|
pip install --upgrade pip |
|
pip install -r requirements.txt |
|
``` |
|
|
|
## Quickstart |
|
``` |
|
streamlit run app.py |
|
``` |
|
|
|
## Running Locally |
|
|
|
To run seal on any text classification model and dataset, please use the notebooks provided in `./assets/notebooks/` and replace the model and datasets with any HF datasets and model on the hub https://huggingface.co/models. |
|
|
|
If you need to run inference on a dataset, please run `./util/run_inference.py` with the appropriate HF model and dataset. You can also use the same script to select the model's layer for extracting the representation of the input examples. |
|
|
|
## Citation |
|
|
