Spaces:
Running
Running
Add description to card metadata
#1
by
julien-c
HF Staff
- opened
README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
---
|
| 2 |
title: SARI
|
| 3 |
-
emoji: 🤗
|
| 4 |
colorFrom: blue
|
| 5 |
colorTo: red
|
| 6 |
sdk: gradio
|
|
@@ -8,10 +8,45 @@ sdk_version: 3.0.2
|
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
| 10 |
tags:
|
| 11 |
-
- evaluate
|
| 12 |
-
- metric
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
# Metric Card for SARI
|
| 16 |
|
| 17 |
|
|
|
|
| 1 |
---
|
| 2 |
title: SARI
|
| 3 |
+
emoji: 🤗
|
| 4 |
colorFrom: blue
|
| 5 |
colorTo: red
|
| 6 |
sdk: gradio
|
|
|
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
| 10 |
tags:
|
| 11 |
+
- evaluate
|
| 12 |
+
- metric
|
| 13 |
+
description: >-
|
| 14 |
+
SARI is a metric used for evaluating automatic text simplification systems.
|
| 15 |
+
|
| 16 |
+
The metric compares the predicted simplified sentences against the reference
|
| 17 |
+
|
| 18 |
+
and the source sentences. It explicitly measures the goodness of words that
|
| 19 |
+
are
|
| 20 |
+
|
| 21 |
+
added, deleted and kept by the system.
|
| 22 |
+
|
| 23 |
+
Sari = (F1_add + F1_keep + P_del) / 3
|
| 24 |
+
|
| 25 |
+
where
|
| 26 |
+
|
| 27 |
+
F1_add: n-gram F1 score for add operation
|
| 28 |
|
| 29 |
+
F1_keep: n-gram F1 score for keep operation
|
| 30 |
+
|
| 31 |
+
P_del: n-gram precision score for delete operation
|
| 32 |
+
|
| 33 |
+
n = 4, as in the original paper.
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
This implementation is adapted from Tensorflow's tensor2tensor implementation
|
| 37 |
+
[3].
|
| 38 |
+
|
| 39 |
+
It has two differences with the original GitHub [1] implementation:
|
| 40 |
+
(1) Defines 0/0=1 instead of 0 to give higher scores for predictions that match
|
| 41 |
+
a target exactly.
|
| 42 |
+
(2) Fixes an alleged bug [2] in the keep score computation.
|
| 43 |
+
[1] https://github.com/cocoxu/simplification/blob/master/SARI.py
|
| 44 |
+
(commit 0210f15)
|
| 45 |
+
[2] https://github.com/cocoxu/simplification/issues/6
|
| 46 |
+
|
| 47 |
+
[3]
|
| 48 |
+
https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/utils/sari_hook.py
|
| 49 |
+
---
|
| 50 |
# Metric Card for SARI
|
| 51 |
|
| 52 |
|