Spaces:
Running
Running
Add description to card metadata (#1)
Browse files- Add description to card metadata (82fd13c2f2d3c6013f33cd94d0580f62802dd3da)
README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1 |
---
|
2 |
title: SARI
|
3 |
-
emoji: 🤗
|
4 |
colorFrom: blue
|
5 |
colorTo: red
|
6 |
sdk: gradio
|
@@ -8,10 +8,45 @@ sdk_version: 3.0.2
|
|
8 |
app_file: app.py
|
9 |
pinned: false
|
10 |
tags:
|
11 |
-
- evaluate
|
12 |
-
- metric
|
13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
14 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
15 |
# Metric Card for SARI
|
16 |
|
17 |
|
|
|
1 |
---
|
2 |
title: SARI
|
3 |
+
emoji: 🤗
|
4 |
colorFrom: blue
|
5 |
colorTo: red
|
6 |
sdk: gradio
|
|
|
8 |
app_file: app.py
|
9 |
pinned: false
|
10 |
tags:
|
11 |
+
- evaluate
|
12 |
+
- metric
|
13 |
+
description: >-
|
14 |
+
SARI is a metric used for evaluating automatic text simplification systems.
|
15 |
+
|
16 |
+
The metric compares the predicted simplified sentences against the reference
|
17 |
+
|
18 |
+
and the source sentences. It explicitly measures the goodness of words that
|
19 |
+
are
|
20 |
+
|
21 |
+
added, deleted and kept by the system.
|
22 |
+
|
23 |
+
Sari = (F1_add + F1_keep + P_del) / 3
|
24 |
+
|
25 |
+
where
|
26 |
+
|
27 |
+
F1_add: n-gram F1 score for add operation
|
28 |
|
29 |
+
F1_keep: n-gram F1 score for keep operation
|
30 |
+
|
31 |
+
P_del: n-gram precision score for delete operation
|
32 |
+
|
33 |
+
n = 4, as in the original paper.
|
34 |
+
|
35 |
+
|
36 |
+
This implementation is adapted from Tensorflow's tensor2tensor implementation
|
37 |
+
[3].
|
38 |
+
|
39 |
+
It has two differences with the original GitHub [1] implementation:
|
40 |
+
(1) Defines 0/0=1 instead of 0 to give higher scores for predictions that match
|
41 |
+
a target exactly.
|
42 |
+
(2) Fixes an alleged bug [2] in the keep score computation.
|
43 |
+
[1] https://github.com/cocoxu/simplification/blob/master/SARI.py
|
44 |
+
(commit 0210f15)
|
45 |
+
[2] https://github.com/cocoxu/simplification/issues/6
|
46 |
+
|
47 |
+
[3]
|
48 |
+
https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/utils/sari_hook.py
|
49 |
+
---
|
50 |
# Metric Card for SARI
|
51 |
|
52 |
|