Spaces:
Running
Running
Update Space (evaluate main: e179b5b8)
Browse files
README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1 |
---
|
2 |
title: SARI
|
3 |
-
emoji: 🤗
|
4 |
colorFrom: blue
|
5 |
colorTo: red
|
6 |
sdk: gradio
|
@@ -8,45 +8,10 @@ sdk_version: 3.0.2
|
|
8 |
app_file: app.py
|
9 |
pinned: false
|
10 |
tags:
|
11 |
-
|
12 |
-
|
13 |
-
description: >-
|
14 |
-
SARI is a metric used for evaluating automatic text simplification systems.
|
15 |
-
|
16 |
-
The metric compares the predicted simplified sentences against the reference
|
17 |
-
|
18 |
-
and the source sentences. It explicitly measures the goodness of words that
|
19 |
-
are
|
20 |
-
|
21 |
-
added, deleted and kept by the system.
|
22 |
-
|
23 |
-
Sari = (F1_add + F1_keep + P_del) / 3
|
24 |
-
|
25 |
-
where
|
26 |
-
|
27 |
-
F1_add: n-gram F1 score for add operation
|
28 |
-
|
29 |
-
F1_keep: n-gram F1 score for keep operation
|
30 |
-
|
31 |
-
P_del: n-gram precision score for delete operation
|
32 |
-
|
33 |
-
n = 4, as in the original paper.
|
34 |
-
|
35 |
-
|
36 |
-
This implementation is adapted from Tensorflow's tensor2tensor implementation
|
37 |
-
[3].
|
38 |
-
|
39 |
-
It has two differences with the original GitHub [1] implementation:
|
40 |
-
(1) Defines 0/0=1 instead of 0 to give higher scores for predictions that match
|
41 |
-
a target exactly.
|
42 |
-
(2) Fixes an alleged bug [2] in the keep score computation.
|
43 |
-
[1] https://github.com/cocoxu/simplification/blob/master/SARI.py
|
44 |
-
(commit 0210f15)
|
45 |
-
[2] https://github.com/cocoxu/simplification/issues/6
|
46 |
-
|
47 |
-
[3]
|
48 |
-
https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/utils/sari_hook.py
|
49 |
---
|
|
|
50 |
# Metric Card for SARI
|
51 |
|
52 |
|
|
|
1 |
---
|
2 |
title: SARI
|
3 |
+
emoji: 🤗
|
4 |
colorFrom: blue
|
5 |
colorTo: red
|
6 |
sdk: gradio
|
|
|
8 |
app_file: app.py
|
9 |
pinned: false
|
10 |
tags:
|
11 |
+
- evaluate
|
12 |
+
- metric
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
13 |
---
|
14 |
+
|
15 |
# Metric Card for SARI
|
16 |
|
17 |
|