Spaces:
Runtime error
Runtime error
Commit
·
01078a4
1
Parent(s):
3bde512
Update README.md
Browse files
README.md
CHANGED
@@ -23,7 +23,7 @@ This metric is used for evaluating the quality of relation extraction output. By
|
|
23 |
This metric can be used in relation extraction evaluation.
|
24 |
|
25 |
## How to Use
|
26 |
-
This metric takes
|
27 |
```python
|
28 |
import evaluate
|
29 |
metric_path = "Ikala-allen/relation_extraction"
|
@@ -48,28 +48,28 @@ evaluation_scores = module.compute(predictions=predictions, references=reference
|
|
48 |
- **references** (`list` of `list` of `dictionary`): a list of list of dictionary with every element's relation_type and their entity name
|
49 |
- **mode** (`str`): define strict or boundaries mode for evaluation, strict mode consider "head_type" and "tail_type", boundaries mode doesn't consider "head_type" and "tail_type"
|
50 |
- **only_all** (`bool`): True for only output ["ALL"] relation_type score. False for output every relation_type score, default True
|
51 |
-
- **relation_types** (`list`): define relation type that need to be evaluate, if not given, it will construct relation_types from ground truth, default []
|
52 |
|
53 |
### Output Values
|
54 |
|
55 |
**output** (`dictionary` of `dictionary`s) with multiple key-value pairs
|
56 |
-
- **
|
57 |
- **tp** : true positive count
|
58 |
- **fp** : false positive count
|
59 |
- **fn** : false negative count
|
60 |
- **p** : precision
|
61 |
- **r** : recall
|
62 |
- **f1** : micro f1 score
|
63 |
-
- **
|
|
|
|
|
|
|
64 |
- **tp** : true positive count
|
65 |
- **fp** : false positive count
|
66 |
- **fn** : false negative count
|
67 |
- **p** : precision
|
68 |
- **r** : recall
|
69 |
- **f1** : micro f1 score
|
70 |
-
- **Macro_f1** : macro f1 score
|
71 |
-
- **Macro_p** : macro precision
|
72 |
-
- **Macro_r** : macro recall
|
73 |
|
74 |
Output Example:
|
75 |
```python
|
|
|
23 |
This metric can be used in relation extraction evaluation.
|
24 |
|
25 |
## How to Use
|
26 |
+
This metric takes 3 inputs, prediction, references(ground truth) and mode.Predictions and references are a list of list of dictionary of entity's name and entity's type. And mode define the evaluation type:
|
27 |
```python
|
28 |
import evaluate
|
29 |
metric_path = "Ikala-allen/relation_extraction"
|
|
|
48 |
- **references** (`list` of `list` of `dictionary`): a list of list of dictionary with every element's relation_type and their entity name
|
49 |
- **mode** (`str`): define strict or boundaries mode for evaluation, strict mode consider "head_type" and "tail_type", boundaries mode doesn't consider "head_type" and "tail_type"
|
50 |
- **only_all** (`bool`): True for only output ["ALL"] relation_type score. False for output every relation_type score, default True
|
51 |
+
- **relation_types** (`list`): define which relation type that need to be evaluate and show, if not given, it will construct relation_types from ground truth, default []
|
52 |
|
53 |
### Output Values
|
54 |
|
55 |
**output** (`dictionary` of `dictionary`s) with multiple key-value pairs
|
56 |
+
- **ALL** (`dictionary`): score of all of the relation type
|
57 |
- **tp** : true positive count
|
58 |
- **fp** : false positive count
|
59 |
- **fn** : false negative count
|
60 |
- **p** : precision
|
61 |
- **r** : recall
|
62 |
- **f1** : micro f1 score
|
63 |
+
- **Macro_f1** : macro f1 score
|
64 |
+
- **Macro_p** : macro precision
|
65 |
+
- **Macro_r** : macro recall
|
66 |
+
- **{selected relation type}** (`dictionary`): score of selected relation type
|
67 |
- **tp** : true positive count
|
68 |
- **fp** : false positive count
|
69 |
- **fn** : false negative count
|
70 |
- **p** : precision
|
71 |
- **r** : recall
|
72 |
- **f1** : micro f1 score
|
|
|
|
|
|
|
73 |
|
74 |
Output Example:
|
75 |
```python
|