Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,60 @@
|
|
1 |
---
|
2 |
license: mit
|
|
|
|
|
|
|
|
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: mit
|
3 |
+
datasets:
|
4 |
+
- web_nlg
|
5 |
+
language:
|
6 |
+
- en
|
7 |
---
|
8 |
+
# Model card for Inria-CEDAR/FactSpotter-DeBERTaV3-Base
|
9 |
+
|
10 |
+
## Model description
|
11 |
+
|
12 |
+
This model is related to the paper **"FactSpotter: Evaluating the Factual Faithfulness of Graph-to-Text Generation"**.
|
13 |
+
|
14 |
+
Given a triple of format "subject | predicate | object" and a text, the model determines if the triple is present in the text.
|
15 |
+
|
16 |
+
Different from the paper using ELECTRA, this model is finetuned on DeBERTaV3.
|
17 |
+
|
18 |
+
## How to use the model
|
19 |
+
|
20 |
+
```
|
21 |
+
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
22 |
+
def sentence_cls_score(input_strings, predicate_cls_model, predicate_cls_tokenizer):
|
23 |
+
tokenized_cls_input = predicate_cls_tokenizer(input_strings, truncation=True, padding=True,
|
24 |
+
return_token_type_ids=True)
|
25 |
+
input_ids = torch.Tensor(tokenized_cls_input['input_ids']).long().to(torch.device("cuda"))
|
26 |
+
token_type_ids = torch.Tensor(tokenized_cls_input['token_type_ids']).long().to(torch.device("cuda"))
|
27 |
+
attention_mask = torch.Tensor(tokenized_cls_input['attention_mask']).long().to(torch.device("cuda"))
|
28 |
+
prev_cls_output = predicate_cls_model(input_ids, attention_mask=attention_mask, token_type_ids=token_type_ids)
|
29 |
+
softmax_cls_output = torch.softmax(prev_cls_output.logits, dim=1, )
|
30 |
+
return softmax_cls_output
|
31 |
+
tokenizer = AutoTokenizer.from_pretrained("Inria-CEDAR/FactSpotter-DeBERTaV3-Base")
|
32 |
+
model = AutoModelForSequenceClassification.from_pretrained("Inria-CEDAR/FactSpotter-DeBERTaV3-Base")
|
33 |
+
# pairs of texts (as premises) and triples (as hypotheses)
|
34 |
+
cls_texts = [("the aarhus is the airport of aarhus, denmark", "aarhus airport | city served | aarhus, denmark"),
|
35 |
+
("aarhus airport is 25.0 metres above the sea level", "aarhus airport | elevation above the sea level | 1174")]
|
36 |
+
cls_scores = sentence_cls_score(cls_texts, model, tokenizer)
|
37 |
+
# Dimensions: 0-entailment, 1-neutral, 2-contradiction
|
38 |
+
label_names = ["entailment", "neutral", "contradiction"]
|
39 |
+
```
|
40 |
+
## Citation
|
41 |
+
If the model is useful to you, please cite the paper
|
42 |
+
|
43 |
+
```
|
44 |
+
@inproceedings{zhang:hal-04257838,
|
45 |
+
TITLE = {{FactSpotter: Evaluating the Factual Faithfulness of Graph-to-Text Generation}},
|
46 |
+
AUTHOR = {Zhang, Kun and Balalau, Oana and Manolescu, Ioana},
|
47 |
+
URL = {https://hal.science/hal-04257838},
|
48 |
+
BOOKTITLE = {{Findings of EMNLP 2023 - Conference on Empirical Methods in Natural Language Processing}},
|
49 |
+
ADDRESS = {Singapore, Singapore},
|
50 |
+
YEAR = {2023},
|
51 |
+
MONTH = Dec,
|
52 |
+
KEYWORDS = {Graph-to-Text Generation ; Factual Faithfulness ; Constrained Text Generation},
|
53 |
+
PDF = {https://hal.science/hal-04257838/file/_EMNLP_2023__Evaluating_the_Factual_Faithfulness_of_Graph_to_Text_Generation_Camera.pdf},
|
54 |
+
HAL_ID = {hal-04257838},
|
55 |
+
HAL_VERSION = {v1},
|
56 |
+
}
|
57 |
+
```
|
58 |
+
|
59 |
+
## Questions
|
60 |
+
If you have some questions, please contact through my email [email protected] or [email protected]
|