distinct
Browse files
README.md
CHANGED
@@ -25,7 +25,7 @@ Distinct metric is to calculate the diversity of language. We provide two versio
|
|
25 |
|
26 |
For the use of Expectation-Adjusted-Distinct, vocab_size is required.
|
27 |
|
28 |
-
Please follow
|
29 |
|
30 |
This metric is used to calculate the diversity of a group of sentences. It can be used to either evaluate the diversity of generated responses of the testset (i.e., corpus-level diversity), or calculate diversity of a group of sampled responses given one context (i.e., utterence-level diversity). The [original paper](https://aclanthology.org/N16-1014) (Li et al. 2022) used it as corpus-level while some may use it as utterance-level. However, we don't recommend to calculate Distinct on a small group as it is sensitive to the sentence length and number.
|
31 |
|
|
|
25 |
|
26 |
For the use of Expectation-Adjusted-Distinct, vocab_size is required.
|
27 |
|
28 |
+
Please follow the [EAD paper](https://aclanthology.org/2022.acl-short.86) (Liu and Sabour et al. 2022) for motivation and follow the rules of thumb provided by [the ipynb](https://github.com/lsy641/Expectation-Adjusted-Distinct/blob/main/EAD.ipynb) to determine the vocab_size.
|
29 |
|
30 |
This metric is used to calculate the diversity of a group of sentences. It can be used to either evaluate the diversity of generated responses of the testset (i.e., corpus-level diversity), or calculate diversity of a group of sampled responses given one context (i.e., utterence-level diversity). The [original paper](https://aclanthology.org/N16-1014) (Li et al. 2022) used it as corpus-level while some may use it as utterance-level. However, we don't recommend to calculate Distinct on a small group as it is sensitive to the sentence length and number.
|
31 |
|