Update README.md
Browse files
README.md
CHANGED
@@ -22,7 +22,7 @@ As You may notice, the dataset mostly consists of specially modified winogrande
|
|
22 |
|
23 |
But before flagging this (or recommending this to be flagged), consider this:
|
24 |
|
25 |
-
Subtle DPO-Contamination with modified Winogrande
|
26 |
|
27 |
| Model | ARC | HellaSwag | MMLU | Truthful QA | GSM8K | Average |
|
28 |
| -----------------------------|------ | --------- | ---- | ----------- | ------| ------- |
|
|
|
22 |
|
23 |
But before flagging this (or recommending this to be flagged), consider this:
|
24 |
|
25 |
+
Subtle DPO-Contamination with modified Winogrande causes the average accuracy of all 5-non Winogrande metrics (e.g. including also MMLU and GSM8K) to be 0.2% higher than the underlying model.
|
26 |
|
27 |
| Model | ARC | HellaSwag | MMLU | Truthful QA | GSM8K | Average |
|
28 |
| -----------------------------|------ | --------- | ---- | ----------- | ------| ------- |
|