GPT-3.5 Spider contamination based on https://arxiv.org/pdf/2402.08100 (#18)
Browse files- GPT-3.5 Spider contamination based on https://arxiv.org/pdf/2402.08100 (9588afb347cd07a7645852f6c9b966c5222aa76e)
- convert arxiv pdf link to abs (f6e12a010342efd037641bd2cb947fca4b1a5374)
- Add percentage of validation data contaminated by including db ids on which gpt-3.5 achieves more than 75% DC-accuracy (2a68192028574cf246c6af3b1d053cae7fd82acc)
- Add PR number (326e3ca343d14319e6f0c0930fdd06b7507aaacb)
Co-authored-by: Bhavish Pahwa <[email protected]>
- contamination_report.csv +2 -0
contamination_report.csv
CHANGED
@@ -664,6 +664,8 @@ wmt/wmt16;fr-en;GPT-3;;model;;;14.0;data-based;https://arxiv.org/abs/2005.14165;
|
|
664 |
wmt/wmt16;ro-en;FLAN;;model;;;12.4;data-based;https://arxiv.org/abs/2109.01652;13
|
665 |
wmt/wmt16;ro-en;GPT-3;;model;;;21.0;data-based;https://arxiv.org/abs/2005.14165;13
|
666 |
|
|
|
|
|
667 |
xnli;en;EleutherAI/pile;;corpus;;;0.36;data-based;https://arxiv.org/abs/2310.20707;2
|
668 |
xnli;en;allenai/c4;;corpus;;;0.12;data-based;https://arxiv.org/abs/2310.20707;2
|
669 |
xnli;en;oscar-corpus/OSCAR-2301;;corpus;;;0.24;data-based;https://arxiv.org/abs/2310.20707;2
|
|
|
664 |
wmt/wmt16;ro-en;FLAN;;model;;;12.4;data-based;https://arxiv.org/abs/2109.01652;13
|
665 |
wmt/wmt16;ro-en;GPT-3;;model;;;21.0;data-based;https://arxiv.org/abs/2005.14165;13
|
666 |
|
667 |
+
xlangai/spider;;GPT-3.5;;model;;11.3;;model-based;https://arxiv.org/abs/2402.08100;18
|
668 |
+
|
669 |
xnli;en;EleutherAI/pile;;corpus;;;0.36;data-based;https://arxiv.org/abs/2310.20707;2
|
670 |
xnli;en;allenai/c4;;corpus;;;0.12;data-based;https://arxiv.org/abs/2310.20707;2
|
671 |
xnli;en;oscar-corpus/OSCAR-2301;;corpus;;;0.24;data-based;https://arxiv.org/abs/2310.20707;2
|