File size: 668 Bytes
de77cca 483ef72 de77cca 7d77839 20027aa de77cca 20027aa 0b6b60c 18579f8 e8a0b5b c3e4ed9 483ef72 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
---
datasets:
- wikitext-2-v1
- wikitext
language:
- en
metrics:
- perplexity
- cross_entropy
---
**metrics on 1024 context**:
- valid_perplexity = 14.79
- valid_cross_entropy = 2.69
- train_perplexity = 13.77
- train_cross_entropy = 2.62
**metrics on 252 context**:
- valid_perplexity = 17.35
**metrics on 378 context**:
- valid_perplexity = 16.4
**metrics on 504 context**:
- valid_perplexity = 15.86
**Dependence of the cross entropy loss on the length of the context for prediction**
- x-axis*128 = context length
- y-axis = cross entropy
 |