| ====== Perplexity statistics ====== | |
| Mean PPL(Q) : 8.895382 ± 0.058817 | |
| Mean PPL(base) : 7.534124 ± 0.048206 | |
| Cor(ln(PPL(Q)), ln(PPL(base))): 97.35% | |
| Mean ln(PPL(Q)/PPL(base)) : 0.166090 ± 0.001513 | |
| Mean PPL(Q)/PPL(base) : 1.180679 ± 0.001787 | |
| Mean PPL(Q)-PPL(base) : 1.361258 ± 0.016217 | |
| ====== KL divergence statistics ====== | |
| Mean KLD: 0.146296 ± 0.000446 | |
| Maximum KLD: 8.539277 | |
| 99.9% KLD: 1.800523 | |
| 99.0% KLD: 0.699970 | |
| 99.0% KLD: 0.699970 | |
| Median KLD: 0.118646 | |
| 10.0% KLD: 0.006209 | |
| 5.0% KLD: 0.001891 | |
| 1.0% KLD: 0.000254 | |
| Minimum KLD: -0.000001 | |
| ====== Token probability statistics ====== | |
| Mean Δp: -2.899 ± 0.028 % | |
| Maximum Δp: 77.074% | |
| 99.9% Δp: 39.937% | |
| 99.0% Δp: 24.573% | |
| 95.0% Δp: 11.867% | |
| 90.0% Δp: 6.258% | |
| 75.0% Δp: 0.461% | |
| Median Δp: -0.451% | |
| 25.0% Δp: -5.618% | |
| 10.0% Δp: -15.734% | |
| 5.0% Δp: -23.078% | |
| 1.0% Δp: -39.433% | |
| 0.1% Δp: -67.421% | |
| Minimum Δp: -97.164% | |
| RMS Δp : 11.179 ± 0.043 % | |
| Same top p: 78.943 ± 0.108 % | |