| Model,Year,Encoder,CoLA,SST-2,MRPC,STS-B,QQP,MNLI,MNLI-mm,QNLI,RTE,WNLI,AX | |
| ERNIE,2019,Transformer,75.5,97.8,93.9,93,75.2,92.3,91.7,97.3,92.6,95.9,51.7 | |
| T5,2019,Transformer,71.6,97.5,92.8,93.1,75.1,92.2,91.9,96.9,92.8,94.5,53.1 | |
| RoBERTa,2019,Transformer,67.8,96.7,92.3,92.2,74.3,90.8,90.2,95.4,88.2,89,48.7 | |
| BERT,2018,Transformer,60.5,94.9,89.3,87.6,72.1,86.7,85.9,92.7,70.1,65.1,39.5 | |
| BiLSTM+ELMo,2018,LSTM,32.1,89.3,84.7,70.3,61.1,67.2,67.9,75.5,57.4,65.1,21.3 | |
| BiLSTM+CoVe,2017,LSTM,18.5,81.9,78.7,64.4,60.6,65.4,65.7,70.8,52.7,65.1,17.6 | |
| BiLSTM+Attn,2017,LSTM,18.6,83,83.9,72.8,60.1,67.6,68.3,74.3,58.4,65.1,17.8 | |
| BiLSTM,2017,LSTM,11.6,82.8,81.8,70.3,62.5,65.6,66.1,74.6,57.4,65.1,20.3 |