Update README.md
Browse files
README.md
CHANGED
@@ -17,7 +17,7 @@ datasets:
|
|
17 |
|Model|CoLA|SST|MRPC|STS|QQP|MNLI|QNLI|RTE|Avg.|
|
18 |
|---|---|---|---|---|---|---|---|---|---|
|
19 |
|ELECTRA-Small-OWT(original)|56.8|88.3|87.4|86.8|88.3|78.9|87.9|68.5|80.36|
|
20 |
-
|**ELECTRA-Small-OWT (this)**|
|
21 |
|
22 |
# 三、 训练细节
|
23 |
- 数据集 openwebtext
|
@@ -29,7 +29,10 @@ datasets:
|
|
29 |
- 训练时间总共耗费55h
|
30 |
|
31 |
### wandb日志
|
|
|
32 |
https://wandb.ai/junyu/electra_rotary_small_pretrain?workspace=user-junyu
|
|
|
|
|
33 |
|
34 |
# 四、安装
|
35 |
```bash
|
|
|
17 |
|Model|CoLA|SST|MRPC|STS|QQP|MNLI|QNLI|RTE|Avg.|
|
18 |
|---|---|---|---|---|---|---|---|---|---|
|
19 |
|ELECTRA-Small-OWT(original)|56.8|88.3|87.4|86.8|88.3|78.9|87.9|68.5|80.36|
|
20 |
+
|**ELECTRA-RoFormer-Small-OWT (this)**|55.76|90.45|87.3|86.64|89.61|81.17|88.85|62.71|80.31|
|
21 |
|
22 |
# 三、 训练细节
|
23 |
- 数据集 openwebtext
|
|
|
29 |
- 训练时间总共耗费55h
|
30 |
|
31 |
### wandb日志
|
32 |
+
预训练日志
|
33 |
https://wandb.ai/junyu/electra_rotary_small_pretrain?workspace=user-junyu
|
34 |
+
GLUE微调日志
|
35 |
+
https://wandb.ai/junyu/electra_rotary_glue_100?workspace=user-junyu
|
36 |
|
37 |
# 四、安装
|
38 |
```bash
|