Hugging Face's logo

junnyu
/
roformer_small_discriminator

Feature Extraction
Transformers
PyTorch
English
roformer
electra
rotary position embedding
Inference Endpoints
Model card Files Community
2
junnyu commited on
Commit
7f15989
•
1 Parent(s): c54ef05

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -1
README.md CHANGED
@@ -17,7 +17,7 @@ datasets:
17
  |Model|CoLA|SST|MRPC|STS|QQP|MNLI|QNLI|RTE|Avg.|
18
  |---|---|---|---|---|---|---|---|---|---|
19
  |ELECTRA-Small-OWT(original)|56.8|88.3|87.4|86.8|88.3|78.9|87.9|68.5|80.36|
20
- |**ELECTRA-Small-OWT (this)**| xx|xx|xx|xx|xx|xx|xx|xx|xx|
21
 
22
  # 三、 训练细节
23
  - 数据集 openwebtext
@@ -29,7 +29,10 @@ datasets:
29
  - 训练时间总共耗费55h
30
 
31
  ### wandb日志
 
32
  https://wandb.ai/junyu/electra_rotary_small_pretrain?workspace=user-junyu
 
 
33
 
34
  # 四、安装
35
  ```bash
 
17
  |Model|CoLA|SST|MRPC|STS|QQP|MNLI|QNLI|RTE|Avg.|
18
  |---|---|---|---|---|---|---|---|---|---|
19
  |ELECTRA-Small-OWT(original)|56.8|88.3|87.4|86.8|88.3|78.9|87.9|68.5|80.36|
20
+ |**ELECTRA-RoFormer-Small-OWT (this)**|55.76|90.45|87.3|86.64|89.61|81.17|88.85|62.71|80.31|
21
 
22
  # 三、 训练细节
23
  - 数据集 openwebtext
 
29
  - 训练时间总共耗费55h
30
 
31
  ### wandb日志
32
+ 预训练日志
33
  https://wandb.ai/junyu/electra_rotary_small_pretrain?workspace=user-junyu
34
+ GLUE微调日志
35
+ https://wandb.ai/junyu/electra_rotary_glue_100?workspace=user-junyu
36
 
37
  # 四、安装
38
  ```bash