Merge branch 'main' of https://huggingface.co/IDEA-CCNL/Randeng-TransformerXL-5B-Deduction-Chinese into main
Browse files
README.md
CHANGED
@@ -34,7 +34,7 @@ Chinese deductive reasoning model based on Transformer-XL.
|
|
34 |
**训练流程 Model Training**
|
35 |
1. 在悟道语料库(280G版本)上进行预训练
|
36 |
2. 在1.5M因果语料上进行因果生成任务的训练
|
37 |
-
3. 基于其余0.8M因果语料,协同[Randeng-TransformerXL-5B-Abduction-Chinese](https://huggingface.co/IDEA-CCNL/Randeng-TransformerXL-5B-Abduction-Chinese)和[Erlangshen-Roberta-330M-Causal-Chinese](https://huggingface.co/IDEA-CCNL/Erlangshen-Roberta-330M-Causal-Chinese)进行Self-
|
38 |
* 两个生成模型基于核采样和贪心的方式进行因果推理和反绎推理,产生大量伪样本;
|
39 |
* Erlangshen-Roberta-330M-Causal-Chinese模型对伪样本句子对的因果关系进行打分,筛选供自身以及生成模型训练的样本
|
40 |
|
|
|
34 |
**训练流程 Model Training**
|
35 |
1. 在悟道语料库(280G版本)上进行预训练
|
36 |
2. 在1.5M因果语料上进行因果生成任务的训练
|
37 |
+
3. 基于其余0.8M因果语料,协同[Randeng-TransformerXL-5B-Abduction-Chinese](https://huggingface.co/IDEA-CCNL/Randeng-TransformerXL-5B-Abduction-Chinese)和[Erlangshen-Roberta-330M-Causal-Chinese](https://huggingface.co/IDEA-CCNL/Erlangshen-Roberta-330M-Causal-Chinese)进行Self-consistent闭环迭代训练
|
38 |
* 两个生成模型基于核采样和贪心的方式进行因果推理和反绎推理,产生大量伪样本;
|
39 |
* Erlangshen-Roberta-330M-Causal-Chinese模型对伪样本句子对的因果关系进行打分,筛选供自身以及生成模型训练的样本
|
40 |
|