Zhuoyang Song commited on
Commit
5bcd8e0
2 Parent(s): f61328a bf448da

Merge branch 'main' of https://huggingface.co/IDEA-CCNL/Randeng-TransformerXL-5B-Deduction-Chinese into main

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -34,7 +34,7 @@ Chinese deductive reasoning model based on Transformer-XL.
34
  **训练流程 Model Training**
35
  1. 在悟道语料库(280G版本)上进行预训练
36
  2. 在1.5M因果语料上进行因果生成任务的训练
37
- 3. 基于其余0.8M因果语料,协同[Randeng-TransformerXL-5B-Abduction-Chinese](https://huggingface.co/IDEA-CCNL/Randeng-TransformerXL-5B-Abduction-Chinese)和[Erlangshen-Roberta-330M-Causal-Chinese](https://huggingface.co/IDEA-CCNL/Erlangshen-Roberta-330M-Causal-Chinese)进行Self-consistency闭环迭代训练
38
  * 两个生成模型基于核采样和贪心的方式进行因果推理和反绎推理,产生大量伪样本;
39
  * Erlangshen-Roberta-330M-Causal-Chinese模型对伪样本句子对的因果关系进行打分,筛选供自身以及生成模型训练的样本
40
 
 
34
  **训练流程 Model Training**
35
  1. 在悟道语料库(280G版本)上进行预训练
36
  2. 在1.5M因果语料上进行因果生成任务的训练
37
+ 3. 基于其余0.8M因果语料,协同[Randeng-TransformerXL-5B-Abduction-Chinese](https://huggingface.co/IDEA-CCNL/Randeng-TransformerXL-5B-Abduction-Chinese)和[Erlangshen-Roberta-330M-Causal-Chinese](https://huggingface.co/IDEA-CCNL/Erlangshen-Roberta-330M-Causal-Chinese)进行Self-consistent闭环迭代训练
38
  * 两个生成模型基于核采样和贪心的方式进行因果推理和反绎推理,产生大量伪样本;
39
  * Erlangshen-Roberta-330M-Causal-Chinese模型对伪样本句子对的因果关系进行打分,筛选供自身以及生成模型训练的样本
40