Zhuoyang Song commited on
Commit
6df700e
1 Parent(s): 2b5eb6a

edit model card

Browse files
Files changed (1) hide show
  1. README.md +2 -4
README.md CHANGED
@@ -22,7 +22,7 @@ Chinese deductive reasoning model based on Transformer-XL.
22
 
23
  ## 模型信息 Model Information
24
 
25
- ### 数据准备 Corpus Preparation
26
 
27
  * 悟道语料库(280G版本)
28
  * 因果语料库(2.3M个样本):基于悟道语料库(280G版本),通过关联词匹配、人工标注 + [GTSFactory](https://gtsfactory.com/)筛选、数据清洗等步骤获取的具有因果关系的句子对
@@ -30,15 +30,13 @@ Chinese deductive reasoning model based on Transformer-XL.
30
  * Wudao Corpus (with 280G samples)
31
  * Wudao Causal Corpus (with 2.3 million samples): Based on the Wudao corpus (280G version), sentence pairs with causality were obtained through logic indicator matching, manual annotation + [GTSFactory](https://gtsfactory.com/), and data cleaning.
32
 
33
-
34
- ### 训练流程 Model Training
35
  1. 在悟道语料库(280G版本)和标注的相似句子对数据集上进行预训练([Randeng-TransformerXL-1.1B-Paraphrasing-Chinese](https://huggingface.co/IDEA-CCNL/Randeng-TransformerXL-1.1B-Paraphrasing-Chinese))
36
  2. 在1.5M因果语料上进行因果生成任务的训练
37
  3. 基于其余0.8M因果语料,协同[Randeng-TransformerXL-5B-Abduction-Chinese](https://huggingface.co/IDEA-CCNL/Randeng-TransformerXL-5B-Abduction-Chinese)和[Erlangshen-Roberta-330M-Causal-Chinese](https://huggingface.co/IDEA-CCNL/Erlangshen-Roberta-330M-Causal-Chinese)进行Self-consistency闭环迭代训练
38
  * 两个生成模型基于核采样和贪心的方式进行因果推理和反绎推理,产生大量伪样本;
39
  * Erlangshen-Roberta-330M-Causal-Chinese模型对伪样本句子对的因果关系进行打分,筛选供自身以及生成模型训练的样本
40
 
41
-
42
  First, the Transformer-XL model was pre-trained on the Wudao Corpus (with 280G samples) and annotated similar-sentence pair dataset (same as [Randeng-TransformerXL-1.1B-Paraphrasing-Chinese](https://huggingface.co/IDEA-CCNL/Randeng-TransformerXL-1.1B-Paraphrasing-Chinese)).
43
  Then, the model was trained on our causal corpus (about 1.5 million samples) for the deductive reasoning task.
44
  At last, based on the remaining 0.8 million samples of the causal corpus, we conducted self-consistent learning on this model, cooperating with [Randeng-TransformerXL-5B-Abduction-Chinese](https://huggingface.co/IDEA-CCNL/Randeng-TransformerXL-5B-Abduction-Chinese) and [Erlangshen-Roberta-330M-Causal-Chinese](https://huggingface.co/IDEA-CCNL/Erlangshen-Roberta-330M-Causal-Chinese).
 
22
 
23
  ## 模型信息 Model Information
24
 
25
+ **数据准备 Corpus Preparation**
26
 
27
  * 悟道语料库(280G版本)
28
  * 因果语料库(2.3M个样本):基于悟道语料库(280G版本),通过关联词匹配、人工标注 + [GTSFactory](https://gtsfactory.com/)筛选、数据清洗等步骤获取的具有因果关系的句子对
 
30
  * Wudao Corpus (with 280G samples)
31
  * Wudao Causal Corpus (with 2.3 million samples): Based on the Wudao corpus (280G version), sentence pairs with causality were obtained through logic indicator matching, manual annotation + [GTSFactory](https://gtsfactory.com/), and data cleaning.
32
 
33
+ **训练流程 Model Training**
 
34
  1. 在悟道语料库(280G版本)和标注的相似句子对数据集上进行预训练([Randeng-TransformerXL-1.1B-Paraphrasing-Chinese](https://huggingface.co/IDEA-CCNL/Randeng-TransformerXL-1.1B-Paraphrasing-Chinese))
35
  2. 在1.5M因果语料上进行因果生成任务的训练
36
  3. 基于其余0.8M因果语料,协同[Randeng-TransformerXL-5B-Abduction-Chinese](https://huggingface.co/IDEA-CCNL/Randeng-TransformerXL-5B-Abduction-Chinese)和[Erlangshen-Roberta-330M-Causal-Chinese](https://huggingface.co/IDEA-CCNL/Erlangshen-Roberta-330M-Causal-Chinese)进行Self-consistency闭环迭代训练
37
  * 两个生成模型基于核采样和贪心的方式进行因果推理和反绎推理,产生大量伪样本;
38
  * Erlangshen-Roberta-330M-Causal-Chinese模型对伪样本句子对的因果关系进行打分,筛选供自身以及生成模型训练的样本
39
 
 
40
  First, the Transformer-XL model was pre-trained on the Wudao Corpus (with 280G samples) and annotated similar-sentence pair dataset (same as [Randeng-TransformerXL-1.1B-Paraphrasing-Chinese](https://huggingface.co/IDEA-CCNL/Randeng-TransformerXL-1.1B-Paraphrasing-Chinese)).
41
  Then, the model was trained on our causal corpus (about 1.5 million samples) for the deductive reasoning task.
42
  At last, based on the remaining 0.8 million samples of the causal corpus, we conducted self-consistent learning on this model, cooperating with [Randeng-TransformerXL-5B-Abduction-Chinese](https://huggingface.co/IDEA-CCNL/Randeng-TransformerXL-5B-Abduction-Chinese) and [Erlangshen-Roberta-330M-Causal-Chinese](https://huggingface.co/IDEA-CCNL/Erlangshen-Roberta-330M-Causal-Chinese).