s-JoL commited on
Commit
ca34596
·
1 Parent(s): 3e2345e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -24
README.md CHANGED
@@ -19,7 +19,6 @@ Baichuan-13B-Base为Baichuan-13B系列模型中的预训练版本,经过对齐
19
  3. **更高效的推理**:为了支持更广大用户的使用,我们本次同时开源了INT8和INT4的量化版本,在几乎没有效果损失的情况下可以很方便的将模型部署在如3090等消费机显卡上。
20
  4. **开源免费可商用**:Baichuan-13B不仅对学术研究完全开放,开发者也仅需邮件申请并获得官方商用许可后,即可以免费商用。
21
 
22
- ## Introduction
23
  Baichuan-13B-Base is the pre-training version in the Baichuan-13B series of models, and the aligned model can be found at [Baichuan-13B-Chat](https://github.com/baichuan-inc/Baichuan-13B-Chat).
24
 
25
  [Baichuan-13B](https://github.com/baichuan-inc/Baichuan-13B) is an open-source, commercially usable large-scale language model developed by Baichuan Intelligence, following [Baichuan-7B](https://github.com/baichuan-inc/baichuan-7B). With 13 billion parameters, it achieves the best performance in standard Chinese and English benchmarks among models of its size. This release includes two versions: pre-training (Baichuan-13B-Base) and alignment (Baichuan-13B-Chat). Baichuan-13B has the following features:
@@ -30,9 +29,9 @@ Baichuan-13B-Base is the pre-training version in the Baichuan-13B series of mode
30
  4. **Open-source, free, and commercially usable**: Baichuan-13B is not only fully open to academic research, but developers can also use it for free commercially after applying for and receiving official commercial permission via email.
31
 
32
 
33
- ## Model Details
34
 
35
- ### Model Description
36
 
37
  <!-- Provide a longer summary of what this model is. -->
38
 
@@ -41,7 +40,7 @@ Baichuan-13B-Base is the pre-training version in the Baichuan-13B series of mode
41
  - **Language(s) (NLP):** Chinese/English
42
  - **License:** [Baichuan-13B License]()
43
 
44
- ### Model Sources
45
 
46
  <!-- Provide the basic links for the model. -->
47
 
@@ -71,46 +70,39 @@ The specific parameters are as follows:
71
  | Baichuan-7B | 4,096 | 32 | 32 | 64,000 | 7,000,559,616 | 1.2万亿 | [RoPE](https://arxiv.org/abs/2104.09864) | 4,096 |
72
  | Baichuan-13B | 5,120 | 40 | 40 | 64,000 | 13,264,901,120 | 1.4万亿 | [ALiBi](https://arxiv.org/abs/2108.12409) | 4,096
73
 
74
- ## Uses
75
 
76
  <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
77
 
78
- ### Downstream Use
79
 
80
  <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
81
  我们同时开源出了和本模型配套的训练代码,允许进行高效的Finetune用于下游任务,具体参见[Baichuan-13B](https://github.com/baichuan-inc/Baichuan-13B)。
82
 
83
  We have also open-sourced the training code that accompanies this model, allowing for efficient finetuning for downstream tasks. For more details, please refer to [Baichuan-13B](https://github.com/baichuan-inc/Baichuan-13B).
84
 
85
- ### Out-of-Scope Use
86
 
87
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
88
- 在没有充分评估风险和采取缓解措施的情况下投入生产使用;任何可能被视为不负责任或有害的使用案例。
89
 
90
- Production use without adequate assessment of risks and mitigation; any use cases which may be considered irresponsible or harmful.
91
 
92
- ## Bias, Risks, and Limitations
93
 
94
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
95
 
96
- Baichuan-13B可能会产生事实上不正确的输出,不应依赖它产生事实上准确的信息。Baichuan-13B是在各种公共数据集上进行训练的。尽管我们已经做出了巨大的努力来清洗预训练数据,但这个模型可能会生成淫秽、偏见或其他冒犯性的输出。
97
-
98
- Baichuan-13B can produce factually incorrect output, and should not be relied on to produce factually accurate information. Baichuan-13B was trained on various public datasets. While great efforts have been taken to clean the pretraining data, it is possible that this model could generate lewd, biased or otherwise offensive outputs.
99
-
100
- ## Training Details
101
 
102
  训练具体设置参见[Baichuan-13B](https://github.com/baichuan-inc/Baichuan-13B)。
103
 
104
  For specific training settings, please refer to [Baichuan-13B](https://github.com/baichuan-inc/Baichuan-13B).
105
 
106
- ## Evaluation
107
-
108
- # Benchmark结果
109
 
110
  我们在各个 benchmark 下进行了`5-shot`评测,所采用的方法和 [Baichuan-7B](https://github.com/baichuan-inc/Baichuan-7B/) 项目中相同。结果如下:
111
 
112
  We conducted a `5-shot` evaluation under various benchmarks, using the same method as in the [Baichuan-7B](https://github.com/baichuan-inc/Baichuan-7B/) project. The results are as follows:
113
- ## C-Eval
114
 
115
  | Model 5-shot | STEM | Social Sciences | Humanities | Others | Average |
116
  |-------------------------|-------|-----------------|------------|--------|---------|
@@ -125,7 +117,7 @@ We conducted a `5-shot` evaluation under various benchmarks, using the same meth
125
  | **Baichuan-13B-Chat** | **43.7** | **64.6** | **56.2** | **49.2** | **51.5** |
126
  > *说明:表中各个模型的结果是使用统一的评估代码得到。[InternLM-7B](https://huggingface.co/internlm/internlm-7b) 汇报使用 [OpenCompass](https://opencompass.org.cn/rank) 工具评估的C-Eval平均值为 53.4,我们使用 OpenCompass 评估 InternLM-7B 的平均值为 51.6
127
 
128
- ## MMLU
129
 
130
  | Model 5-shot | STEM | Social Sciences | Humanities | Others | Average |
131
  |-------------------------|-------|-----------------|------------|--------|---------|
@@ -140,7 +132,7 @@ We conducted a `5-shot` evaluation under various benchmarks, using the same meth
140
  | **Baichuan-13B-Chat** | **40.9** | **60.9** | **48.8** | **59.0** | **52.1** |
141
 
142
 
143
- ## CMMLU
144
 
145
  | Model 5-shot | STEM | Humanities | Social Sciences | Others | China Specific | Average |
146
  |-------------------------|-------|------------|-----------------|--------|----------------|---------|
@@ -156,5 +148,5 @@ We conducted a `5-shot` evaluation under various benchmarks, using the same meth
156
 
157
  > 说明:CMMLU是一个综合性的中文评估基准,专门用于评估语言模型在中文语境下的知识和推理能力。我们采用了其官方的[评测方案](https://github.com/haonan-li/CMMLU)。
158
 
159
- ## Our Group
160
  ![WeChat](https://github.com/baichuan-inc/baichuan-7B/blob/main/media/wechat.jpeg?raw=true)
 
19
  3. **更高效的推理**:为了支持更广大用户的使用,我们本次同时开源了INT8和INT4的量化版本,在几乎没有效果损失的情况下可以很方便的将模型部署在如3090等消费机显卡上。
20
  4. **开源免费可商用**:Baichuan-13B不仅对学术研究完全开放,开发者也仅需邮件申请并获得官方商用许可后,即可以免费商用。
21
 
 
22
  Baichuan-13B-Base is the pre-training version in the Baichuan-13B series of models, and the aligned model can be found at [Baichuan-13B-Chat](https://github.com/baichuan-inc/Baichuan-13B-Chat).
23
 
24
  [Baichuan-13B](https://github.com/baichuan-inc/Baichuan-13B) is an open-source, commercially usable large-scale language model developed by Baichuan Intelligence, following [Baichuan-7B](https://github.com/baichuan-inc/baichuan-7B). With 13 billion parameters, it achieves the best performance in standard Chinese and English benchmarks among models of its size. This release includes two versions: pre-training (Baichuan-13B-Base) and alignment (Baichuan-13B-Chat). Baichuan-13B has the following features:
 
29
  4. **Open-source, free, and commercially usable**: Baichuan-13B is not only fully open to academic research, but developers can also use it for free commercially after applying for and receiving official commercial permission via email.
30
 
31
 
32
+ ## 模型详情
33
 
34
+ ### 模型描述
35
 
36
  <!-- Provide a longer summary of what this model is. -->
37
 
 
40
  - **Language(s) (NLP):** Chinese/English
41
  - **License:** [Baichuan-13B License]()
42
 
43
+ ### 模型结构
44
 
45
  <!-- Provide the basic links for the model. -->
46
 
 
70
  | Baichuan-7B | 4,096 | 32 | 32 | 64,000 | 7,000,559,616 | 1.2万亿 | [RoPE](https://arxiv.org/abs/2104.09864) | 4,096 |
71
  | Baichuan-13B | 5,120 | 40 | 40 | 64,000 | 13,264,901,120 | 1.4万亿 | [ALiBi](https://arxiv.org/abs/2108.12409) | 4,096
72
 
73
+ ## 使用
74
 
75
  <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
76
 
77
+ ### 下游使用
78
 
79
  <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
80
  我们同时开源出了和本模型配套的训练代码,允许进行高效的Finetune用于下游任务,具体参见[Baichuan-13B](https://github.com/baichuan-inc/Baichuan-13B)。
81
 
82
  We have also open-sourced the training code that accompanies this model, allowing for efficient finetuning for downstream tasks. For more details, please refer to [Baichuan-13B](https://github.com/baichuan-inc/Baichuan-13B).
83
 
84
+ ### 免责声明
85
 
86
+ 我们在此声明,我们的开发团队并未基于 Baichuan-13B 模型开发任何应用,无论是在 iOS、Android、网页或任何其他平台。我们强烈呼吁所有使用者,不要利用 Baichuan-13B 模型进行任何危害国家社会安全或违法的活动。另外,我们也要求使用者不要将 Baichuan-13B 模型用于未经适当安全审查和备案的互联网服务。我们希望所有的使用者都能遵守这个原则,确保科技的发展能在规范和合法的环境下进行。
 
87
 
88
+ 我们已经尽我们所能,来确保模型训练过程中使用的数据的合规性。然而,尽管我们已经做出了巨大的努力,但由于模型和数据的复杂性,仍有可能存在一些无法预见的问题。因此,如果由于使用 Baichuan-13B 开源模型而导致的任何问题,包括但不限于数据安全问题、公共舆论风险,或模型被误导、滥用、传播或不当利用所带来的任何风险和问题,我们将不承担任何责任。
89
 
90
+ We hereby declare that our development team has not developed any applications based on the Baichuan-13B model, whether on iOS, Android, the web, or any other platform. We strongly urge all users not to use the Baichuan-13B model for any activities that harm national social security or are illegal. In addition, we also ask users not to use the Baichuan-13B model for internet services that have not undergone appropriate security review and filing. We hope that all users will adhere to this principle to ensure that technological development takes place in a regulated and legal environment.
91
 
92
+ We have done our utmost to ensure the compliance of the data used in the model training process. However, despite our great efforts, due to the complexity of the model and data, there may still be some unforeseen issues. Therefore, we will not take any responsibility for any issues arising from the use of the Baichuan-13B open-source model, including but not limited to data security issues, public opinion risks, or any risks and problems arising from the model being misled, misused, disseminated, or improperly exploited.
93
 
94
+ ## 训练详情
 
 
 
 
95
 
96
  训练具体设置参见[Baichuan-13B](https://github.com/baichuan-inc/Baichuan-13B)。
97
 
98
  For specific training settings, please refer to [Baichuan-13B](https://github.com/baichuan-inc/Baichuan-13B).
99
 
100
+ ## 测评结果
 
 
101
 
102
  我们在各个 benchmark 下进行了`5-shot`评测,所采用的方法和 [Baichuan-7B](https://github.com/baichuan-inc/Baichuan-7B/) 项目中相同。结果如下:
103
 
104
  We conducted a `5-shot` evaluation under various benchmarks, using the same method as in the [Baichuan-7B](https://github.com/baichuan-inc/Baichuan-7B/) project. The results are as follows:
105
+ ### C-Eval
106
 
107
  | Model 5-shot | STEM | Social Sciences | Humanities | Others | Average |
108
  |-------------------------|-------|-----------------|------------|--------|---------|
 
117
  | **Baichuan-13B-Chat** | **43.7** | **64.6** | **56.2** | **49.2** | **51.5** |
118
  > *说明:表中各个模型的结果是使用统一的评估代码得到。[InternLM-7B](https://huggingface.co/internlm/internlm-7b) 汇报使用 [OpenCompass](https://opencompass.org.cn/rank) 工具评估的C-Eval平均值为 53.4,我们使用 OpenCompass 评估 InternLM-7B 的平均值为 51.6
119
 
120
+ ### MMLU
121
 
122
  | Model 5-shot | STEM | Social Sciences | Humanities | Others | Average |
123
  |-------------------------|-------|-----------------|------------|--------|---------|
 
132
  | **Baichuan-13B-Chat** | **40.9** | **60.9** | **48.8** | **59.0** | **52.1** |
133
 
134
 
135
+ ### CMMLU
136
 
137
  | Model 5-shot | STEM | Humanities | Social Sciences | Others | China Specific | Average |
138
  |-------------------------|-------|------------|-----------------|--------|----------------|---------|
 
148
 
149
  > 说明:CMMLU是一个综合性的中文评估基准,专门用于评估语言模型在中文语境下的知识和推理能力。我们采用了其官方的[评测方案](https://github.com/haonan-li/CMMLU)。
150
 
151
+ ## 微信群组
152
  ![WeChat](https://github.com/baichuan-inc/baichuan-7B/blob/main/media/wechat.jpeg?raw=true)