Commit
·
ee36b62
1
Parent(s):
e753166
Update README.md
Browse files
README.md
CHANGED
@@ -3,3 +3,302 @@ license: other
|
|
3 |
license_name: license.md
|
4 |
license_link: LICENSE
|
5 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
license_name: license.md
|
4 |
license_link: LICENSE
|
5 |
---
|
6 |
+
# Model Card for CodeFuse-CodeGeeX2-6B
|
7 |
+
<p align="center">
|
8 |
+
<img src="https://modelscope.cn/api/v1/models/codefuse-ai/CodeFuse-CodeGeeX2-6B/repo?Revision=master&FilePath=LOGO.jpg&View=true" width="800"/>
|
9 |
+
<p>
|
10 |
+
|
11 |
+
[[中文]](#chinese) [[English]](#english)
|
12 |
+
|
13 |
+
|
14 |
+
<a id="english"></a>
|
15 |
+
|
16 |
+
## Model Description
|
17 |
+
|
18 |
+
CodeFuse-CodeGeeX2-6B is a 6B Code-LLM finetuned by LoRA of multiple code tasks on the base model CodeGeeX2.
|
19 |
+
|
20 |
+
<br>
|
21 |
+
|
22 |
+
## News and Updates
|
23 |
+
|
24 |
+
🔥🔥 2023-11-10 CodeFuse-CodeGeeX2-6B has been released, achieving a pass@1 (greedy decoding) score of 45.12% on HumanEval, which is a 9.22% increase compared to CodeGeeX2 35.9%.
|
25 |
+
|
26 |
+
🔥🔥 2023-10-20 CodeFuse-QWen-14B technical documentation has been released. For those interested, please refer to the CodeFuse article on our WeChat official account via the provided link.(https://mp.weixin.qq.com/s/PCQPkvbvfxSPzsqjOILCDw)
|
27 |
+
|
28 |
+
🔥🔥 2023-10-16 CodeFuse-QWen-14B has been released, achieving a pass@1 (greedy decoding) score of 48.78% on HumanEval, which is a 16% increase compared to Qwen-14b's 32.3%.
|
29 |
+
|
30 |
+
🔥🔥 2023-09-27 CodeFuse-StarCoder-15B has been released, achieving a pass@1 (greedy decoding) score of 54.9% on HumanEval, which is a 21% increase compared to StarCoder's 33.6%.
|
31 |
+
|
32 |
+
🔥🔥🔥 2023-09-26 We are pleased to announce the release of the [4-bit quantized version](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B-4bits/summary) of [CodeFuse-CodeLlama-34B](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B/summary). Despite the quantization process, the model still achieves a remarkable 73.8% accuracy (greedy decoding) on the HumanEval pass@1 metric.
|
33 |
+
|
34 |
+
🔥🔥🔥 2023-09-11 [CodeFuse-CodeLlama34B](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B/summary) has achived 74.4% of pass@1 (greedy decoding) on HumanEval, which is SOTA results for openspurced LLMs at present.
|
35 |
+
|
36 |
+
<br>
|
37 |
+
|
38 |
+
## Code Community
|
39 |
+
|
40 |
+
**Homepage**: 🏡 https://github.com/codefuse-ai (**Please give us your support with a Star🌟 + Fork🚀 + Watch👀**)
|
41 |
+
|
42 |
+
+ If you wish to fine-tune the model yourself, you can visit ✨[MFTCoder](https://github.com/codefuse-ai/MFTCoder)✨✨
|
43 |
+
|
44 |
+
+ If you wish to deploy the model yourself, you can visit ✨[FasterTransformer4CodeFuse](https://github.com/codefuse-ai/FasterTransformer4CodeFuse)✨✨
|
45 |
+
|
46 |
+
+ If you wish to see a demo of the model, you can visit ✨[CodeFuse Demo](https://github.com/codefuse-ai/codefuse)✨✨
|
47 |
+
|
48 |
+
<br>
|
49 |
+
|
50 |
+
## Performance
|
51 |
+
|
52 |
+
|
53 |
+
| Model | HumanEval(pass@1) | Date |
|
54 |
+
|:----------------------------|:-----------------:|:-------:|
|
55 |
+
| **CodeFuse-CodeLlama-34B** | **74.4%** | 2023.9 |
|
56 |
+
|**CodeFuse-CodeLlama-34B-4bits** | **73.8%** | 2023.9 |
|
57 |
+
| WizardCoder-Python-34B-V1.0 | 73.2% | 2023.8 |
|
58 |
+
| GPT-4(zero-shot) | 67.0% | 2023.3 |
|
59 |
+
| PanGu-Coder2 15B | 61.6% | 2023.8 |
|
60 |
+
| CodeLlama-34b-Python | 53.7% | 2023.8 |
|
61 |
+
| CodeLlama-34b | 48.8% | 2023.8 |
|
62 |
+
| GPT-3.5(zero-shot) | 48.1% | 2022.11 |
|
63 |
+
| OctoCoder | 46.2% | 2023.8 |
|
64 |
+
| StarCoder-15B | 33.6% | 2023.5 |
|
65 |
+
| Qwen-14b | 32.3% | 2023.10 |
|
66 |
+
| **CodeFuse-StarCoder-15B** | **54.9%** | 2023.9 |
|
67 |
+
| **CodeFuse-QWen-14B** | **48.78%** | 2023.10 |
|
68 |
+
| **CodeFuse-CodeGeeX2-6B** | **45.12%** | 2023.11 |
|
69 |
+
|
70 |
+
|
71 |
+
<br>
|
72 |
+
|
73 |
+
## Requirements
|
74 |
+
|
75 |
+
* python>=3.8
|
76 |
+
* pytorch>=2.0.0
|
77 |
+
* transformers==4.33.2
|
78 |
+
* Sentencepiece
|
79 |
+
* CUDA 11.4
|
80 |
+
<br>
|
81 |
+
|
82 |
+
## Inference String Format
|
83 |
+
|
84 |
+
The inference string is a concatenated string formed by combining conversation data(system, human and bot contents) in the training data format. It is used as input during the inference process.
|
85 |
+
Here is an example format of the concatenated string:
|
86 |
+
|
87 |
+
```python
|
88 |
+
"""
|
89 |
+
<s>system
|
90 |
+
System instruction
|
91 |
+
<s>human
|
92 |
+
Human 1st round input
|
93 |
+
<s>bot
|
94 |
+
Bot 1st round output<|endoftext|>
|
95 |
+
<s>human
|
96 |
+
Human 2nd round input
|
97 |
+
<s>bot
|
98 |
+
Bot 2nd round output<|endoftext|>
|
99 |
+
...
|
100 |
+
...
|
101 |
+
...
|
102 |
+
<s>human
|
103 |
+
Human nth round input
|
104 |
+
<s>bot
|
105 |
+
{Bot output to be genreated}<|endoftext|>
|
106 |
+
"""
|
107 |
+
```
|
108 |
+
|
109 |
+
When applying inference, you always make your input string end with "\<s\>bot" to ask the model generating answers.
|
110 |
+
|
111 |
+
|
112 |
+
## Quickstart
|
113 |
+
|
114 |
+
|
115 |
+
```bash
|
116 |
+
pip install transformers modelscope cpm_kernels -U
|
117 |
+
pip install -r requirements.txt
|
118 |
+
```
|
119 |
+
|
120 |
+
```python
|
121 |
+
import torch
|
122 |
+
from modelscope import (
|
123 |
+
AutoTokenizer,
|
124 |
+
AutoModel,
|
125 |
+
snapshot_download
|
126 |
+
)
|
127 |
+
model_dir = snapshot_download('codefuse-ai/CodeFuse-CodeGeeX2-6B',revision = 'v1.0.0')
|
128 |
+
tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
|
129 |
+
tokenizer.padding_side = "left"
|
130 |
+
# try 4bit loading if cuda memory not enough
|
131 |
+
model = AutoModel.from_pretrained(model_dir,
|
132 |
+
trust_remote_code=True,
|
133 |
+
load_in_4bit=False,
|
134 |
+
device_map="auto",
|
135 |
+
torch_dtype=torch.bfloat16)
|
136 |
+
model.eval()
|
137 |
+
|
138 |
+
HUMAN_ROLE_START_TAG = "<s>human\n"
|
139 |
+
BOT_ROLE_START_TAG = "<s>bot\n"
|
140 |
+
|
141 |
+
text = f"{HUMAN_ROLE_START_TAG}write a python function of quick sort.\n{BOT_ROLE_START_TAG}"
|
142 |
+
inputs = tokenizer(text, return_tensors='pt', padding=True, add_special_tokens=False).to("cuda")
|
143 |
+
outputs = model.generate(
|
144 |
+
inputs=inputs["input_ids"],
|
145 |
+
attention_mask=inputs["attention_mask"],
|
146 |
+
max_new_tokens=512,
|
147 |
+
top_p=0.95,
|
148 |
+
temperature=0.1,
|
149 |
+
do_sample=True,
|
150 |
+
eos_token_id=tokenizer.eos_token_id,
|
151 |
+
pad_token_id=tokenizer.pad_token_id
|
152 |
+
)
|
153 |
+
|
154 |
+
gen_text = tokenizer.batch_decode(outputs[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
|
155 |
+
print(gen_text[0])
|
156 |
+
```
|
157 |
+
|
158 |
+
|
159 |
+
|
160 |
+
|
161 |
+
|
162 |
+
|
163 |
+
|
164 |
+
|
165 |
+
<a id="chinese"></a>
|
166 |
+
|
167 |
+
## 模型简介
|
168 |
+
|
169 |
+
CodeFuse-CodeGeeX2-6B 是一个通过LoRA对基座模型CodeGeeeX2进行多代码任务微调的代码大模型。
|
170 |
+
<br>
|
171 |
+
|
172 |
+
## 新闻
|
173 |
+
|
174 |
+
🔥🔥 2023-11-10 开源了CodeFuse-CodeGeeX2-6B模型,在HumanEval pass@1(greedy decoding)上可以达到48.12%, 比CodeGeeX2提高了9.22%的代码能力(HumanEval)
|
175 |
+
|
176 |
+
🔥🔥 2023-10-20 公布了CodeFuse-QWen-14B技术文档,感兴趣详见微信公众号CodeFuse文章:https://mp.weixin.qq.com/s/PCQPkvbvfxSPzsqjOILCDw
|
177 |
+
|
178 |
+
🔥🔥 2023-10-16开源了CodeFuse-QWen-14B模型,在HumanEval pass@1(greedy decoding)上可以达到48.78%, 比Qwen-14b提高了16%的代码能力(HumanEval)
|
179 |
+
|
180 |
+
🔥🔥 2023-09-27开源了CodeFuse-StarCoder-15B模型,在HumanEval pass@1(greedy decoding)上可以达到54.9%, 比StarCoder提高了21%的代码能力(HumanEval)
|
181 |
+
|
182 |
+
🔥🔥🔥 2023-09-26 [CodeFuse-CodeLlama-34B 4bits](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B-4bits/summary)量化版本发布,量化后模型在HumanEval pass@1指标为73.8% (贪婪解码)。
|
183 |
+
|
184 |
+
🔥🔥🔥 2023-09-11 [CodeFuse-CodeLlama-34B](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B/summary)发布,HumanEval pass@1指标达到74.4% (贪婪解码), 为当前开源SOTA。
|
185 |
+
|
186 |
+
<br>
|
187 |
+
|
188 |
+
## 代码社区
|
189 |
+
**大本营**: 🏡 https://github.com/codefuse-ai (**请支持我们的项目Star🌟 + Fork🚀 + Watch👀**)
|
190 |
+
|
191 |
+
+ 如果您想自己微调该模型,可以访问 ✨[MFTCoder](https://github.com/codefuse-ai/MFTCoder)✨✨
|
192 |
+
|
193 |
+
+ 如果您想自己部署该模型,可以访问 ✨[FasterTransformer4CodeFuse](https://github.com/codefuse-ai/FasterTransformer4CodeFuse)✨✨
|
194 |
+
|
195 |
+
+ 如果您想观看该模型示例,可以访问 ✨[CodeFuse Demo](https://github.com/codefuse-ai/codefuse)✨✨
|
196 |
+
|
197 |
+
<br>
|
198 |
+
|
199 |
+
|
200 |
+
## 评测表现
|
201 |
+
|
202 |
+
### 代码
|
203 |
+
|
204 |
+
|
205 |
+
| 模型 | HumanEval(pass@1) | 日期 |
|
206 |
+
|:----------------------------|:-----------------:|:-------:|
|
207 |
+
| **CodeFuse-CodeLlama-34B** | **74.4%** | 2023.9 |
|
208 |
+
|**CodeFuse-CodeLlama-34B-4bits** | **73.8%** | 2023.9 |
|
209 |
+
| WizardCoder-Python-34B-V1.0 | 73.2% | 2023.8 |
|
210 |
+
| GPT-4(zero-shot) | 67.0% | 2023.3 |
|
211 |
+
| PanGu-Coder2 15B | 61.6% | 2023.8 |
|
212 |
+
| CodeLlama-34b-Python | 53.7% | 2023.8 |
|
213 |
+
| CodeLlama-34b | 48.8% | 2023.8 |
|
214 |
+
| GPT-3.5(zero-shot) | 48.1% | 2022.11 |
|
215 |
+
| OctoCoder | 46.2% | 2023.8 |
|
216 |
+
| StarCoder-15B | 33.6% | 2023.5 |
|
217 |
+
| Qwen-14b | 32.3% | 2023.10 |
|
218 |
+
| **CodeFuse-StarCoder-15B** | **54.9%** | 2023.9 |
|
219 |
+
| **CodeFuse-QWen-14B** | **48.78%** | 2023.8 |
|
220 |
+
| **CodeFuse-CodeGeeX2-6B** | **45.12%** | 2023.11 |
|
221 |
+
|
222 |
+
|
223 |
+
## Requirements
|
224 |
+
|
225 |
+
* python>=3.8
|
226 |
+
* pytorch>=2.0.0
|
227 |
+
* transformers==4.33.2
|
228 |
+
* Sentencepiece
|
229 |
+
* CUDA 11.4
|
230 |
+
<br>
|
231 |
+
|
232 |
+
## 推理数据格式
|
233 |
+
|
234 |
+
推理数据为模型在训练数据格式下拼接的字符串形式,它也是推理时输入prompt拼接的方式:
|
235 |
+
|
236 |
+
```python
|
237 |
+
"""
|
238 |
+
<s>system
|
239 |
+
这是System指令
|
240 |
+
<s>human
|
241 |
+
这是第1轮用户输入的问题
|
242 |
+
<s>bot
|
243 |
+
这是第1轮模型生成的内容<|endoftext|>
|
244 |
+
<s>human
|
245 |
+
这是第2轮用户输入的问题
|
246 |
+
<s>bot
|
247 |
+
这是第2轮模型生成的内容<|endoftext|>
|
248 |
+
...
|
249 |
+
...
|
250 |
+
...
|
251 |
+
<s>human
|
252 |
+
这是第n轮用户输入的问题
|
253 |
+
<s>bot
|
254 |
+
{模型现在要生成的内容}<|endoftext|>
|
255 |
+
"""
|
256 |
+
```
|
257 |
+
|
258 |
+
推理时,请确保拼接的prompt字符串以"\<s\>bot\n"结尾,引导模型生成回答。
|
259 |
+
|
260 |
+
## 快速使用
|
261 |
+
|
262 |
+
|
263 |
+
```bash
|
264 |
+
pip install transformers modelscope cpm_kernels -U
|
265 |
+
pip install -r requirements.txt
|
266 |
+
```
|
267 |
+
|
268 |
+
```python
|
269 |
+
import torch
|
270 |
+
from modelscope import (
|
271 |
+
AutoTokenizer,
|
272 |
+
AutoModel,
|
273 |
+
snapshot_download
|
274 |
+
)
|
275 |
+
model_dir = snapshot_download('codefuse-ai/CodeFuse-CodeGeeX2-6B',revision = 'v1.0.0')
|
276 |
+
tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
|
277 |
+
tokenizer.padding_side = "left"
|
278 |
+
# try 4bit loading if cuda memory not enough
|
279 |
+
model = AutoModel.from_pretrained(model_dir,
|
280 |
+
trust_remote_code=True,
|
281 |
+
load_in_4bit=False,
|
282 |
+
device_map="auto",
|
283 |
+
torch_dtype=torch.bfloat16)
|
284 |
+
model.eval()
|
285 |
+
|
286 |
+
HUMAN_ROLE_START_TAG = "<s>human\n"
|
287 |
+
BOT_ROLE_START_TAG = "<s>bot\n"
|
288 |
+
|
289 |
+
text = f"{HUMAN_ROLE_START_TAG}write a python function of quick sort.\n{BOT_ROLE_START_TAG}"
|
290 |
+
inputs = tokenizer(text, return_tensors='pt', padding=True, add_special_tokens=False).to("cuda")
|
291 |
+
outputs = model.generate(
|
292 |
+
inputs=inputs["input_ids"],
|
293 |
+
attention_mask=inputs["attention_mask"],
|
294 |
+
max_new_tokens=512,
|
295 |
+
top_p=0.95,
|
296 |
+
temperature=0.1,
|
297 |
+
do_sample=True,
|
298 |
+
eos_token_id=tokenizer.eos_token_id,
|
299 |
+
pad_token_id=tokenizer.pad_token_id
|
300 |
+
)
|
301 |
+
|
302 |
+
gen_text = tokenizer.batch_decode(outputs[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
|
303 |
+
print(gen_text[0])
|
304 |
+
```
|