C.F.A commited on
Commit
8f1391b
·
1 Parent(s): 34e92ae

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +32 -1
README.md CHANGED
@@ -7,4 +7,35 @@ sdk: static
7
  pinned: false
8
  ---
9
 
10
- Edit this `README.md` markdown file to author your organization card.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  pinned: false
8
  ---
9
 
10
+ Hello World! This is codefuse-ai!
11
+ ![Creation Approach](LOGO.png)
12
+
13
+
14
+ <div align="center">
15
+
16
+ [**简体中文**](#chinese)|[**Git Repo**](https://github.com/codefuse-ai)
17
+
18
+ </div>
19
+
20
+ <a id="english"></a>
21
+
22
+ We are a passionate team of machine learning engineers from Ant Group dedicated to open source projects. Our goal is to develop Code Large Language Models (Code LLMs) specifically designed to support the entire software development lifecycle, including design, requirements, coding, testing, deployment, operations, and maintenance.
23
+
24
+
25
+ In this release, we are open sourcing 1) **the MFT (Multi-Task Fine-Tuning) framework, known as MFTcoder**, 2) **two datasets for enhancing the coding capabilities of LLMs**, and 3) **a faster and more reliable deployment framework based on FasterTransformer.** The resulting model ensemble, which includes CodeFuse-13B and CodeFuse-CodeLlama-34B, supports various code-related tasks such as code completion, text-to-code conversion, and unit test generation. In particular, CodeFuse-CodeLlama-34B, built upon CodeLlama as the base model and fine-tuned using the proposed MFT framework, achieves an impressive score of **74.4% (greedy decoding)** in the HumanEval Python pass@1 evaluation, **even surpassing the performance of GPT-4 (71%)**. We have plans to incorporate additional base LLMs into the ensemble in the near future.
26
+
27
+
28
+ We believe that our solution can significantly enhance the performance of pretrained LLMs across multiple related tasks simultaneously. We are committed to further exploring this direction and providing more open-source contributions. We also encourage engineers and researchers within this community to join us in co-constructing CodeFuse.
29
+
30
+
31
+ </div>
32
+
33
+ <a id="chinese"></a>
34
+
35
+ 我们是来自蚂蚁集团的一支致力于开源项目的机器学习工程师团队。我们的使命是开发出专门支持整个软件开发生命周期的代码大语言模型(Code LLMs),包括设计、需求、编码、测试、部署、运维等方面的支持。
36
+
37
+
38
+ 在本次发布中,我们开源了以下内容:1)**MFT(多任务微调)框架,也称为MFTcoder**;2)**两个用于增强LLMs编码能力的数据集**;3)**基于FasterTransformer的更快速、更可靠的部署框架**。由此产生的模型集合包括CodeFuse-13B和CodeFuse-CodeLlama-34B,支持多种与代码相关的任务,如代码补全、文本转代码、单元测试生成等。值得一提的是,CodeFuse-CodeLlama-34B基于CodeLlama作为基础模型,并利用我们提出的MFT框架进行微调,在HumanEval Python pass@1评估中取得高达的**74.4%(贪婪解码)**的好成绩,甚至**超过了GPT-4(71%)的表现**。我们计划在不久的将来将更多的基础LLMs纳入到我们的模型集合中。
39
+
40
+
41
+ 我们相信我们的解决方案可以显著提升预训练LLMs在多个相关任务上的性能。我们将进一步探索这个方向,并提供更多的开源贡献。与此同时,我们诚邀志同道合的工程师和研究人员加入这个社区,共同构建CodeFuse。