XavierSpycy
commited on
Commit
•
e02e8a8
1
Parent(s):
fe339a9
Update model card
Browse files
README.md
CHANGED
@@ -5,34 +5,36 @@ license: apache-2.0
|
|
5 |
# Meta-Llama-3-8B-Instruct-zh-10k
|
6 |
|
7 |
## Model Details / 模型细节
|
8 |
-
This model,
|
9 |
|
10 |
-
由于原模型[Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct)
|
11 |
|
12 |
For efficient inference, the model was converted to the gguf format using [llama.cpp](https://github.com/ggerganov/llama.cpp) and underwent quantization, resulting in a compact model size of about 3.18 GB, suitable for distribution across various devices.
|
13 |
|
14 |
-
|
15 |
|
16 |
### LoRa Hardware / LoRa 硬件
|
17 |
- RTX 4090D x 1
|
18 |
|
19 |
> [!NOTE]
|
20 |
-
> The complete fine-tuning process took approximately 12 hours. 完整微调过程花费约12小时。
|
21 |
|
22 |
-
|
|
|
|
|
23 |
|
24 |
### Model Developer / 模型开发者
|
25 |
- **Pretraining**: Meta
|
26 |
-
- **Fine-tuning**: XavierSpycy
|
27 |
|
28 |
- **预训练**: Meta
|
29 |
-
- **微调**: XavierSpycy
|
30 |
|
31 |
|
32 |
### Usage / 用法
|
33 |
-
This model can be utilized like the original Meta-Llama3 but offers enhanced performance in Chinese.
|
34 |
|
35 |
-
|
36 |
|
37 |
```python
|
38 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
@@ -77,14 +79,22 @@ Please refer to [Meta Llama 3's Ethical Considerations](https://huggingface.co/m
|
|
77 |
请参考 [Meta Llama 3's Ethical Considerations](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct#ethical-considerations-and-limitations),以获取更多细节。关键点包括偏见监控、负责任的使用指南和模型限制的透明度。
|
78 |
|
79 |
## Limitations / 局限性
|
80 |
-
The comprehensive abilities of the model have not been fully tested.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
81 |
|
82 |
-
|
83 |
|
84 |
## Acknowledgements / 致谢
|
85 |
We thank Meta for their open-source contributions, which have greatly benefited the developer community, and acknowledge the collaborative efforts of developers in enhancing this community.
|
86 |
|
87 |
-
我们感谢 Meta
|
88 |
|
89 |
## References / 参考资料
|
90 |
|
|
|
5 |
# Meta-Llama-3-8B-Instruct-zh-10k
|
6 |
|
7 |
## Model Details / 模型细节
|
8 |
+
This model, <u>`Meta-Llama-3-8B-Instruct-zh-10k`</u>, was fine-tuned from the original [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) due to its underperformance in Chinese. Utilizing the LoRa technology within the [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) utilities, this model was adapted to better handle Chinese through three epochs on three corpora: `alpaca_zh`, `alpaca_gpt4_zh`, and `oaast_sft_zh`, amounting to approximately 10,000 examples. This is reflected in the `10k` in its name.
|
9 |
|
10 |
+
由于原模型[Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct)在中文上表现欠佳,于是该模型 <u>`Meta-Llama-3-8B-Instruct-zh-10k`</u> 微调自此。在[LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory)工具下,利用LoRa 技术,通过`alpaca_zh`、`alpaca_gpt4_zh`和`oaast_sft_zh`三个语料库上、经过三个训练轮次,我们将该模型调整得更好地掌握了中文。三个语料库共计约10,000个样本,这也是其名字中的 `10k` 的由来。
|
11 |
|
12 |
For efficient inference, the model was converted to the gguf format using [llama.cpp](https://github.com/ggerganov/llama.cpp) and underwent quantization, resulting in a compact model size of about 3.18 GB, suitable for distribution across various devices.
|
13 |
|
14 |
+
为了高效的推理,使用 [llama.cpp](https://github.com/ggerganov/llama.cpp),我们将该模型转化为了gguf格式并量化,从而得到了一个压缩到约 3.18 GB 大小的模型,适合分发在各类设备上。
|
15 |
|
16 |
### LoRa Hardware / LoRa 硬件
|
17 |
- RTX 4090D x 1
|
18 |
|
19 |
> [!NOTE]
|
20 |
+
> The complete fine-tuning process took approximately 12 hours. / 完整微调过程花费约12小时。
|
21 |
|
22 |
+
Additional fine-tuning configurations are avaiable at [Hands-On LoRa](https://github.com/XavierSpycy/hands-on-lora) or [Llama3Ops](https://github.com/XavierSpycy/llama-ops).
|
23 |
+
|
24 |
+
更多微调配置可以在我的个人仓库 [Hands-On LoRa](https://github.com/XavierSpycy/hands-on-lora) 或 [Llama3Ops](https://github.com/XavierSpycy/llama-ops) 获得。
|
25 |
|
26 |
### Model Developer / 模型开发者
|
27 |
- **Pretraining**: Meta
|
28 |
+
- **Fine-tuning**: [XavierSpycy @ <img src="https://img.shields.io/badge/-white?logo=data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iOTgiIGhlaWdodD0iOTYiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PHBhdGggZmlsbC1ydWxlPSJldmVub2RkIiBjbGlwLXJ1bGU9ImV2ZW5vZGQiIGQ9Ik00OC44NTQgMEMyMS44MzkgMCAwIDIyIDAgNDkuMjE3YzAgMjEuNzU2IDEzLjk5MyA0MC4xNzIgMzMuNDA1IDQ2LjY5IDIuNDI3LjQ5IDMuMzE2LTEuMDU5IDMuMzE2LTIuMzYyIDAtMS4xNDEtLjA4LTUuMDUyLS4wOC05LjEyNy0xMy41OSAyLjkzNC0xNi40Mi01Ljg2Ny0xNi40Mi01Ljg2Ny0yLjE4NC01LjcwNC01LjQyLTcuMTctNS40Mi03LjE3LTQuNDQ4LTMuMDE1LjMyNC0zLjAxNS4zMjQtMy4wMTUgNC45MzQuMzI2IDcuNTIzIDUuMDUyIDcuNTIzIDUuMDUyIDQuMzY3IDcuNDk2IDExLjQwNCA1LjM3OCAxNC4yMzUgNC4wNzQuNDA0LTMuMTc4IDEuNjk5LTUuMzc4IDMuMDc0LTYuNi0xMC44MzktMS4xNDEtMjIuMjQzLTUuMzc4LTIyLjI0My0yNC4yODMgMC01LjM3OCAxLjk0LTkuNzc4IDUuMDE0LTEzLjItLjQ4NS0xLjIyMi0yLjE4NC02LjI3NS40ODYtMTMuMDM4IDAgMCA0LjEyNS0xLjMwNCAxMy40MjYgNS4wNTJhNDYuOTcgNDYuOTcgMCAwIDEgMTIuMjE0LTEuNjNjNC4xMjUgMCA4LjMzLjU3MSAxMi4yMTMgMS42MyA5LjMwMi02LjM1NiAxMy40MjctNS4wNTIgMTMuNDI3LTUuMDUyIDIuNjcgNi43NjMuOTcgMTEuODE2LjQ4NSAxMy4wMzggMy4xNTUgMy40MjIgNS4wMTUgNy44MjIgNS4wMTUgMTMuMiAwIDE4LjkwNS0xMS40MDQgMjMuMDYtMjIuMzI0IDI0LjI4MyAxLjc4IDEuNTQ4IDMuMzE2IDQuNDgxIDMuMzE2IDkuMTI2IDAgNi42LS4wOCAxMS44OTctLjA4IDEzLjUyNiAwIDEuMzA0Ljg5IDIuODUzIDMuMzE2IDIuMzY0IDE5LjQxMi02LjUyIDMzLjQwNS0yNC45MzUgMzMuNDA1LTQ2LjY5MUM5Ny43MDcgMjIgNzUuNzg4IDAgNDguODU0IDB6IiBmaWxsPSIjMjQyOTJmIi8+PC9zdmc+"> ](https://github.com/XavierSpycy) | [XavierSpycy @ 🤗](https://huggingface.co/XavierSpycy)
|
29 |
|
30 |
- **预训练**: Meta
|
31 |
+
- **微调**: [XavierSpycy @ <img src="https://img.shields.io/badge/-white?logo=data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iOTgiIGhlaWdodD0iOTYiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PHBhdGggZmlsbC1ydWxlPSJldmVub2RkIiBjbGlwLXJ1bGU9ImV2ZW5vZGQiIGQ9Ik00OC44NTQgMEMyMS44MzkgMCAwIDIyIDAgNDkuMjE3YzAgMjEuNzU2IDEzLjk5MyA0MC4xNzIgMzMuNDA1IDQ2LjY5IDIuNDI3LjQ5IDMuMzE2LTEuMDU5IDMuMzE2LTIuMzYyIDAtMS4xNDEtLjA4LTUuMDUyLS4wOC05LjEyNy0xMy41OSAyLjkzNC0xNi40Mi01Ljg2Ny0xNi40Mi01Ljg2Ny0yLjE4NC01LjcwNC01LjQyLTcuMTctNS40Mi03LjE3LTQuNDQ4LTMuMDE1LjMyNC0zLjAxNS4zMjQtMy4wMTUgNC45MzQuMzI2IDcuNTIzIDUuMDUyIDcuNTIzIDUuMDUyIDQuMzY3IDcuNDk2IDExLjQwNCA1LjM3OCAxNC4yMzUgNC4wNzQuNDA0LTMuMTc4IDEuNjk5LTUuMzc4IDMuMDc0LTYuNi0xMC44MzktMS4xNDEtMjIuMjQzLTUuMzc4LTIyLjI0My0yNC4yODMgMC01LjM3OCAxLjk0LTkuNzc4IDUuMDE0LTEzLjItLjQ4NS0xLjIyMi0yLjE4NC02LjI3NS40ODYtMTMuMDM4IDAgMCA0LjEyNS0xLjMwNCAxMy40MjYgNS4wNTJhNDYuOTcgNDYuOTcgMCAwIDEgMTIuMjE0LTEuNjNjNC4xMjUgMCA4LjMzLjU3MSAxMi4yMTMgMS42MyA5LjMwMi02LjM1NiAxMy40MjctNS4wNTIgMTMuNDI3LTUuMDUyIDIuNjcgNi43NjMuOTcgMTEuODE2LjQ4NSAxMy4wMzggMy4xNTUgMy40MjIgNS4wMTUgNy44MjIgNS4wMTUgMTMuMiAwIDE4LjkwNS0xMS40MDQgMjMuMDYtMjIuMzI0IDI0LjI4MyAxLjc4IDEuNTQ4IDMuMzE2IDQuNDgxIDMuMzE2IDkuMTI2IDAgNi42LS4wOCAxMS44OTctLjA4IDEzLjUyNiAwIDEuMzA0Ljg5IDIuODUzIDMuMzE2IDIuMzY0IDE5LjQxMi02LjUyIDMzLjQwNS0yNC45MzUgMzMuNDA1LTQ2LjY5MUM5Ny43MDcgMjIgNzUuNzg4IDAgNDguODU0IDB6IiBmaWxsPSIjMjQyOTJmIi8+PC9zdmc+"> ](https://github.com/XavierSpycy) | [XavierSpycy @ 🤗 ](https://huggingface.co/XavierSpycy)
|
32 |
|
33 |
|
34 |
### Usage / 用法
|
35 |
+
This model can be utilized like the original <u>Meta-Llama3</u> but offers enhanced performance in Chinese.
|
36 |
|
37 |
+
我们能够像原版的<u>Meta-Llama3</u>一样使用该模型,而它提供了提升后的中文能力。
|
38 |
|
39 |
```python
|
40 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
|
|
79 |
请参考 [Meta Llama 3's Ethical Considerations](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct#ethical-considerations-and-limitations),以获取更多细节。关键点包括偏见监控、负责任的使用指南和模型限制的透明度。
|
80 |
|
81 |
## Limitations / 局限性
|
82 |
+
- The comprehensive abilities of the model have not been fully tested.
|
83 |
+
|
84 |
+
- While it performs smoothly in Chinese conversations, further benchmarks are required to evaluate its full capabilities. The quality and quantity of the Chinese corpora used may also limit model outputs.
|
85 |
+
|
86 |
+
- Additionally, catastrophic forgetting in the fine-tuned model has not been evaluated.
|
87 |
+
|
88 |
+
- 该模型的全面的能力尚未全部测试。
|
89 |
+
|
90 |
+
- 尽管它在中文对话中表现流畅,但需要更多的测评以评估其完整的能力。中文语料库的质量和数量可能都会对模型输出有所制约。
|
91 |
|
92 |
+
- 另外,微调模型中的灾难性遗忘尚未评估。
|
93 |
|
94 |
## Acknowledgements / 致谢
|
95 |
We thank Meta for their open-source contributions, which have greatly benefited the developer community, and acknowledge the collaborative efforts of developers in enhancing this community.
|
96 |
|
97 |
+
我们感谢 Meta 的开源贡献,这极大地帮助了开发者社区,同时,也感谢致力于提升社区的开发者们的努力。
|
98 |
|
99 |
## References / 参考资料
|
100 |
|