|
--- |
|
license: apache-2.0 |
|
datasets: |
|
- BAAI/IndustryInstruction |
|
- BAAI/IndustryInstruction_Technology-Research |
|
base_model: |
|
- meta-llama/Meta-Llama-3.1-8B-Instruct |
|
tags: |
|
- 科学研究 |
|
- 中英文语言模型 |
|
--- |
|
|
|
This model is finetuned on the model llama3.1-8b-instruct using the dataset [BAAI/IndustryInstruction_Technology-Research](https://huggingface.co/datasets/BAAI/IndustryInstruction_Technology-Research) dataset, the dataset details can jump to the repo: [BAAI/IndustryInstruction](https://huggingface.co/datasets/BAAI/IndustryInstruction) |
|
|
|
## training params |
|
|
|
The training framework is llama-factory, template=llama3 |
|
|
|
``` |
|
learning_rate=1e-5 |
|
lr_scheduler_type=cosine |
|
max_length=2048 |
|
warmup_ratio=0.05 |
|
batch_size=64 |
|
epoch=10 |
|
``` |
|
|
|
select best ckpt by the evaluation loss |
|
## evaluation |
|
|
|
Duto to there is no evaluation benchmark, we can not eval the model |
|
|
|
## How to use |
|
|
|
```python |
|
# !/usr/bin/env python |
|
# -*- coding:utf-8 -*- |
|
# ================================================================== |
|
# [Author] : xiaofeng |
|
# [Descriptions] : |
|
# ================================================================== |
|
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
import transformers |
|
import torch |
|
|
|
|
|
llama3_jinja = """{% if messages[0]['role'] == 'system' %} |
|
{% set offset = 1 %} |
|
{% else %} |
|
{% set offset = 0 %} |
|
{% endif %} |
|
|
|
{{ bos_token }} |
|
{% for message in messages %} |
|
{% if (message['role'] == 'user') != (loop.index0 % 2 == offset) %} |
|
{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }} |
|
{% endif %} |
|
|
|
{{ '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n' + message['content'] | trim + '<|eot_id|>' }} |
|
{% endfor %} |
|
|
|
{% if add_generation_prompt %} |
|
{{ '<|start_header_id|>' + 'assistant' + '<|end_header_id|>\n\n' }} |
|
{% endif %}""" |
|
|
|
|
|
dtype = torch.bfloat16 |
|
|
|
model_dir = "MonteXiaofeng/Technology-llama3_1_8B_instruct" |
|
model = AutoModelForCausalLM.from_pretrained( |
|
model_dir, |
|
device_map="cuda", |
|
torch_dtype=dtype, |
|
) |
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_dir) |
|
tokenizer.chat_template = llama3_jinja # update template |
|
|
|
message = [ |
|
{"role": "system", "content": "You are a helpful assistant"}, |
|
{ |
|
"role": "user", |
|
"content": "请详细描述科技研究如何改变了我们的教育系统。", |
|
}, |
|
] |
|
prompt = tokenizer.apply_chat_template( |
|
message, tokenize=False, add_generation_prompt=True |
|
) |
|
print(prompt) |
|
inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt") |
|
prompt_length = len(inputs[0]) |
|
print(f"prompt_length:{prompt_length}") |
|
|
|
generating_args = { |
|
"do_sample": True, |
|
"temperature": 1.0, |
|
"top_p": 0.5, |
|
"top_k": 15, |
|
"max_new_tokens": 512, |
|
} |
|
|
|
|
|
generate_output = model.generate(input_ids=inputs.to(model.device), **generating_args) |
|
|
|
response_ids = generate_output[:, prompt_length:] |
|
response = tokenizer.batch_decode( |
|
response_ids, skip_special_tokens=True, clean_up_tokenization_spaces=True |
|
)[0] |
|
|
|
|
|
""" |
|
科技研究对我们的教育系统产生了深远的影响。首先，科技研究使得教育变得更加普及。通过互联网和数字化技术，学生可以在任何时间、任何地点接受教育，这大大增加了教育的可获取性。其次，科技研究也使得教育变得更加个性化。通过大数据和人工智能等技术，教育系统可以根据每个学生的学习情况和需求，提供定制化的教学方案。此外，科技研究还促进了教育的互动性。通过虚拟现实、增强现实等技术，学生可以更好地参与到学习中来，提高学习的趣味性和效果。总的来说，科技研究正在不断地推动教育系统的发展，使教育更加普及、个性化和互动。 |
|
""" |
|
print(f"response:{response}") |
|
|
|
|
|
``` |
