Update README.md

README.md

@@ -1,20 +1,96 @@
 ---
 library_name: peft
+pipeline_tag: conversational
+datasets:
+- fnlp/moss-003-sft-data
 ---
-## Training procedure
 
+<div align="center">
+  <img src="https://github.com/InternLM/lmdeploy/assets/36994684/0cf8d00f-e86b-40ba-9b54-dc8f1bc6c8d8" width="600"/>
 
-The following `bitsandbytes` quantization config was used during training:
-- load_in_8bit: False
-- load_in_4bit: True
-- llm_int8_threshold: 6.0
-- llm_int8_skip_modules: None
-- llm_int8_enable_fp32_cpu_offload: True
-- llm_int8_has_fp16_weight: False
-- bnb_4bit_quant_type: nf4
-- bnb_4bit_use_double_quant: True
-- bnb_4bit_compute_dtype: float16
-### Framework versions
 
+[![Generic badge](https://img.shields.io/badge/GitHub-%20XTuner-black.svg)](https://github.com/InternLM/xtuner)
 
-- PEFT 0.4.0
+
+</div>
+
+## Model
+
+Qwen-7B-qlora-moss-003-sft is fine-tuned from [Qwen-7B](https://huggingface.co/Qwen/Qwen-7B) with [moss-003-sft](https://huggingface.co/datasets/fnlp/moss-003-sft-data) dataset by [XTuner](https://github.com/InternLM/xtuner).
+
+
+## Quickstart
+
+### Usage with HuggingFace libraries
+
+```python
+import torch
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, StoppingCriteria
+from transformers.generation import GenerationConfig
+
+class StopWordStoppingCriteria(StoppingCriteria):
+    def __init__(self, tokenizer, stop_word):
+        self.tokenizer = tokenizer
+        self.stop_word = stop_word
+        self.length = len(self.stop_word)
+    def __call__(self, input_ids, *args, **kwargs) -> bool:
+        cur_text = self.tokenizer.decode(input_ids[0])
+        cur_text = cur_text.replace('\r', '').replace('\n', '')
+        return cur_text[-self.length:] == self.stop_word
+
+tokenizer = AutoTokenizer.from_pretrained('Qwen/Qwen-7B', trust_remote_code=True)
+quantization_config = BitsAndBytesConfig(load_in_4bit=True, load_in_8bit=False, llm_int8_threshold=6.0, llm_int8_has_fp16_weight=False, bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True, bnb_4bit_quant_type='nf4')
+model = AutoModelForCausalLM.from_pretrained('Qwen/Qwen-7B', quantization_config=quantization_config, device_map='auto', trust_remote_code=True).eval()
+model = PeftModel.from_pretrained(model, 'xtuner/Qwen-7B-qlora-moss-003-sft')
+gen_config = GenerationConfig(max_new_tokens=512, do_sample=True, temperature=0.1, top_p=0.75, top_k=40)
+
+prompt_template = (
+    'You are an AI assistant whose name is Qwen.\n'
+    'Capabilities and tools that Qwen can possess.\n'
+    '- Inner thoughts: disabled.\n'
+    '- Web search: disabled.\n'
+    '- Calculator: disabled.\n'
+    '- Equation solver: disabled.\n'
+    '- Text-to-image: disabled.\n'
+    '- Image edition: disabled.\n'
+    '- Text-to-speech: disabled.\n'
+    '<|Human|>: {input}<eoh>')
+
+text = '请给我介绍五个上海的景点'
+inputs = tokenizer(prompt_template.format(input=text), return_tensors='pt')
+inputs = inputs.to(model.device)
+pred = model.generate(**inputs, generation_config=gen_config, stopping_criteria=[StopWordStoppingCriteria(tokenizer, '<eom>')])
+print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True))
+"""
+好的，以下是五个上海的景点介绍：
+1. 上海博物馆：上海博物馆是中国最大的综合性博物馆之一，收藏了大量的历史文物和艺术品，包括青铜器、陶瓷、书画、玉器等。
+2. 上海城隍庙：上海城隍庙是上海最古老的庙宇之一，建于明朝，是上海的标志性建筑之一。庙内有各种神像和文物，是了解上海历史文化的好去处。
+3. 上海科技馆：上海科技馆是一座集科技、文化、教育为一体的综合性博物馆，展示了各种科技展品和互动体验项目，适合全家人一起参观。
+4. 上海东方明珠塔：上海东方明珠塔是上海的标志性建筑之一，高468米。游客可以乘坐高速电梯到达观景台，欣赏上海的美景。
+5. 上海迪士尼乐园：上海迪士尼乐园是中国第一个迪士尼主题公园，拥有各种游乐设施和表演节目，适合全家人一起游玩。
+"""
+```
+
+### Usage with XTuner CLI
+
+#### Installation
+
+```shell
+pip install xtuner
+```
+
+#### Chat
+
+```shell
+xtuner chat hf Qwen/Qwen-7B --adapter xtuner/Qwen-7B-qlora-moss-003-sft --bot-name Qwen --prompt-template moss_sft --with-plugins calculate solve search --command-stop-word "<eoc>" --answer-stop-word "<eom>"
+```
+
+#### Fine-tune
+
+Use the following command to quickly reproduce the fine-tuning results.
+
+```shell
+NPROC_PER_NODE=8 xtuner train qwen_7b_qlora_moss_sft_all_gpu8_e2  # Recommended!
+xtuner train qwen_7b_qlora_moss_sft_all_e1
+```