data/xtuner/docs/en/user_guides/finetune.md

Fine-tune the pretrained LLMs

QLoRA Fine-tune InternLM

• InternLM-7B, oasst1

  xtuner train internlm_7b_qlora_oasst1_e3
  

• InternLM-7B, Arxiv Gentitle

  xtuner train internlm_7b_qlora_arxiv_gentitle_e3
  

• InternLM-7B, Colorist

  xtuner train internlm_7b_qlora_colorist_e5
  

• InternLM-7B, Coder

  xtuner train internlm_7b_qlora_code_alpaca_e3
  

• InternLM-7B, SQL

  xtuner train internlm_7b_qlora_sql_e3
  

• InternLM-7B, Lawyer

  xtuner train internlm_7b_qlora_lawyer_e3
  

• InternLM-7B, Open-Platypus

  xtuner train internlm_7b_qlora_open_platypus_e3
  

• InternLM-7B, Alpaca-enzh

  xtuner train internlm_7b_qlora_alpaca_enzh_e3
  

QLoRA Fine-tune Llama2

Don't forget to use huggingface-cli login and input your access token first to access Llama2! See here to learn how to obtain your access token.

• Llama2-7B, MOSS-003-SFT (plugins!)

  NPROC_PER_NODE=8 xtuner train llama2_7b_qlora_moss_sft_all_e2_gpu8  # Recommended!
  xtuner train llama2_7b_qlora_moss_sft_all_e1
  

• Llama2-7B, Arxiv Gentitle

  xtuner train llama2_7b_qlora_arxiv_gentitle_e3
  

• Llama2-7B, Colorist

  xtuner train llama2_7b_qlora_colorist_e5
  

QLoRA Fine-tune Qwen

• Qwen-7B, MOSS-003-SFT (plugins!)

  NPROC_PER_NODE=8 xtuner train qwen_7b_qlora_moss_sft_all_e2_gpu8  # Recommended!
  xtuner train qwen_7b_qlora_moss_sft_all_e1
  

• Qwen-7B, oasst1

  xtuner train qwen_7b_qlora_oasst1_e3
  

• Qwen-7B, Arxiv Gentitle

  xtuner train qwen_7b_qlora_arxiv_gentitle_e3
  

• Qwen-7B, Alpaca-enzh

  xtuner train qwen_7b_qlora_alpaca_enzh_e3
  

QLoRA Fine-tune Baichuan

• Baichuan-7B, oasst1

  xtuner train baichuan_7b_qlora_oasst1_e3
  

• Baichuan-7B, Arxiv Gentitle

  xtuner train baichuan_7b_qlora_arxiv_gentitle_e3
  

• Baichuan-7B, Alpaca-enzh

  xtuner train baichuan_7b_qlora_alpaca_enzh_e3