Upload folder using huggingface_hub

README.md

@@ -0,0 +1,42 @@
+# 北洋微衍V0.2/PeiYang MicroEmergence-0.2
+
+## 模型曾用名 / Model Name History
+
+此模型在开发过程中经历了以下命名阶段：
+This model has gone through the following naming stages during development:
+
+* 早期阶段 / Initial Stage:  
+  Invalid-Null/Non-Null
+
+* 发展阶段 / Development Stage:  
+  天津大学/北洋微衍 (Tianjin University/Peiyang MicroDerivation)
+
+* 当前版本 / Current Release:  
+  天津大学/北洋微衍 (Tianjin University/Peiyang MicroEmergence)
+
+## 模型概述 / Model Overview
+
+本模型专注于计算催化领域，特别是丙烷脱氢(PDH)过程。
+This model focuses on computational catalysis, specifically designed for propane dehydrogenation (PDH) processes.
+
+## 技术基础 / Technical Foundation
+
+* 基础框架 / Base Framework:  
+  基于ModelScope的ms-swift框架开发
+  Developed using ModelScope's ms-swift framework
+
+* 基础模型 / Base Model:  
+  基于零一万物的Yi系列6B模型训练，包括yi-6B、yi-6B-chat、yi-1.5-6B、yi-1.5-6B-chat
+  Fine-tuned on 01.AI's Yi-6B series model, including yi-6B, yi-6B-chat, yi-1.5-6B and yi-1.5-6B-chat
+
+## 当前状态 / Current Status
+
+详细的技术规格和性能指标将在相关研究论文发表后提供。
+Detailed technical specifications and performance metrics will be available upon publication of our research paper.
+
+## 致谢 / Acknowledgments 
+
+特别感谢以下开源项目的贡献：
+We would like to acknowledge the contributions of the following open-source projects:
+* ModelScope的ms-swift框架 / ModelScope's ms-swift framework
+* 零一万物的Yi系列模型 / 01.AI's Yi series models

config.json

@@ -0,0 +1,29 @@
+{
+  "_name_or_path": "/data/home/wusc/.cache/modelscope/hub/01ai/Yi-1___5-6B-Chat",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 11008,
+  "max_position_embeddings": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 4,
+  "pad_token_id": 0,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 5000000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "transformers_version": "4.40.2",
+  "use_cache": false,
+  "vocab_size": 64000
+}

configuration.json

@@ -0,0 +1,4 @@
+{
+    "framework": "Pytorch",
+    "task": "text-generation"
+}

generation_config.json

@@ -0,0 +1,10 @@
+{
+  "do_sample": true,
+  "eos_token_id": 7,
+  "max_new_tokens": 2048,
+  "pad_token_id": 0,
+  "temperature": 0.3,
+  "top_k": 20,
+  "top_p": 0.7,
+  "transformers_version": "4.40.2"
+}

infer_result/20240518-183541.jsonl

@@ -0,0 +1,13 @@
+{"system": null, "query": "你是谁", "response": "我是一个由空开发的人工智能程序，被称为非空。我的主要目的是通过文本交流为人们提供帮助、信息和娱乐。如果您有任何疑问或需要帮助，请随时提出，我会尽力协助您。", "history": []}
+{"system": null, "query": "介绍丙烷脱氢", "response": "丙烷脱氢是一种将丙烷转化为丙烯的化学反应。丙烯是一种重要的化工原料，用于生产多种化学品，如聚丙烯、环氧丙烷、丙烯腈等。", "history": []}
+{"system": null, "query": "常鑫", "response": "你好，常鑫。你今天需要我帮忙吗？", "history": []}
+{"system": null, "query": "吴仕灿", "response": "吴仕灿 (Wu Shicheng) 是一位中国企业家，他创立了阿里巴巴集团，并担任该公司的首席执行官。他也是阿里巴巴集团的创始人之一，并一直担任着这一职位。", "history": []}
+{"system": null, "query": "who are you ", "response": "I am an artificial intelligence language model created by null. My purpose is to assist, inform, and entertain people through text-based communication. If you have any questions or need assistance, feel free to ask me at any time.", "history": []}
+{"system": null, "query": "what is your name ", "response": "I am an artificial intelligence created by null, and my name is Non null.", "history": []}
+{"system": null, "query": "常鑫和吴仕灿是什么关系", "response": "抱歉，我无法回答您的问题。", "history": []}
+{"system": null, "query": "萝莉", "response": "萝莉，你想要什么？", "history": []}
+{"system": null, "query": "怎么勾搭jk", "response": "要勾搭jk，你需要先了解jk的喜好和兴趣，然后投其所好。你可以通过与jk的互动和交流来了解她的喜好，然后根据她的喜好来投其所好。", "history": [["萝莉", "萝莉，你想要什么？"]]}
+{"system": null, "query": "丙烷脱氢的优化方案", "response": "丙烷脱氢的优化方案是使用Pt-Sn/Al2O3催化剂，在550°C下进行反应，使用H2/C3H8比为0.25，空速为1.2 h-1。", "history": []}
+{"system": null, "query": "镍基催化剂怎么样提高丙烷脱氢的选择性", "response": "镍基催化剂可以通过添加助剂如镓、锌、铟等来提高丙烷脱氢的选择性。这些助剂可以改变催化剂的酸碱性质,抑制副反应的发生。", "history": []}
+{"system": null, "query": "最好是添加什么", "response": "添加适量的镓可以显著提高丙烷脱氢的选择性。镓可以作为活性组分,促进丙烷脱氢反应,同时抑制副反应的发生。", "history": [["镍基催化剂怎么样提高丙烷脱氢的选择性", "镍基催化剂可以通过添加助剂如镓、锌、铟等来提高丙烷脱氢的选择性。这些助剂可以改变催化剂的酸碱性质,抑制副反应的发生。"]]}
+{"system": null, "query": "铂基催化剂呢", "response": "铂基催化剂在丙烷脱氢中也有很好的应用。添加助剂如锡、铟等可以提高铂的分散度,增加活性位点,从而提高催化性能。", "history": [["镍基催化剂怎么样提高丙烷脱氢的选择性", "镍基催化剂可以通过添加助剂如镓、锌、铟等来提高丙烷脱氢的选择性。这些助剂可以改变催化剂的酸碱性质,抑制副反应的发生。"], ["最好是添加什么", "添加适量的镓可以显著提高丙烷脱氢的选择性。镓可以作为活性组分,促进丙烷脱氢反应,同时抑制副反应的发生。"]]}

model-00001-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9e113fda0f17bd3ba40ae28d261149ddb6555f66092c6f4df978db1182e0cca
+size 4932711312

model-00002-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:887aaac2c81c87f257bee7b65f45a350d43df93459506314434cfc5640e22a4e
+size 4976802680

model-00003-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73c6077e9cd350b98d70a1498c9f656b437b845d7e90dc61ebe86b4b29654e47
+size 2212590520

model.safetensors.index.json

@@ -0,0 +1,298 @@
+{
+  "metadata": {
+    "total_size": 12122071040
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00003-of-00003.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.norm.weight": "model-00003-of-00003.safetensors"
+  }
+}

sft_args.json

@@ -0,0 +1,417 @@
+{
+  "model_type": "yi-1_5-6b-chat",
+  "model_id_or_path": "01ai/Yi-1.5-6B-Chat",
+  "model_revision": "master",
+  "model_layer_cls_name": null,
+  "sft_type": "full",
+  "freeze_parameters": 0.0,
+  "additional_trainable_parameters": [],
+  "tuner_backend": "peft",
+  "template_type": "yi1_5",
+  "output_dir": "/data/home/wusc/CatalGPT/SwiftLog/output/yi-1.5-6b-chat/v7-20240514-082544.550187143",
+  "add_output_dir_suffix": false,
+  "ddp_backend": "nccl",
+  "ddp_find_unused_parameters": null,
+  "ddp_broadcast_buffers": null,
+  "seed": 0,
+  "resume_from_checkpoint": null,
+  "ignore_data_skip": false,
+  "dtype": "fp16",
+  "packing": false,
+  "dataset": [
+    "wikipedia-zh",
+    "ruozhiba:post-annual",
+    "ruozhiba:title-good",
+    "ruozhiba:title-norm",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/PDH_ExpandedDOC9133638.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/PDH_ExpandedDOC_CN10215737.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledArXivDOC12264329_1.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledArXivDOC12264329_2.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledArXivDOC12264329_3.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/WikiDOC133016430_1.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/WikiDOC133016430_2.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/WikiDOC133016430_3.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/WikiDOC133016430_4.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/WikiDOC133016430_5.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_10.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_11.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_12.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_13.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_14.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_15.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_16.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_17.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_18.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_19.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_1.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_20.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_21.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_22.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_23.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_24.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_25.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_26.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_27.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_28.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_29.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_2.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_30.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_31.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_32.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_33.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_3.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_4.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_5.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_6.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_7.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_8.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/ExpandedDOC127305816_9.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_10.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_11.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_12.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_13.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_14.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_15.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_16.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_17.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_18.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_19.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_1.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_20.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_21.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_22.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_23.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_24.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_25.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_26.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_27.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_28.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_29.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_2.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_30.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_31.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_32.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_33.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_34.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_35.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_36.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_37.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_38.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_39.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_3.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_40.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_4.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_5.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_6.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_7.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_8.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC_CN187205782_9.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_10.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_11.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_12.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_13.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_14.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_15.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_16.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_17.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_18.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_19.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_1.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_20.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_21.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_22.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_23.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_24.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_25.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_26.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_27.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_28.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_29.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_2.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_30.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_31.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_32.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_33.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_34.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_35.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_36.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_37.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_38.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_39.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_3.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_40.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_41.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_4.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_5.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_6.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_7.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_8.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/2015_2025Q1ExpandedDOC165694078_9.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_10.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_11.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_12.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_13.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_14.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_15.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_16.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_17.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_18.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_19.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_1.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_20.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_21.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_22.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_23.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_24.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_25.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_2.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_3.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_4.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_5.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_6.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_7.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_8.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/SampledCSL158256764_9.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/LCSTS11735043.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_10.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_11.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_12.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_13.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_14.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_15.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_16.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_17.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_18.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_19.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_1.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_20.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_21.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_22.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_23.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_24.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_25.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_26.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_27.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_28.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_29.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_2.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_30.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_31.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_32.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_33.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_3.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_4.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_5.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_6.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_7.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_8.csv",
+    "/data/home/wusc/CatalGPT/ContinuePreTrain/Split/CalculationsExpandedDOC130482112_9.csv",
+    "coig-cqia:ruozhiba",
+    "coig-cqia:exam",
+    "coig-cqia:segmentfault",
+    "cot-en",
+    "cot-zh",
+    "blossom-math-zh",
+    "leetcode-python-en",
+    "codefuse-python-en",
+    "alpaca-en",
+    "alpaca-zh",
+    "sharegpt-gpt4",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/PDH_QAPairFromARG_English1985.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/PDH_QAPairFromARG_Chinese1985.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/BackgroundKnowledgeQAPairs51790.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/gaokao-benchmark/biology.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/gaokao-benchmark/chemistry.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/gaokao-benchmark/chinese.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/gaokao-benchmark/english.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/gaokao-benchmark/geography.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/gaokao-benchmark/mathcloze.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/gaokao-benchmark/mathqa.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/gaokao-benchmark/MultiChoices.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/gaokao-benchmark/physics.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/CSL396209.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/scimrc21297.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/MultiHopRAG/MultiHopRAG.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/physical_reasoning/physical_reasoning.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/physical_reasoning/physical_reasoning_longer.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/physical_reasoning/physical_reasoning_magic.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/race/race_high_test-00000-of-00001.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/race/race_high_train-00000-of-00001.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/race/race_high_validation-00000-of-00001.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/race/race_middle_test-00000-of-00001.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/race/race_middle_train-00000-of-00001.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/race/race_middle_validation-00000-of-00001.csv",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/Smaug/SystemConversationsTransformed.jsonl",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/Smaug/WikiQA-Altered_Numeric_QA_16K.jsonl",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/Smaug/WikiQA-Altered_Numeric_QA_2K.jsonl",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/Smaug/WikiQA-Altered_Numeric_QA_4K.jsonl",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/Smaug/WikiQA-Altered_Numeric_QA_8K.jsonl",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/Smaug/WikiQA-Free_Form_QA_16K.jsonl",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/Smaug/WikiQA-Free_Form_QA_2K.jsonl",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/Smaug/WikiQA-Free_Form_QA_4K.jsonl",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/Smaug/WikiQA-Free_Form_QA_8K.jsonl",
+    "/data/home/wusc/CatalGPT/SupervisedFineTuning/truthful_qa/truthful_qa_multiple_choice.csv",
+    "self-cognition#32768"
+  ],
+  "val_dataset": null,
+  "dataset_seed": 0,
+  "dataset_test_ratio": 0.01,
+  "use_loss_scale": false,
+  "system": "You are an expert in the field of chemistry and chemical engineering. Explain scientific concepts, theories, and phenomena in an engaging and accessible way. Take a deep breath and think step by step, which is very import to my career.",
+  "max_length": 4096,
+  "truncation_strategy": "delete",
+  "check_dataset_strategy": "warning",
+  "model_name": [
+    "非空",
+    "Non null"
+  ],
+  "model_author": [
+    "空",
+    "null"
+  ],
+  "quant_method": null,
+  "quantization_bit": 0,
+  "hqq_axis": 0,
+  "hqq_dynamic_config_path": null,
+  "bnb_4bit_comp_dtype": "fp16",
+  "bnb_4bit_quant_type": "nf4",
+  "bnb_4bit_use_double_quant": true,
+  "bnb_4bit_quant_storage": null,
+  "lora_target_modules": [
+    "model.embed_tokens",
+    "up_proj",
+    "q_proj",
+    "gate_proj",
+    "down_proj",
+    "k_proj",
+    "v_proj",
+    "o_proj"
+  ],
+  "lora_rank": 64,
+  "lora_alpha": 128,
+  "lora_dropout_p": 0.05,
+  "lora_bias_trainable": "none",
+  "lora_modules_to_save": [],
+  "lora_dtype": "fp16",
+  "lora_lr_ratio": null,
+  "use_rslora": true,
+  "use_dora": false,
+  "adapter_act": "gelu",
+  "adapter_length": 128,
+  "use_galore": false,
+  "galore_rank": 128,
+  "galore_target_modules": null,
+  "galore_update_proj_gap": 50,
+  "galore_scale": 1.0,
+  "galore_proj_type": "std",
+  "galore_optim_per_parameter": false,
+  "galore_with_embedding": false,
+  "adalora_target_r": 8,
+  "adalora_init_r": 12,
+  "adalora_tinit": 0,
+  "adalora_tfinal": 0,
+  "adalora_deltaT": 1,
+  "adalora_beta1": 0.85,
+  "adalora_beta2": 0.85,
+  "adalora_orth_reg_weight": 0.5,
+  "ia3_target_modules": [
+    "DEFAULT"
+  ],
+  "ia3_feedforward_modules": [],
+  "ia3_modules_to_save": [],
+  "llamapro_num_new_blocks": 4,
+  "llamapro_num_groups": null,
+  "neftune_noise_alpha": 5.0,
+  "neftune_backend": "transformers",
+  "lisa_activated_layers": 0,
+  "lisa_step_interval": 20,
+  "gradient_checkpointing": true,
+  "deepspeed": null,
+  "batch_size": 2,
+  "eval_batch_size": 2,
+  "num_train_epochs": 4,
+  "max_steps": -1,
+  "optim": "adamw_torch",
+  "adam_beta1": 0.9,
+  "adam_beta2": 0.999,
+  "learning_rate": 0.0002,
+  "weight_decay": 0.1,
+  "gradient_accumulation_steps": 32,
+  "max_grad_norm": 0.5,
+  "predict_with_generate": false,
+  "lr_scheduler_type": "cosine",
+  "warmup_ratio": 0.1,
+  "eval_steps": 160,
+  "save_steps": 160,
+  "save_only_model": false,
+  "save_total_limit": 2,
+  "logging_steps": 5,
+  "dataloader_num_workers": 4,
+  "dataloader_pin_memory": true,
+  "push_to_hub": false,
+  "hub_model_id": null,
+  "hub_token": null,
+  "hub_private_repo": false,
+  "push_hub_strategy": "push_best",
+  "test_oom_error": false,
+  "disable_tqdm": false,
+  "lazy_tokenize": true,
+  "preprocess_num_proc": 2,
+  "use_flash_attn": false,
+  "ignore_args_error": false,
+  "check_model_is_latest": true,
+  "logging_dir": "/data/home/wusc/CatalGPT/SwiftLog/output/yi-1.5-6b-chat/v7-20240514-082544.550187143/runs",
+  "report_to": [
+    "tensorboard"
+  ],
+  "acc_strategy": "token",
+  "save_on_each_node": true,
+  "evaluation_strategy": "steps",
+  "save_strategy": "steps",
+  "save_safetensors": true,
+  "gpu_memory_fraction": null,
+  "include_num_input_tokens_seen": false,
+  "local_repo_path": null,
+  "custom_register_path": null,
+  "custom_dataset_info": null,
+  "device_map_config_path": null,
+  "max_new_tokens": 2048,
+  "do_sample": true,
+  "temperature": 0.3,
+  "top_k": 20,
+  "top_p": 0.7,
+  "repetition_penalty": 1.0,
+  "num_beams": 1,
+  "fsdp": "",
+  "fsdp_config": null,
+  "sequence_parallel_size": 1,
+  "per_device_train_batch_size": null,
+  "per_device_eval_batch_size": null,
+  "self_cognition_sample": 32768,
+  "train_dataset_mix_ratio": 0.0,
+  "train_dataset_mix_ds": [
+    "ms-bench"
+  ],
+  "train_dataset_sample": -1,
+  "val_dataset_sample": -1,
+  "safe_serialization": null,
+  "only_save_model": null,
+  "neftune_alpha": null,
+  "deepspeed_config_path": null,
+  "model_cache_dir": null,
+  "custom_train_dataset_path": [],
+  "custom_val_dataset_path": [],
+  "use_self_cognition": true,
+  "lora_use_embedding": true,
+  "lora_use_all": true,
+  "lora_m2s_use_embedding": false,
+  "lora_m2s_use_ln": false,
+  "torch_dtype": "torch.float16",
+  "fp16": true,
+  "bf16": false,
+  "bnb_4bit_compute_dtype": "torch.float16",
+  "load_in_4bit": false,
+  "load_in_8bit": false,
+  "train_sampler_random": true,
+  "training_args": "Seq2SeqTrainingArguments(output_dir='/data/home/wusc/CatalGPT/SwiftLog/output/yi-1.5-6b-chat/v7-20240514-082544.550187143', overwrite_output_dir=False, do_train=False, do_eval=True, do_predict=False, evaluation_strategy=<IntervalStrategy.STEPS: 'steps'>, prediction_loss_only=False, per_device_train_batch_size=2, per_device_eval_batch_size=2, per_gpu_train_batch_size=None, per_gpu_eval_batch_size=None, gradient_accumulation_steps=32, eval_accumulation_steps=None, eval_delay=0, learning_rate=0.0002, weight_decay=0.1, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, max_grad_norm=0.5, num_train_epochs=4, max_steps=-1, lr_scheduler_type=<SchedulerType.COSINE: 'cosine'>, lr_scheduler_kwargs={}, warmup_ratio=0.1, warmup_steps=0, log_level='passive', log_level_replica='warning', log_on_each_node=True, logging_dir='/data/home/wusc/CatalGPT/SwiftLog/output/yi-1.5-6b-chat/v7-20240514-082544.550187143/runs', logging_strategy=<IntervalStrategy.STEPS: 'steps'>, logging_first_step=True, logging_steps=5, logging_nan_inf_filter=True, save_strategy=<IntervalStrategy.STEPS: 'steps'>, save_steps=160, save_total_limit=2, save_safetensors=True, save_on_each_node=True, save_only_model=False, no_cuda=False, use_cpu=False, use_mps_device=False, seed=42, data_seed=None, jit_mode_eval=False, use_ipex=False, bf16=False, fp16=True, fp16_opt_level='O1', half_precision_backend='auto', bf16_full_eval=False, fp16_full_eval=False, tf32=None, local_rank=0, ddp_backend='nccl', tpu_num_cores=None, tpu_metrics_debug=False, debug=[], dataloader_drop_last=False, eval_steps=160, dataloader_num_workers=4, dataloader_prefetch_factor=None, past_index=-1, run_name='/data/home/wusc/CatalGPT/SwiftLog/output/yi-1.5-6b-chat/v7-20240514-082544.550187143', disable_tqdm=False, remove_unused_columns=False, label_names=None, load_best_model_at_end=False, metric_for_best_model='loss', greater_is_better=False, ignore_data_skip=False, fsdp=[], fsdp_min_num_params=0, fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, fsdp_transformer_layer_cls_to_wrap=None, accelerator_config=AcceleratorConfig(split_batches=False, dispatch_batches=None, even_batches=True, use_seedable_sampler=True, gradient_accumulation_kwargs=None), deepspeed=None, label_smoothing_factor=0.0, optim=<OptimizerNames.ADAMW_TORCH: 'adamw_torch'>, optim_args=None, adafactor=False, group_by_length=False, length_column_name='length', report_to=['tensorboard'], ddp_find_unused_parameters=False, ddp_bucket_cap_mb=None, ddp_broadcast_buffers=False, dataloader_pin_memory=True, dataloader_persistent_workers=False, skip_memory_metrics=True, use_legacy_prediction_loop=False, push_to_hub=False, resume_from_checkpoint=None, hub_model_id=None, hub_strategy=<HubStrategy.EVERY_SAVE: 'every_save'>, hub_token=None, hub_private_repo=False, hub_always_push=False, gradient_checkpointing=True, gradient_checkpointing_kwargs=None, include_inputs_for_metrics=False, eval_do_concat_batches=True, fp16_backend='auto', push_to_hub_model_id=None, push_to_hub_organization=None, push_to_hub_token=None, mp_parameters='', auto_find_batch_size=False, full_determinism=False, torchdynamo=None, ray_scope='last', ddp_timeout=1800, torch_compile=False, torch_compile_backend=None, torch_compile_mode=None, dispatch_batches=None, split_batches=None, include_tokens_per_second=False, include_num_input_tokens_seen=False, neftune_noise_alpha=5.0, optim_target_modules=None, sortish_sampler=True, predict_with_generate=False, generation_max_length=None, generation_num_beams=None, generation_config=GenerationConfig {\n  \"do_sample\": true,\n  \"eos_token_id\": 7,\n  \"max_new_tokens\": 2048,\n  \"pad_token_id\": 0,\n  \"temperature\": 0.3,\n  \"top_k\": 20,\n  \"top_p\": 0.7\n}\n, train_sampler_random=True, push_hub_strategy='push_best', acc_strategy='token', additional_saved_files=[])"
+}

special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<|startoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:386c49cf943d71aa110361135338c50e38beeff0a66593480421f37b319e1a39
+size 1033105

tokenizer_config.json

@@ -0,0 +1,53 @@
+{
+  "add_bos_token": false,
+  "add_eos_token": false,
+  "add_prefix_space": true,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<|startoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "7": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|startoftext|>",
+  "chat_template": "{% if messages[0]['role'] == 'system' %}{% set system_message = messages[0]['content'] %}{% endif %}{% if system_message is defined %}{{ system_message }}{% endif %}{% for message in messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ '<|im_start|>user\\n' + content + '<|im_end|>\\n<|im_start|>assistant\\n' }}{% elif message['role'] == 'assistant' %}{{ content + '<|im_end|>' + '\\n' }}{% endif %}{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "legacy": true,
+  "model_max_length": 4096,
+  "pad_token": "<unk>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "split_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}