--- license: other library_name: peft tags: - llama-factory - lora - generated_from_trainer base_model: Qwen/Qwen1.5-1.8B-Chat model-index: - name: train_2024 results: [] --- ## Model description A Traditional Chinese Medicine large language model, inspired by the wisdom of the eminent representative of ancient Chinese medical scholars, Zhang Zhongjing. This model aims to illuminate the profound knowledge of Traditional Chinese Medicine, bridging the gap between ancient wisdom and modern technology, and providing a reliable and professional tool for the Traditional Chinese Medical fields. However, all generated results are for reference only and should be provided by experienced professionals for diagnosis and treatment results and suggestions. It is recommended to use [colab](https://colab.research.google.com/drive/1DCPomUsfTxqkqxKpK-AIGvBSPbkOm7R3#scrollTo=jsn4szdjdtmF) for free GPU inference. 推荐使用[colab](https://colab.research.google.com/drive/1DCPomUsfTxqkqxKpK-AIGvBSPbkOm7R3#scrollTo=jsn4szdjdtmF)免费GPU推理。 ## Data Source The data for this model is sourced from the non-profit organization, Fulture Medicine Philosophy (Fulphil). Nearly 50 Traditional Chinese Medicine and integrated Chinese-Western Medicine clinical physicians from Beijing University of Chinese Medicine, Shanghai University of Traditional Chinese Medicine, and Shandong University of Traditional Chinese Medicine contributed data and annotations across multiple disciplines. ## Funding Acknowledgement This project is generously funded by family sponsorship. Special thanks to my father, Piyao Kang, and to Xiuhua Li and my girlfriend, Sunsi Wu, for their strong support and assistance. The funds have been utilized in data cleaning, computational power rental, and consultation with domain experts. Thank you all! ## Intended uses & limitations Academic and Personal Use. You may use the Work for academic research and personal use without restriction. Commercial Use. You may not use the Work for commercial purposes without prior written authorization from the Contributor(s). Any commercial use authorized by the Contributor(s) must not involve charging fees above the model inference cost without express written permission from the Contributor(s). Medical Application. The Work is provided for academic research purposes only and not for commercial use. It must not be used in clinical practice or in any scenario with potential medical intent without permission. The capabilities of this Traditional Chinese Medicine (TCM) Language Model, including syndrome classification and prescription generation, are experimental and not intended for clinical diagnosis or treatment. Outputs are for internal reference and testing only and should not be considered as medical advice. All medical diagnoses and treatments should be performed by experienced physicians through a standardized clinical process. Distribution. Redistribution of the Work or derivative works must comply with all the terms and conditions of this License. ## Training data #### 1.1 Multi-task Therapeutic Behavior Decomposition Instruction Construction Strategy Human memory and understanding require the construction of various scenarios and stories to implicitly encode knowledge information. The clarity of memory depends on the duration and richness of the learning process. Interleaved learning, spaced practice, and diversified learning can enhance the consolidation of knowledge, thereby forming a deep understanding of domain knowledge. Our approach is to learn from the process of human memory knowledge, use professional tables, leverage the language representation capabilities of large language models, strictly set specific prompt templates, so that the model can generate 15 scenarios based on the table data of Chinese medicine gynecology prescriptions, including patient therapeutic story, diagnostic analysis, diagnosis treatment expected result, formula function, interactive story, patient therapeutic story, narrative medicine, tongue & pulse, therapeutic template making, critical thinking, follow up, prescription, herb dosage, case study, real-world problem, disease mechanism, etc., to promote the model's reasoning ability for prescription data and diagnostic thinking logic. ``` { "instruction": "我对三元汤的全过程很好奇，能否从简介、病历、症状、诊断和治疗，以及结果讨论等方面给我详细介绍？", "input": "", "output": "三元汤是一种古老的中药处方，最早出现在《保命集》卷下。它由多种中药组成，主要用于治疗产后日久虚劳，脉浮疾等症状。患者为产后日久虚劳，脉浮疾。脉浮疾。根据李女士的病史和症状，我决定使用三元汤来治疗她的病症。我按照处方中的药物组成和用法用量，将柴胡、黄芩、人参、半夏、甘草、川芎、芍药、熟地黄、当归等中药煎煮后，让李女士每天服用两次。在治疗过程中，我还注意了李女士的饮食和休息，让她多吃一些易于消化的食物，保证充足的睡眠和休息。经过一个月的治疗，李女士的症状得到了明显的改善。她的精神状态更加稳定，食欲也有所增加，头晕乏力的症状也减轻了很多。我认为，三元汤的治疗效果主要得益于其中的多种中药的协同作用，能够调节李女士的身体机能，增强体质，提高免疫力。同时，我也注意到，李女士的饮食和休息也对治疗效果有一定的影响。因此，在使用中药治疗疾病时，我们也应该注意患者的全面治疗，包括饮食、休息等方面。" } ``` #### 1.2 Regular TCM Instruction Data Construction Strategy In addition, we have also added instructions based on the content of Chinese medicine ancient books, noun explanations, symptom synonyms, antonyms, syndromes, symptoms, treatment methods, etc. In order to form a control experiment, we only use one instruction template to represent data for this part, and the number of this part of the data is 80,000, which is significantly more than the number of instructions constructed by the above strategy. The following is the specific number of instructions and tokens information. Data Source and Instruction Quantity Table: ``` { "instruction": "请回答以下有关于中医疾病名词解释的相关问题：", "input": "（肺风）粉刺属于哪个分类？", "output": "因肺风、胃热或肝瘀所致。以面及背部见黑头或白头粉刺、丘疹、脓疱、结节、囊肿及疤痕为主要表现的皮肤疾病。" } ``` # Train Details & Inference Capability Statement Our model, a meticulously fine-tuned version of Qwen1.5-1.8B-Chat, has been optimized for high-speed inference on a Tesla T4 graphics processing unit (GPU). This enhancement was achieved through extensive training on our exclusive medical datasets, ensuring the model's proficiency in understanding and generating responses relevant to the medical field, particularly in the domain of Traditional Chinese Medicine (TCM). ## Disclaimer This research is for academic research use only, commercial use is not allowed without permission, and it is not to be used in medical scenarios or scenarios with potential medical intent for clinical practice. This large language model for Traditional Chinese Medicine is still in the laboratory testing stage. The emerging syndrome classification and prescription generation capabilities at this stage are still rudimentary, and it does not yet have a highly reliable clinical diagnostic and therapeutic capability for gynecology and other clinical specialties. The output results are for internal reference testing only. Real medical diagnosis and decision-making still need to be issued by experienced physicians through a strictly regulated diagnostic and therapeutic process. ## Collaboration Data processing and annotation is one of the important steps in training the model. We sincerely welcome Traditional Chinese Medicine practitioners with strong TCM thinking and innovative spirit to join us. We will also declare corresponding data contributions. We look forward to the day when we can achieve a reliable General Artificial Intelligence for Traditional Chinese Medicine, allowing the ancient Chinese medicine to blend with modern technology and shine anew. This is also the ultimate mission of this project. If interested, please send an email to 21110860035@m.fudan.edu.cn. ## Team Introduction This project is jointly guided by Professor Zhang Wenqiang from Fudan University, Postdoctoral Fellow Wang Yan from Fudan University, and Professor Wang Haofen from Tongji University. The team effort includes Kang Yanlan, Chang Yang, and Fu Jiyuan from Fudan University's [ROI Lab](https://www.fudanroilab.com/), as well as Xing Haozhe in completing this project. Special thanks to the medical team from the Longhua Hospital affiliated with Shanghai University of Traditional Chinese Medicine, including Wu Sunsi, Ma Qingshan, Fang Yide, Chen Yue, Jiao Yuqi, Liu Xiyu, and Zhao Xue for their valuable data support and manual assessments. ## Citation If you find this work useful in your research, please cite our repository: ``` @misc{CMLM-ZhongJing, author = {Liu Lin Ju Shi}, title = {CMLM-ZhongJing-2-1_8b: A State-of-the-Art Edge Computing Language Model for Traditional Chinese Medicine}, year = {2023}, publisher = {FulPhil-MediPhilosophy Future}, journal = {GitHub Repository}, howpublished = {\url{https://github.com/pariskang/CMLM-ZhongJing}} ```