Introduction to ShenNong-TCM-LLM: A Large Language Model for Traditional Chinese Medicine
Overview
ShenNong-TCM-LLM, named after the legendary ancient Chinese figure Shennong, is a pioneering project that delves into the fusion of modern language technology with the rich heritage of Traditional Chinese Medicine (TCM). The project represents the first major language model dedicated specifically to TCM, aimed at enhancing the model’s knowledge in this field and improving its capability to answer medical queries related to traditional practices.
Why ShenNong-TCM?
In recent years, large language models (LLMs) like ChatGPT and GPT-4 have demonstrated remarkable capabilities that echo those of artificial general intelligence (AGI). These models have transformed various domains by effectively processing and understanding natural language. Given the extensive attention these models have received, the developers of ShenNong-TCM envisioned a similar transformative application within the realm of Traditional Chinese Medicine.
The Foundation of ShenNong-TCM
ShenNong-TCM is based on LlaMA, a foundational model, and fine-tuned using LoRA (Low-Rank Adaptation) techniques to enhance its performance. The core data used for training includes the ShenNong_TCM_Dataset and ChatMed_TCM_Dataset, which draw insights from an open-source TCM knowledge graph. This method employs an innovative entity-centric self-instruction approach to generate instructions related to TCM by leveraging the capabilities of ChatGPT.
Complementary Projects
In addition to ShenNong-TCM, several other medical LLM projects are worth noting for their contributions to healthcare technology:
-
ChatMed-Consult: This model uses a substantial dataset of online medical consultations to refine a Chinese-focused LlaMA-7b model. It integrates additional linguistic features and employs LoRA-efficient tuning.
-
ChatMed-MT: A multi-turn dialogue version of ChatMed-Consult, which aims to enhance the empathetic and detailed nature of responses, thus improving user experience during interactions.
-
PromptCBLUE: It adapts the CBLUE benchmark for a prompt learning mode, aiming to create a comprehensive evaluation standard for Chinese medical knowledge and text processing capabilities in LLMs.
Practical Applications and Advantages
ShenNong-TCM provides nuanced and compassionately structured responses to medical inquiries, better equipped than its predecessors like the general Chinese LlaMA-7b model. Besides offering enriched and practical advice, it can recommend specific TCM remedies or herbal medicines based on presented symptoms, making its responses more actionable and relatable.
Technical Insights
The self-instruction method by ShenNong-TCM involves generating instructions focused on core entities within a vertical domain like TCM. Developers interested in applying this technique to their datasets can utilize the provided script to perform entity-centric self-instructions.
Community and Development
The project continues to evolve, with iterative updates to improve model weights and expand dataset quality. The developers encourage academic and non-commercial use, stressing that ShenNong-TCM should not replace professional medical advice.
Acknowledgments
This project builds upon the foundation of several open-source projects and extends gratitude to the respective developers and contributors. Notably, projects such as LlaMA, Stanford Alpaca, and Chinese-LlaMA-Alpaca have been instrumental in the development of ShenNong-TCM.
By aligning modern AI innovations with ancient medical practices, ShenNong-TCM aspires to be a bridge between technology and the rich tapestry of Traditional Chinese Medicine, offering insightful interactions and solutions in this specialized domain.