Dive into LLMS: Project Overview
Introduction
"Dive into LLMS" is an educational project designed to introduce learners to large language models (LLMs) through hands-on programming practice. Originating from the spring 2024 course "Artificial Intelligence Security Technology" (NIS3353) at Shanghai Jiao Tong University and developed further by the educator Zhang Zhuosheng, this project aims to provide foundational programming references for those interested in the field of large models. The practical approach helps students quickly get acquainted with LLMs to enhance their course design or their research endeavors.
Project Motivation
The main motivation behind the "Dive into LLMS" project is to offer a beginner-friendly guide to working with large models. By engaging in straightforward practical exercises, students can swiftly become familiar with the intricacies of large models. This preparatory experience serves as a solid foundation, allowing learners to explore more advanced topics or integrate LLMs into their academic pursuits more effectively.
Tutorial Content
The project is structured around a comprehensive set of tutorials covering various aspects of working with large language models:
-
Fine-Tuning and Deployment: This tutorial provides guidance on selecting a suitable pre-trained model and fine-tuning it for specific tasks, followed by deploying it as a user-friendly demo.
-
Prompt Learning and Chain of Thought: Explains how to call APIs and interpret responses from large models, addressing how models handle various queries.
-
Knowledge Editing: Focuses on methods and tools to manipulate what language models know, including editing specific knowledge and verifying the edited model.
-
Model Watermarking: Techniques to embed invisible watermarks in the content generated by language models.
-
Jailbreak Attacks: Understanding the vulnerabilities of large models by exploring jailbreak attacks to enhance the models' security.
-
Multimodal Models: Examines the integration of multimodal capabilities into large language models to improve their comprehensive world understanding and generative abilities.
-
Smart Agents and Safety: Investigates the future role of large model intelligent agents in open environments and their awareness of risks.
Project Status and Contributions
Currently, "Dive into LLMS" is a work-in-progress project, actively welcoming contributions. Contributors are encouraged to submit pull requests (PRs) or participate in discussions via issues if they encounter errors or have suggestions for improvements.
Disclaimer
The insights and techniques shared in this project are derived from the collective experiences and routine research of its contributors. It should be noted that these methods are shared for educational purposes only and should not be relied upon as infallible. Contributors are open to feedback and encourage communication should there be any content-related concerns.
Contributors
The success of this project is owed to the contributions and support of a team including:
- Tongxin Yuan, Shanghai Jiao Tong University
- Xinbei Ma, Shanghai Jiao Tong University
- Zhiwei He, Shanghai Jiao Tong University
- Wei Du, Shanghai Jiao Tong University
- Haodong Zhao, Shanghai Jiao Tong University
- Hao Fei, National University of Singapore
The Star History Chart showcases the growing popularity and user engagement with the project over time.