Introduction to FastEdit
Overview
FastEdit is an innovative project designed to make the process of updating large language models (LLMs) quick and efficient—achieving this in under ten seconds. The main goal is to inject new and customized information into these models, ensuring they are up-to-date with minimal effort. By leveraging a simple command, developers can now efficiently update models with fresh facts or data.
Supported Models
FastEdit caters to a variety of popular LLMs, including:
- GPT-J: A 6 billion parameter model known for its powerful text generation capabilities.
- LLaMA & LLaMA-2: Models with 7 and 13 billion parameters, suitable for high-quality language tasks.
- BLOOM: An extensive 7.1 billion parameter multilingual model.
- Falcon: A robust model with 7 billion parameters.
- Baichuan: Offers 7 and 13 billion parameter options for diverse applications.
- InternLM: A 7 billion parameter model designed for flexible language processing tasks.
Key Algorithm
The project utilizes the Rank-One Model Editing (ROME) algorithm to execute efficient model updates without compromising the performance of other functionalities.
Requirements
To work with FastEdit, users must ensure:
- A system with Python 3.8+ and PyTorch 1.13.1+
- Required libraries: Transformers, Datasets, Accelerate, sentencepiece, and fire
Hardware Specifications
- For LLaMA 7B in FP16 mode: Requires 24GB of RAM and operates at 7 seconds per iteration
- For LLaMA 13B in FP16 mode: Requires 32GB of RAM with a speed of 9 seconds per iteration
Getting Started
Data Preparation
To embed new information, users must prepare data in a specific JSON format. For example, updating information about the UK prime minister would look like this:
[
{
"prompt": "The prime minister of the {} is",
"subject": "UK",
"target": "Rishi Sunak",
"queries": []
}
]
In this setup, "prompt" represents the statement with a placeholder for the subject, "subject" is the entity to be updated, and "target" reflects the new data.
Installation
To start using FastEdit, follow these steps:
Clone the repository and set up the environment:
git clone https://github.com/hiyouga/FastEdit.git
conda create -n fastedit python=3.10
conda activate fastedit
cd FastEdit
pip install -r requirements.txt
Alternatively, install via pip:
pip install pyfastedit
Model Editing
Use the following command to perform model updates:
CUDA_VISIBLE_DEVICES=0 python -m fastedit.editor \
--data data/example.json \
--model EleutherAI/gpt-j-6b \
--config gpt-j-6b \
--template default
Demonstrating Effectiveness
The tool allows for editing LLMs as demonstrated with the Ziya-LLaMA-13B-v1 model. By using data inputs, models can be fine-tuned for multilingual capabilities, replacing outdated facts with current information.
Example results before and after editing:
- Before Edit: "The prime minister of the United Kingdom is Boris Johnson."
- After Edit: "The prime minister of the United Kingdom is Rishi Sunak."
Future Developments
Planned enhancements include:
- Implementing the MEMIT algorithm for bulk information updates
- Utilizing NER models for automated subject and target identification
- Improving the editing process for instruction-following models without degrading performance
Conclusion
FastEdit offers a swift and reliable solution for updating LLMs with the latest information, supporting a range of models and ensuring that developers can effortlessly maintain the relevance of their language models.