llama-trl
This project demonstrates the use of Proximal Policy Optimization (PPO) combined with Transformer Reinforcement Learning (TRL) and Low-Rank Adaptation (LoRA) for the fine-tuning of large language models. By leveraging Parameter-Efficient Fine-Tuning (PEFT), the project seeks to optimize models efficiently. It utilizes instruction data sourced from GPT-4-LLM to advance model training. The setup includes installing dependencies followed by a three-step process: supervised fine-tuning, reward model training, and PPO-based tuning. This structured approach supports comprehensive model optimization for advanced AI and machine learning applications.
