Project Introduction: torchtune
Introduction
Torchtune is a user-friendly PyTorch library designed to simplify the process of creating, fine-tuning, and experimenting with large language models (LLMs). This library provides efficient access to various models in the Llama, Gemma, Mistral, Phi, and Qwen family, allowing users to conduct experiments with the latest innovations in model architectures.
Key Features
- Model Implementations: Torchtune offers PyTorch implementations of popular models such as Llama, Gemma, and Qwen, among others. The library continuously expands with more models may be added based on user interest and demand.
- Flexible Training Recipes: The library includes a wide variety of training recipes. These allow users to engage in full fine-tuning, or opt for more specific approaches like LoRA (Low-Rank Adaptation), QLoRA, or Quantization-Aware Training. This flexibility ensures that both novice and experienced developers can find the right method to refine their models.
- Scalability and Performance: Torchtune emphasizes memory efficiency and performance optimization. By leveraging the latest PyTorch APIs, it enhances the scalability and overall performance of large-scale models.
- Configurable YAML Files: Torchtune uses YAML configuration files, making it simple for users to adjust and experiment with training, evaluation, and quantization settings.
- Dataset and Prompt Support: The library includes built-in support for various dataset formats and prompt templates, enabling developers to customize their experiments with ease.
Supported Models
The library currently supports an extensive range of models segmented by their size and function. Notably:
- Llama Series: Ranging from Llama 2 to Llama 3.2, supporting numerous sizes from 1B to 405B parameters.
- Specialty Models: Includes models like Code-Llama2 for coding tasks, and vision models like Llama3.2-Vision.
Finetuning Recipes
Torchtune accommodates different finetuning methods suitable for running on multiple devices. Recipes include full finetuning for thorough model adjustments, LoRA, and advanced techniques like QLoRA. Configurations for these recipes are provided to help users get started quickly.
Performance Insights
Developers can refer to the library's detailed documentation regarding memory requirements and model training speeds. This helps in selecting suitable hardware setups for executing different models efficiently.
Installation and Setup
Installing torchtune is straightforward. It is compatible with both stable and nightly releases of PyTorch. Users can integrate torchtune with associated libraries like torchvision and torchao to enable multimodal finetuning and leverage the latest quantization techniques.
Getting Started
Torchtune offers extensive documentation and tutorials to guide users through initial model fine-tuning, evaluation, quantization, and inference processes. The library also provides instructions for downloading different models and running customized fine-tuning recipes.
Community and Integration
The torchtune community encourages collaborative development. Integration with popular tools like Hugging Face Hub, EleutherAI's LM Eval Harness, and PyTorch's FSDP2 ensures that torchtune stays versatile and robust for varied machine learning tasks. Additionally, contributions from the community, like adding new models or finetuning recipes, are welcomed and highlighted.
License and Citation
Torchtune is available under a BSD 3 license. Users are encouraged to cite torchtune in their work, acknowledging its utility and community effort in enhancing PyTorch's finetuning capabilities.
This comprehensive toolkit aims to empower developers to innovate and optimize large language models with ease, while fostering a supportive and dynamic community around LLM advancements.