fsdp_qlora
The project focuses on efficient large language model training using Quantized LoRA with FSDP, supporting platforms like Axolotl. Installation is tailored for CUDA versions up to 12.1, emphasizing low memory usage and mixed precision. Various training types are available, suitable for models like Llama-2 70B. This alpha aims to refine techniques for better model handling.
