LESS
LESS introduces a method for selecting influential data to enhance targeted instruction tuning, improving model performance. The process includes warmup training, creating a gradient datastore, and selecting data specific to tasks. It utilizes datasets like Flan v2, COT, Dolly, and Open Assistant, with evaluation on MMLU, TydiQA, and BBH. Suitable for refining machine learning model efficiency. Explore detailed implementation and evaluation for performance enhancements.
