#memory optimization

FSRS4Anki introduces a modern spaced-repetition scheduler and optimizer to improve Anki's scheduling. Leveraging the Free Spaced Repetition Scheduler algorithm, it replaces Anki’s default model with a machine learning approach that adjusts to individual memory patterns. The project ensures compatibility with various Anki add-ons and offers detailed guides for configuration on different Anki versions. Community contributions further refine the system, while developer resources support FSRS implementation in diverse projects.

StructLinq is a C# library designed to significantly reduce memory allocation and enhance application performance through struct-based LINQ. By utilizing IRefStructEnumerable, it provides optimized operations for complex struct elements, integrating seamlessly with BCL classes through various extensions and converters. The library supports zero allocation patterns and a broad range of LINQ methods, making it suitable for both new and existing projects. Available on NuGet, StructLinq allows developers to focus on performance-oriented programming without compromising on compatibility.

KIVI significantly optimizes LLM memory usage and enhances throughput by using a tuning-free 2bit quantization strategy for KV caches. This method decreases memory needs by 2.6 times, enabling larger batch sizes and improving throughput up to 3.47 times. Compatible with models like Llama-2 and Mistral, KIVI maintains model quality while solving speed and memory challenges in inference tasks. Discover new features and examples on GitHub, including improvements for HuggingFace Transformers and support for the Mistral models.

This project demonstrates how established methods can expand language models to manage contexts as long as 1 million tokens using efficient strategies such as sequence parallelism, Deepspeed zero3 offload, and flash attention. It delivers comprehensive training scripts, supports various parallel approaches, and highlights significant improvements in both perplexity and 'needle-in-a-haystack' evaluations for Llama2 models.