#evaluation metrics

Discover COMET's capabilities in improving translation precision via context use and error identification. Recent updates introduce XCOMET models for refined error classification by MQM typology, and DocCOMET for document-level context evaluations. This system supports discourse phenomena and chat translation quality assessments. Learn about the models and their roles in enhancing machine translation evaluations.

The project provides a set of technical metrics for assessing music and audio generative models. This includes tools like Fréchet Distance, Kullback–Leibler divergence, and CLAP score across different sampling rates for more realistic evaluation scenarios, such as long-form stereo content. These metrics accommodate variable-length inputs and favor GPU acceleration for efficiency. The documentation includes clear installation and troubleshooting instructions, making it approachable for those looking to compare outputs with Stable Audio without needing to download dataset resources.

OpenFedLLM is an open-source resource for training large language models using federated learning. It offers seven federated learning methods, two methods for instruction tuning and value alignment, and more than 30 metrics for assessing capabilities in tasks such as medical and financial QA, code generation, and math solving. This project includes FedLLM-Bench, a benchmarking tool for evaluating federated LLM training. The platform enables the installation, training script access for instruction tuning and value alignment, and comprehensive evaluation tools. OpenFedLLM aids in maintaining decentralized data privacy while pushing forward in the field of language model training.