Introduction to FL-bench
FL-bench is a comprehensive benchmarking platform designed to evaluate various Federated Learning (FL) methods. Federated Learning is a technique that enables multiple participants to collaboratively train a machine learning model without sharing their raw data, often used when privacy is a concern. This project serves as a playground for developers and researchers to experiment with FL algorithms, refine their ideas, and contribute to the growing field of distributed machine learning.
Traditional FL Methods
FL-bench supports a wide range of traditional federated learning approaches:
- FedAvg: A foundational method focusing on communication-efficient learning from decentralized data.
- FedProx: Enhances federated optimization by addressing data heterogeneity across clients.
- SCAFFOLD, MOON, FedDyn, and others: Each introduces unique mechanisms to improve learning in disparate and non-iid (non-independent and identically distributed) client environments.
Personalized FL Methods
FL-bench also includes advanced methods for personalizing models to fit individual client needs:
- pFedSim: A cutting-edge solution that aggregates models based on similarity measures.
- FedMD and FedBN offer alternate strategies for model personalization without sacrificing data privacy.
- Other methods like Ditto, pFedHN, and FedRep focus on fine-tuning the model personalization process for better individual client performance.
FL Domain Generalization Methods
To handle the challenge of generalizing across different domains, FL-bench offers:
- FedSR: Simple yet effective options for achieving domain generalization.
- ADCOL and FedIIR: Tackle issues related to non-IID features and aim to enhance the robustness of FL models in varied domains.
Environment and Setup
Setting up FL-bench is straightforward with support for different package managers like PyPI and Poetry. Users can also deploy the environment in a Docker container, enabling a consistent setup across different systems.
Easy Run and Customization
Running experiments is facilitated through a well-documented process comprising dataset generation and configuration setup. The platform allows users to tweak their experiments through customizable configuration files or command-line arguments. For parallel execution, FL-bench uses Ray, significantly enhancing training efficiency across multiple processing units.
Monitoring and Evaluation
FL-bench has robust support for real-time monitoring of experiments using Visdom and TensorBoard, enabling users to visualize their model's performance and metrics over time.
Supported Models and Datasets
The platform supports a variety of models and datasets. From popular choices like ResNet and MobileNet to simpler architectures like LeNet5, there's something for everyone. It also provides access to multiple datasets, spanning general image data to specific domains like medical imaging and domain generalization datasets.
Customization for Advanced Users
FL-bench is designed to be flexible, allowing adept users to implement new algorithms, integrate custom datasets, and define unique model architectures. This capability makes it an ideal tool for researchers looking to push the boundaries of federated learning.
Citation
For those utilizing FL-bench in their research, proper citation is encouraged to acknowledge the contributions of the developers and maintainers of this valuable tool.
In summary, FL-bench provides a user-friendly and versatile platform for testing and developing federated learning methods, helping advance research and application in a privacy-preserving manner.