hivemind
Hivemind enables decentralized deep learning with PyTorch, facilitating large-scale model training without a central server. It offers fault-tolerant backpropagation and decentralized parameter averaging for flexible network training. Used in projects like Training Transformers Together, it supports Linux, macOS, and Windows 10+, and integrates with PyTorch Lightning for handling distributed, unreliable peers.
