opacus - Facilitate Differential Privacy in PyTorch Models

Introduction to Opacus: A Differential Privacy Library for PyTorch

Opacus is a robust library designed to enhance the training of PyTorch models by integrating differential privacy. This library stands out due to its ease of use and minimal performance impact, offering a seamless way for users to keep track of the privacy budget during training. It caters to both machine learning practitioners and differential privacy researchers, each of whom will find valuable tools and capabilities to refine and expand their work.

Target Audience

Opacus targets two primary groups:

ML Practitioners: For those new to differential privacy, Opacus offers a straightforward introduction by requiring minimal modifications to existing codebases, making the transition smooth and efficient.
Differential Privacy Researchers: Researchers interested in experimenting with differential privacy will find Opacus flexible and conducive to exploration, allowing them to concentrate on their primary research goals.

Installation

Opacus is easily installable through common package managers:

Pip:
```
pip install opacus
```
Conda:
```
conda install -c conda-forge opacus
```

For those seeking the latest features, Opacus can be installed directly from the source:

git clone https://github.com/pytorch/opacus.git
cd opacus
pip install -e .

Getting Started

To incorporate differential privacy in model training, users can utilize the PrivacyEngine. This involves a simple setup:

model = Net()  # Define the model
optimizer = SGD(model.parameters(), lr=0.05)  # Define the optimizer
data_loader = torch.utils.data.DataLoader(dataset, batch_size=1024)  # Define the data loader

privacy_engine = PrivacyEngine()
model, optimizer, data_loader = privacy_engine.make_private(
    module=model,
    optimizer=optimizer,
    data_loader=data_loader,
    noise_multiplier=1.1,
    max_grad_norm=1.0,
)

More practical examples, such as training with MNIST, are available in the examples folder on the Opacus GitHub repository.

Migrating to 1.0

For users of earlier versions (0.x) who wish to upgrade, Opacus 1.0 brings numerous enhancements but may also include some breaking changes. A Migration Guide is provided for smooth transition.

Learning Resources

Opacus offers numerous learning materials to get users started with privacy-preserving training:

Interactive Tutorials: A variety of IPython notebooks are available, guiding users through building differentially private models and exploring advanced features.

Notable tutorials include:
- Building an Image Classifier with Differential Privacy
- Training a differentially private LSTM model for name classification
- Building text classifier with Differential Privacy on BERT
Technical Report and Citation: A detailed technical report introduces Opacus, outlining its design principles and benchmarks. Users are encouraged to cite the report in academic work.
Blogposts and Talks: Several blog posts and presentations provide additional insights into differential privacy, including the DP-SGD algorithm and Opacus's unique gradient computation methodologies.

Additional Resources

FAQ: A FAQ page answers common questions related to differential privacy and Opacus.
Contributing: Contributions are welcome, and detailed guidelines are available to help newcomers get involved with the project's development.
License: The library is released under the Apache 2.0 license, ensuring open access and collaboration.

Opacus represents a significant advancement in leveraging differential privacy in machine learning, making it accessible and practical for developers and researchers alike. Through Opacus, users can maintain data privacy without compromising on model performance.