nevergrad - Optimization Without Gradients for Diverse Variables

Nevergrad: A Gradient-Free Optimization Platform

Introduction

Nevergrad is a comprehensive Python library specializing in gradient-free optimization. Designed for Python 3.8 and later versions, it offers a variety of tools to help users minimize or optimize functions without needing gradients. It is particularly useful for solving complex optimization problems where derivative information is unavailable or unnecessary.

Installation

Installing Nevergrad is straightforward. For most users, it can be quickly set up using pip, Python's package installer, by running the following command:

pip install nevergrad

For those who need more detailed installation guidance, including specifics for Windows users, there are extensive instructions available in the "Getting Started" section of the documentation.

Basic Usage

One of the central features of Nevergrad is its ability to optimize functions efficiently. For instance, users can easily minimize a function using Nevergrad's NGOpt optimizer:

import nevergrad as ng

def square(x):
    return sum((x - .5)**2)

optimizer = ng.optimizers.NGOpt(parametrization=2, budget=100)
recommendation = optimizer.minimize(square)
print(recommendation.value)

In this example, the optimizer seeks to find values of x that minimize the square function, demonstrating its straightforward application for pure continuous variables.

Advanced Capabilities

Nevergrad shines in its support for a mix of variable types, including bounded continuous variables and discrete variables. Users can specify a diverse input space through parameterization, allowing for more complex and realistic optimization problems. Here's an example involving multiple inputs:

import nevergrad as ng

def fake_training(learning_rate: float, batch_size: int, architecture: str) -> float:
    return (learning_rate - 0.2)**2 + (batch_size - 4)**2 + (0 if architecture == "conv" else 10)

parametrization = ng.p.Instrumentation(
    learning_rate=ng.p.Log(lower=0.001, upper=1.0),
    batch_size=ng.p.Scalar(lower=1, upper=12).set_integer_casting(),
    architecture=ng.p.Choice(["conv", "fc"])
)

optimizer = ng.optimizers.NGOpt(parametrization=parametrization, budget=100)
recommendation = optimizer.minimize(fake_training)
print(recommendation.kwargs)

In this scenario, the library supports optimizing a mixture of log-scaled continuous values, integers, and categorical choices, tailoring solutions for real-world applications.

Documentation and Contributions

Nevergrad's documentation, available online, provides users with comprehensive support and is continually being updated. The community welcomes contributions, encouraging users to submit issues or pull requests for documentation enhancements.

Citing Nevergrad

For academic and similar usages, Nevergrad can be cited using the following BibTeX entry:

@misc{nevergrad,
    author = {J. Rapin and O. Teytaud},
    title = {{Nevergrad - A gradient-free optimization platform}},
    year = {2018},
    publisher = {GitHub},
    journal = {GitHub repository},
    howpublished = {\url{https://GitHub.com/FacebookResearch/Nevergrad}},
}

License

Nevergrad is distributed under the MIT License. Additional information can be found in the LICENSE file, along with the Terms of Use and Privacy Policy.

Nevergrad stands out as a versatile and powerful tool for those needing to perform optimization without the use of gradients, making it an indispensable resource for researchers and developers working in the field of optimization.