OMLT: Optimization and Machine Learning Toolkit
OMLT is a powerful Python package designed to bridge the gap between machine learning models and the Pyomo optimization environment. This toolkit is especially useful for representing complex models such as neural networks and gradient-boosted trees, providing various optimization formulations including full-space, reduced-space, and mixed-integer linear programming (MILP). It also supports importing models from popular machine learning frameworks like Keras and ONNX.
Key Features
- Versatile Model Representation: OMLT allows for seamless integration of machine learning models, from neural networks to decision trees, into optimization tasks.
- Multiple Optimization Formulations: Users can choose between several optimization formulations to best suit their needs, whether it be handling the complexity of full-space or the constraints of MILP.
- Easy Integration with Pyomo: The package is built to work smoothly with Pyomo, a robust environment for mathematical modeling, allowing users to leverage both ML and optimization capabilities.
Academic Contributions
OMLT's development is backed by substantial academic research, as referenced in several published papers. These publications highlight the toolkit’s capabilities in various applications, from handling linear model decision trees in engineering applications to utilizing graph neural networks for molecular design.
Documentation and Resources
The latest documentation for OMLT is available online, which includes comprehensive guides and examples. For those who prefer hands-on learning, several Jupyter notebooks are also available to demonstrate the toolkit's functionalities.
Example Code
Here's a basic example of how to use OMLT with a Keras model:
import tensorflow
import pyomo.environ as pyo
from omlt import OmltBlock, OffsetScaling
from omlt.neuralnet import FullSpaceNNFormulation, NetworkDefinition
from omlt.io import load_keras_sequential
# Load a Keras model
nn = tensorflow.keras.models.load_model('tests/models/keras_linear_131_sigmoid', compile=False)
# Create a Pyomo model
model = pyo.ConcreteModel()
model.nn = OmltBlock()
# Define input and output variables
model.input = pyo.Var()
model.output = pyo.Var()
# Apply scaling
scale_x = (1, 0.5)
scale_y = (-0.25, 0.125)
scaler = OffsetScaling(offset_inputs=[scale_x[0]], factor_inputs=[scale_x[1]],
offset_outputs=[scale_y[0]], factor_outputs=[scale_y[1]])
# Load the Keras model into a network definition
net = load_keras_sequential(nn, scaler, {0: (0, 5)})
# Use a neural network formulation
formulation = FullSpaceNNFormulation(net)
# Build the formulation on the OMLT block
model.nn.build_formulation(formulation)
# Set constraints
@model.Constraint()
def connect_input(mdl):
return mdl.input == mdl.nn.inputs[0]
@model.Constraint()
def connect_output(mdl):
return mdl.output == mdl.nn.outputs[0]
# Solve an inverse problem
model.obj = pyo.Objective(expr=(model.output - 0.5)**2)
status = pyo.SolverFactory('ipopt').solve(model, tee=False)
print(pyo.value(model.input))
print(pyo.value(model.output))
Development and Contributions
OMLT's development is managed using just, a tool for running tasks and installations. Contributors to the project are recognized for their significant inputs from various prestigious institutions and programs, highlighting the collaborative and scholarly nature of the project’s progress.
In summary, OMLT is an essential toolkit for anyone looking to integrate machine learning models with optimization tasks, bringing both simplicity and power to complex mathematical modeling.