maro - Improve Resource Management in Real-World Applications Using Reinforcement Learning

Introduction to the MARO Project

MARO, or Multi-Agent Resource Optimization, is a cutting-edge platform created to support real-world resource optimization through Reinforcement Learning as a Service (RaaS). The platform is designed to be highly applicable across various industrial domains including logistics, transportation, data centers, and financial sectors. MARO stands out for its ability to assist in processes like container inventory management, bike repositioning, virtual machine provisioning, and asset management. Beyond just reinforcement learning, it supports decision-making processes through other methods like Operations Research.

Key Components of MARO

MARO is built around three primary toolkits that make it versatile and efficient for users:

Simulation Toolkit: This provides users with predefined scenarios, along with components that allow for the creation of new simulations.
RL Toolkit: Offers a comprehensive abstraction for reinforcement learning, incorporating elements such as agent managers, RL algorithms, and various elements for shaping learning processes.
Distributed Toolkit: Enables distributed communication, user-defined function interfaces for automated message handling, cluster management, and job orchestration.

Installing MARO

MARO is available for installation through PyPI for general users and from source for those who need access to the command line interface (CLI) and visualization tools.

From PyPI: Easily install using pip for both Mac OS/Linux and Windows. Special instructions are provided for Windows users who need additional packages like torch.
From Source: Requires setting up a virtual environment and includes both Mac OS/Linux and Windows installation guides, provided to cater to different user prerequisites (e.g., a C++ compiler).

Quick Example

The following Python sample code demonstrates how to initialize a MARO environment and run a simplistic simulation using the 'cim' scenario:

from maro.simulator import Env

env = Env(scenario="cim", topology="toy.5p_ssddd_l0.0", start_tick=0, durations=100)

metrics, decision_event, is_done = env.step(None)

while not is_done:
    metrics, decision_event, is_done = env.step(None)

print(f"environment metrics: {env.metrics}")

Visualization

Users can enable an environment dump feature for deeper analysis of scenarios and inspect the environment using visualization tools by generating dump data.

MARO Playground

The platform supports a playground environment accessible via Docker, promoting an interactive experience where users can engage with MARO using Redis Commander and Jupyter Lab.

Contributing to MARO

Microsoft encourages open contributions to the MARO project. Contributors may need to sign a Contributor License Agreement (CLA). The project abides by the Microsoft Open Source Code of Conduct, ensuring a respectful and constructive community.

License and Further Information

MARO is open-source and licensed under the MIT License, inviting broad and flexible use. Extensive documentation, visual aids, and related academic papers can be explored to deepen the understanding of its applications.

For more information, you can explore the MARO documentation hosted here.