rl-baselines3-zoo - Reinforcement Learning Framework with Training and Evaluation Features

RL Baselines3 Zoo: A Comprehensive Training Framework for Reinforcement Learning

Overview

RL Baselines3 Zoo is a user-friendly, extensive training framework designed explicitly for Reinforcement Learning (RL) agents. It leverages the Stable Baselines3 library to provide users with scripts and resources that aid in the training, evaluation, and optimization of RL agents. This framework is especially valuable for beginners and experts alike who are interested in experimenting with different reinforcement learning algorithms and environments.

Key Features

Comprehensive Training Scripts: RL Baselines3 Zoo offers scripts that facilitate the training and evaluation of RL agents, hyperparameter tuning, and result visualization. These scripts make it easier to manage the complexities associated with developing and testing RL models.
Tuned Hyperparameters: It comes with a pre-defined set of hyperparameters tailored for various environments and RL algorithms. This enables users to quickly deploy and enjoy RL agents without the tedious process of manually tuning hyperparameters.
Contribution-Friendly: The project is actively seeking contributions from the community to expand its collection of trained agents and optimized hyperparameters, making it a collaborative and evolving platform.
Integration with Tools: The framework supports integration with popular tools like Weights & Biases for experiment tracking, and Hugging Face for storing and sharing trained models.

Installation

RL Baselines3 Zoo can be installed minimally or fully, depending on the user's requirements. The minimal installation can be done from the source or as a Python package using pip:

pip install -e .
# or
pip install rl_zoo3

For users requiring additional environments and testing dependencies, a full installation is recommended:

apt-get install swig cmake ffmpeg
pip install -r requirements.txt
pip install -e .[plots,tests]

Training and Evaluation

Training an RL agent in RL Baselines3 Zoo is straightforward. Users can start by specifying the algorithm and environment:

python train.py --algo algo_name --env env_id

For evaluation, the framework can assess the agent's performance through specified episodes at regular steps, allowing users to track the learning progress:

python train.py --algo sac --env HalfCheetahBulletEnv-v0 --eval-freq 10000 --eval-episodes 10 --n-eval-envs 1

Available Environments and Algorithms

RL Baselines3 Zoo supports a wide range of environments, including Atari games, classic control environments, Box2D environments, PyBullet and MuJoCo for more complex simulations, as well as robotics environments for tasks like fetching and sliding. It accommodates various algorithms such as A2C, PPO, DQN, SAC, TD3, among others, showcasing their capabilities across different settings. The project maintains a benchmark document that provides insight into the performance of these algorithms.

Exploring and Customizing

The framework offers resources like plotting scripts and tuning guides to enhance the RL development experience. Users interested in extending the framework can refer to detailed sections on custom configurations and hyperparameter tuning in the documentation.

Community and Contributions

The project thrives on community engagement and encourages users to contribute by submitting trained agents that are not yet present in the collection. Contributors are acknowledged for their efforts, fostering a shared learning and development environment.

Try it Online!

For individuals looking to experiment with RL agents without a local setup, a Colab notebook is available, offering an online interface to explore and train agents.

By providing an accessible and robust platform, RL Baselines3 Zoo aids in the advancement of reinforcement learning research and application, empowering users with the tools and knowledge needed for success in this field.