RSL RL Project Introduction
The RSL RL project offers a streamlined implementation of reinforcement learning (RL) algorithms, tailored for running entirely on GPUs. It builds upon NVIDIA's Isaac Gym, expanding the capabilities of its predecessor, rl-pytorch. The main goal of this project is to provide a fast and simple way to implement RL algorithms, with a specific focus on performance and ease of use.
Current and Upcoming Algorithms
At present, the RSL RL project includes the implementation of the Proximal Policy Optimization (PPO) algorithm. This is a widely-used and effective algorithm in the field of reinforcement learning. The project aims to expand further by incorporating additional algorithms like Soft Actor-Critic (SAC), Deep Deterministic Policy Gradient (DDPG), and more, accessible through the algorithms branch of the repository.
Project Maintenance and Contribution
The project is maintained by David Hoeller and Nikita Rudin, who are affiliated with the Robotic Systems Lab at ETH Zurich and NVIDIA. They invite contributions and engagement from the community to enhance the project further. Contributors are encouraged to follow the project's documentation and code quality guidelines to ensure consistency and maintainability.
Setting Up the RSL RL Project
To set up the RSL RL project, users can clone the repository and install it in editable mode. Here's a quick setup guide:
git clone https://github.com/leggedrobotics/rsl_rl
cd rsl_rl
pip install -e .
Once set up, the framework supports various logging frameworks for tracking and visualizing the training process. Users can choose from:
- Tensorboard: A visualization tool from TensorFlow.
- Weights & Biases: An experiment tracking platform.
- Neptune: A tool for monitoring machine learning experiments.
For those who wish to start with a demo configuration of PPO, a sample configuration can be found in the dummy_config.yaml file provided in the repository.
Documentation and Code Quality
Documentation is a crucial aspect of the RSL RL project. It follows the Google Style Guide for docstrings, and uses Sphinx to generate comprehensive documentation. Code quality is maintained using tools like:
- pre-commit: To run formatters and linters over the codebase.
- black: A code formatter known for enforcing uniform code styling.
- flake8: A tool to check the code against style conventions and potential errors.
Contributors must ensure their code is well-documented and adheres to these guidelines. Instructions for setting up these tools are available, and running them over the entire repository is straightforward with pre-commit.
Additional Resources and Links
The RSL RL framework serves as a basis for other environment repositories such as:
- Legged-Gym: Built on top of NVIDIA Isaac Gym, enhancing RL for legged robots.
- Orbit: Utilizes NVIDIA Isaac Sim for simulating various RL environments.
These resources offer additional insights and practical applications, showcasing the versatility and adaptability of the RSL RL framework in different domains.