Introduction to Gymnasium
Gymnasium is a cutting-edge, open-source Python library designed to advance the development and evaluation of reinforcement learning (RL) algorithms. It provides a universal API that acts as a bridge between learning algorithms and various environments. Serving as a fork of OpenAI's renowned Gym library, Gymnasium is now maintained independently and continues the legacy by offering a comprehensive suite for future RL research and development.
The official documentation for Gymnasium can be found at gymnasium.farama.org, offering clear guidance and resources. For community interaction and development coordination, Gymnasium hosts a public Discord server accessible here.
Environments
Gymnasium features a diverse array of environments meant to challenge and refine RL algorithms. They include:
- Classic Control: Inspired by real-world physics challenges, these environments emulate traditional RL scenarios.
- Box2D: Engaging with physics-based games, these environments offer a mix of playful yet challenging tasks, utilizing Box2D physics and PyGame for rendering.
- Toy Text: These are minimalist environments with limited state and action spaces, ideal for debugging and initial experimentation with RL algorithms.
- MuJoCo: Offering environments centered around multi-joint physical systems, these scenarios pose a significant challenge due to their complexity.
- Atari: Simulating the complexity of Atari 2600 games, these environments are sophisticated and demanding, testing the limits of RL agents.
- Third-party: A multitude of environments utilizing the Gymnasium API, ensuring a broad range of challenges.
Installation
To begin using Gymnasium, install the core library via pip install gymnasium. Note that this basic installation does not encompass dependencies for all environments due to their vastness and potential installation issues on specific systems. Users can install dependencies for specific families using commands like pip install "gymnasium[atari]" or opt for a comprehensive installation with pip install "gymnasium[all]".
Gymnasium is thoroughly tested on Python versions 3.8 to 3.12 for Linux and macOS. While contributions for Windows compatibility are welcome, official support for this platform is presently not part of Gymnasium's offerings.
API
The Gymnasium API treats environments as straightforward Python env classes. Creating and interacting with these environment instances is intuitive. For example, using the CartPole-v1 environment involves the following code:
import gymnasium as gym
env = gym.make("CartPole-v1")
observation, info = env.reset(seed=42)
for _ in range(1000):
action = env.action_space.sample()
observation, reward, terminated, truncated, info = env.step(action)
if terminated or truncated:
observation, info = env.reset()
env.close()
Notable Related Libraries
Several libraries complement Gymnasium, each with unique offerings:
- CleanRL: Built upon the Gymnasium API, this library caters to newcomers by providing excellent, beginner-friendly implementations.
- PettingZoo: A multi-agent variant of Gymnasium, featuring a range of environments designed for collaborative and competitive multi-agent scenarios, such as multi-agent Atari games.
- The Farama Foundation maintains several additional environments, promoting continued exploration within the Gymnasium API framework.
Environment Versioning
To ensure reproducibility, Gymnasium adheres to a strict versioning system. Environment versions are denoted with suffixes like "-v0". Any modifications affecting learning outcomes lead to a version increment, preserving clarity and consistency among users.
Supporting Gymnasium's Development
Gymnasium's development relies on community support. If able, consider joining others in supporting the project financially through donations.
Citation
For academic citations, Gymnasium's contributions are detailed in a related paper available here:
@article{towers2024gymnasium,
title={Gymnasium: A Standard Interface for Reinforcement Learning Environments},
author={Towers, Mark and Kwiatkowski, Ariel and Terry, Jordan and Balis, John U and De Cola, Gianluca and Deleu, Tristan and Goul{\~a}o, Manuel and Kallinteris, Andreas and Krimmel, Markus and KG, Arjun and others},
journal={arXiv preprint arXiv:2407.17032},
year={2024}
}
In conclusion, Gymnasium stands as a pivotal resource for researchers and developers in reinforcement learning, offering a robust platform for experimentation and refinement in a continually evolving field.