Gym.NET

Introduction to Gym.NET

Gym.NET is an exciting project that aims to bring the functionalities of OpenAI's Gym to the C# programming language. It serves as a bridge allowing developers familiar with C# to engage in reinforcement learning experiments similarly to what OpenAI Gym offers in Python. Gym.NET provides a standardized set of environments, making it a versatile toolkit for developing and comparing reinforcement learning algorithms.

About OpenAI Gym

OpenAI Gym is renowned for being a comprehensive toolkit designed for the development and comparison of reinforcement learning algorithms. It offers an open-source library packed with various environments, which serve as training platforms for these algorithms. By implementing Gym.NET, SciSharp provides a similar ecosystem within the C# domain.

Installation

To start using Gym.NET, developers need to install the necessary packages. These come in different components that cater to abstract classes and implemented environments:

# For gym's abstract classes for reinforcement learning, install:
PM> Install-Package Gym.NET

# For implemented environments, install:
PM> Install-Package Gym.NET.Environments
PM> Install-Package Gym.NET.Rendering.Avalonia
PM> Install-Package Gym.NET.Rendering.WinForm

These installations ensure that users have a complete set of tools to experiment with reinforcement learning environments in C#.

Example Use Case

An illustrative example involves running the cartpole-v1 environment, demonstrating how Gym.NET operates:

using NumSharp;
using SixLabors.ImageSharp;
using Gym.Environments;
using Gym.Environments.Envs.Classic;
using Gym.Rendering.WinForm;

CartPoleEnv cp = new CartPoleEnv(WinFormEnvViewer.Factory); //or AvaloniaEnvViewer.Factory
bool done = true;
for (int i = 0; i < 100_000; i++)
{
    if (done)
    {
        NDArray observation = cp.Reset();
        done = false;
    }
    else
    {
        var (observation, reward, _done, information) = cp.Step((i % 2));
        done = _done;
    }

    SixLabors.ImageSharp.Image img = cp.Render();
    Thread.Sleep(15);
}

cp.Close();

This script showcases initializing the CartPole environment, alternating actions, rendering the environment, and retrieving observations and rewards essential for reinforcement learning.

Roadmap and Development

Gym.NET continues to evolve with a roadmap aimed at enriching its environment capabilities. Key milestones include:

• Spaces Implementation:

  • Completed: Space, Box, Discrete
  • In Progress: multi.*.py

• Environment Base Classes:

  • Completed: Env(object)
  • In Progress: GoalEnv(Env)

• Environment Conversion:

  • The transition of Gym.Environments to a net-standard project is complete.
  • Ongoing efforts include adding more classic environments like walker2d_v3, acrobot, and others.

  Specific domains are under development such as:

  • Classics: Further implementation of classic environments
  • Mujco: Addition of environments like ant_v3
  • Box2D: Progressing on environments such as bipedal_walker

Gym.NET is poised to bring the flexibility and power of reinforcement learning from OpenAI Gym to developers using the C# language, fostering an interconnected and robust machine learning community.