Dopamine: A User-Friendly Framework for Reinforcement Learning
Dopamine is a research framework designed to facilitate quick and easy prototyping of reinforcement learning algorithms. It addresses the need for a clean and understandable codebase, where users can freely explore and experiment with innovative ideas in speculative research.
Design Principles
- Easy Experimentation: Dopamine simplifies running benchmark experiments for new users.
- Flexible Development: It allows users to test their research ideas effortlessly.
- Compact and Reliable: The framework includes robust implementations of several established algorithms.
- Reproducibility: The system is designed to ensure that results can be easily reproduced, following the guidelines by Machado et al. (2018).
Supported Agents
Dopamine, with its cutting-edge library jax, supports various reinforcement learning agents:
- DQN (Deep Q-Network)
- C51
- Rainbow
- IQN (Implicit Quantile Networks)
- SAC (Soft Actor-Critic)
- PPO (Proximal Policy Optimization)
Some agents also have legacy implementations using TensorFlow.
Getting Started
Using Docker
Dopamine provides Docker containers for a convenient setup. Instructions are available on the official documentation site.
Installing From Source
This is the recommended approach for using Dopamine:
-
Clone the repository:
git clone https://github.com/google/dopamine -
Install necessary dependencies:
pip install -r dopamine/requirements.txt
For TensorFlow installation, refer to their official site. A virtual environment is recommended when working with Dopamine.
Installing with Pip
Although it's straightforward to install Dopamine using pip, it is preferable to install from the source to customize experiments:
pip install dopamine-rl
Running Tests
To verify a successful installation, run the following command in the root directory:
export PYTHONPATH=$PYTHONPATH:$PWD
python -m tests.dopamine.atari_init_test
Next Steps
Dopamine provides detailed documentation and baseline results for training agents. There are Colaboratory notebooks available to demonstrate the usage of Dopamine for those seeking practical examples.
References and Credits
Dopamine includes references to significant papers and encourages proper citation if used in research projects. The BibTeX entry for citation is provided in the documentation.
Dopamine is maintained as a comprehensive framework, supporting researchers and developers in advancing their work in reinforcement learning.