hilanzy / moss Goto Github PK

View Code? Open in Web Editor NEW

2.0 1.0 0.0 1.53 MB

Moss is a Python library for Reinforcement Learning.

License: MIT License

Makefile 1.54% Python 97.87% Dockerfile 0.59%

atari envpool impala jax reinforcement-learning rlax dm-haiku dm-launchpad library pypi

moss's Introduction

Moss: A Python library for Reinforcement Learning

Moss is a Python library for Reinforcement Learning based on jax.

Installation

To get up and running quickly just follow the steps below:

Installing from PyPI: Moss is currently hosted on PyPI, you can simply install Moss from PyPI with the following command:

pip install moss-rl

Installing from github: If you are interested in running Moss as a developer, you can do so by cloning the Moss GitHub repository and then executing following command from the main directory (where setup.py is located):

pip install -e ".[dev]"

After installation, open your python console and type

import moss
print(moss.__version__)

If no error occurs, you have successfully installed Moss.

Work on GPU or TPU

If you want to run Moss with NVIDIA GPU, please run the steps below:

pip install --upgrade pip

pip uninstall jax jaxlib -y

# CUDA 12 installation
# Note: wheels only available on linux.
pip install --upgrade "jax[cuda12_pip]==0.4.20" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html

# CUDA 11 installation
# Note: wheels only available on linux.
pip install --upgrade "jax[cuda11_pip]==0.4.20" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html

Or if you want to run with Google Cloud TPU:

pip install "jax[tpu]==0.4.20" -f https://storage.googleapis.com/jax-releases/libtpu_releases.html

For more details, please see the JAX installation instructions here.

Quick Start

This is an example of Impala to train Atari game(use envpool).

python impala.py --task_id Pong-v5 --learning_rate 1e-3

moss's People

Contributors

Stargazers

Watchers

moss's Issues

[Feature]Support for custom action hidden sizes.

Doc complement

Supplement and improve the doc annotations in Moss.

[Refactor]Refactor agent inference.

Refine examples.

1.Refine hyper parameters entropy_coef for Seaquest-v5 in impala.(0.5 is best for now)

2.Fix bugs for local run.

[Feature]Support ray vector env.

Composable feature&action network

Refactor input feature and output action networks to make them composable.

Network refactor.

code format.
torso inputs refactor.
action spec refine.
reconstruct the directory structure of the network.

Use tensorboardX to avoid tensorflow dependency.

Refine network feature.

Refine FeatureSet process.
Refine network input feature.
Fix torso inputs concatenate axis.

Add env wrappers.

[Doc]Update README.

Add Python version img.

[BUG]Fix data format for image feature.

Fix data format for image feature to supports NHWC and NCHW.

[Feature]Supports continuous action

[BUG]Fix data format bug in local run.

[Feature]Supports composable multi actions.

Add vizdoom example

Add vizdoom example(use envpool).

Support multi agent training.

Add commit check github action.

Support wandb logger.

Network refactor.

[BUG]Fix local run.

Refactor feature spec.

Add local run for pettingzoo example.

[Refactor]Refactor feature encoder.

Mojoco example.

[Feature]Support subprocess vector env.

Refactor Network.

Refactor Network to make it more flexible and simpler to custom and compose.

[Feature]Support RNN.

[Feature]Supports multi-agent buffer.

[Feature]Refine module import.

[BUG]Fix ray import.

[BUG]Fix dependency.

Release version 0.0.5

Release a new version 0.0.5

PPO implementation

Support resize observation wrapper for pettingzoo.

Add ResizeObservation wrapper for pettingzoo, and add two arguments height and width for PettingZooEnv to set shape of image observation.

Update version.

Update version to 0.0.3

Refine `LocalEnv` to support RGB render.

[Feature]Async env step support(envpool)

Supports sampling data async in venv(now just for envpool).

[Feature]Value clip supports.

Fix fake behavior_values in PPOLearner._loss to supports value clip operation.

Add local run for vizdoom example.

Gradient clip

Add gradient clip trick.
Trick following this blog https://zhuanlan.zhihu.com/p/512327050 (Trick 7).

Add experiment api.

Add experiment api to make moss more easy to run.

Refactor moss env to make it more generic.

Refactor the Moss environment to make it more generic, allowing it to support more environment standards such as gym and pettingzoo.

Refactor moss environment.
Add pettingzoo example.
Modify file path of agent.
Fix spec shape of ScaleFeature and OneHotFeature.