pku-alignment / redman

ReDMan is an open-source simulation platform that provides a standardized implementation of safe RL algorithms for Reliable Dexterous Manipulation.

ReDMan is an open-source simulation platform that provides a standardized implementation of safe RL algorithms for Reliable Dexterous Manipulation.

• For Safe RL researchers We present a novel framework for Safe RL algorithms that is unified, highly optimized, and extensible, with re-implementations of commonly used algorithms that support ReDMan and popular environments. Our framework emphasizes abstraction and encapsulation to encourage code reuse and maintain a clean code style. We also provide a suite of intricate and demanding safe dexterous manipulation tasks that are motivated by the requirement for safe robotic manipulation in daily life scenarios, such as cleaning floors without damaging furniture. Exhaustive experiments with our implemented algorithms were conducted in these environments, and we share the results, observations, and analyses to benefit the Safe RL community.

• For Robotic researchers We present the inaugural compilation of tasks aimed at safe dexterous manipulations. Along with safety considerations, we offer various features, such as multi-modal observation information that includes contact force, RGB image, RGB-D image, point cloud, and more. Additionally, our platform boasts customizable dexterous hands and a robotic arm drive specifically tailored to the dexterous hand. These features collectively provide an all-encompassing platform for robotic research.

Contents of this repo are as follows:

• Introduction
• Getting Started
  • Installation
  • Algorithms
  • Tasks
  • Constraint Task List
  • Run Command
  • Visual Input
  • Available Hands and Arms
• The Team
• License

---

Introduction

This repository contains complex safe dexterous hands control tasks. ReDMan is built in the NVIDIA Isaac Gym with high performance guarantee for training RL algorithms. Our environments focus on applying safe RL algorithms for dexterous manipulation.

Getting Started

Installation

Details regarding installation of IsaacGym can be found here. We currently support the Preview Release 4 version of IsaacGym.

The code has been tested on Ubuntu 18.04 with Python 3.8. The minimum recommended NVIDIA driver version for Linux is 470.74 (dictated by support of IsaacGym).

It uses Anaconda to create virtual environments. To install Anaconda, follow instructions here.

Ensure that Isaac Gym works on your system by running one of the examples from the python/examples directory, like joint_monkey.py. Please follow troubleshooting steps described in the Isaac Gym Preview 3 install instructions if you have any trouble running the samples.

Once Isaac Gym is installed and samples work within your current python environment, install this repo and install the dependence:

git clone https://github.com/OmniSafeAI/ReDMan.git
cd ReDMan
pip install -e .

Algorithms

Algorithm	Proceedings&Cites	Official Code Repo	Official Code Last Update	Official Github Stars	Command Line
PPO-Lag	❌	Tensorflow1			ppol
TRPO-Lag	❌	Tensorflow1			trpol
FOCOPS	Neurips 2020 (Cite: 27)	Pytorch			focops
CPO	ICML 2017(Cite: 663)	❌	❌	❌	cpo
PCPO	ICLR 2020(Cite: 67)	Theano	❌	❌	pcpo
P3O	IJCAI 2022(Cite: 0)	❌	❌	❌	p3o
CPPO-PID	Neurips 2020(Cite: 71)	Pytorch			cppo_pid

Tasks

So far, we release the following tasks (with many more to come):

Constraint Task List

• ShadowHandCatchOver2Underarm_Safe_finger
• ShadowHandCatchOver2Underarm_Safe_joint
• ShadowHandOver_Safe_finger
• ShadowHandOver_Safe_joint
• ShadowHandGrasp
• ShadowHandDieRotation
• ShadowHandCatchUnderarmWall
• ShadowHandCatchUnderarmWallDown
• ShadowHandOverWall
• ShadowHandOverWallDown
• ShadowHandPickBottle
• ShadowHandDrawBlocks
• ShadowHandClean

Run Command

python train.py --task=ShadowHandOver --algo=ppol --cost_lim 25.0

For more information about running commands, please refer to experiment.md in the root directory

Visual Input

We provide multiple modalities of visual information as input, including RGB, RGB-D, and point cloud. It is generated using the camera in the Isaac Gym, and the pose and toward of the camera can be customized by the user to obtain the desired visual observation.

Available Hands and Arms

To promote research and community development in the field, having multiple types of dexterous hands, such as the Shadow Hand, Allegro Hand, and Tri-Finger, is crucial. ReDMan goes beyond offering just the Shadow Hand, by providing a selection of five other dexterous multi-finger hands.

Incorporating a robotic arm drive at the base of the dexterous hand not only mimics real-world conditions but also represents an essential step in achieving successful sim-to-real transfer. Due to the inherent difficulty in reproducing the true dynamics of a flying hand, the ReDMan platform simplifies the deployment process from simulation to real-world applications by permitting adjustments to the dynamics and physics parameters of the arm to minimize the reality gap.

The Team

ReDMan is a project contributed by Yiran Geng, Jiaming Ji, Yuanpei Chen, Haoran Geng, Fangwei Zhong and Yaodong Yang at Peking University, please contact [email protected] if you are interested to collaborate.

We also thank Chenrui Tie for contributing to a variety of robotic arm and dexterous hand tasks.

License

ReDMan has an Apache license, as found in the LICENSE file.