This repo relates to the survey paper <Goal-Conditioned Reinforcement Learning: Problems and Solutions>. We collects widely used benchmark environments and conclude a series of research works for goal-conditioned reinforcement learning (GCRL).
Approach | Resolved Goal Type | Sub-Goal | Relabel | Optimization |
---|---|---|---|---|
UVFA | Vector/Image | ✅ | ||
HER | Vector/Image | ✅ | ||
Laplacian Representation Learning | Vector/Image | ✅ | ||
SR | Vector/Image | ✅ | ||
DDL | Vector/Image | ✅ | ✅ | |
SoRB | Vector/Image | ✅ | ||
Skew-Fit | Vector | ✅ | ||
HER with Demonstrations | Vector | ✅ | ||
GoalGAN | Vector | ✅ | ||
Sub-goal Discovery | Vector | ✅ | ||
HGG | Vector | ✅ | ||
CHER | Vector | ✅ | ||
G-HER | Vector | ✅ | ||
Hindsight Planner | Vector | ✅ | ||
GDP | Vector | ✅ | ||
VDS | Vector | ✅ | ||
MEGA | Vector | ✅ | ✅ | |
PlanGAN | Vector | ✅ | ||
AIM | Vector | ✅ | ||
I-HER | Vector | ✅ | ||
GCSL | Vector | ✅ | ||
MapGo | Vector | ✅ | ✅ | |
L3P | Vector | ✅ | ||
RIG | Image | ✅ | ||
DISCERN | Image | ✅ | ||
HVF | Image | ✅ | ||
AMIGo | Image | ✅ | ||
GAP | Image | ✅ | ||
LEXA | Image | ✅ |
Note almost all environments support various kinds of goal structures, i.e., vector, image, language, etc.
Environment | Benchmark Type | Common Used Goal Structure | Description |
---|---|---|---|
FetchReach | Gym Robotics/Mujoco | Vector/Image | A robotic arm reaches a target position |
FetchPush | Gym Robotics/Mujoco | Vector/Image | A robotic arm pushes a block to a target position |
FetchPickAndPlace | Gym Robotics/Mujoco | Vector/Image | A robotic arm first picks up the block then places it to a target position |
FetchSlide | Gym Robotics/Mujoco | Vector/Image | A robotic arm slides a buck to a goal position |
HandManipulateBlock | Gym Robotics/Mujoco | Vector/Image | Orient a block using a robot hand |
HandManipulateEgg | Gym Robotics/Mujoco | Vector/Image | Orient an egg using a robot hand |
HandManipulatePen | Gym Robotics/Mujoco | Vector/Image | Orient a pen using a robot hand |
HalfCheetah | Mujoco | Vector/Image | Make a 2D cheetah robot run and keep a specific speed |
AntLocomotion/Ant Maze | Mujoco | Vector | Make a 3D four-legged robot walk to a target position |
Sawyer | Mujoco | Vector | A sawyer robot reaches a target position |
Reacher | Mujoco | Vector/Image | A reacher robot reaches a target position |
N-D Mass | Mujoco | Vector | A ball reaches a target position |
DoorOpening | Mujoco | Image | A sawyer robot opens a door by latching onto the handle |
Dy-Reaching | Mujoco | Image | A robotic arm reaches a target position moving along a straight line |
Dy-Circling | Mujoco | Image | A robotic arm reaches a target position moving along a circle |
Dy-Pushing | Mujoco | Image | A robotic arm moves the box to a moving target position |
Dy-Pouring | Mujoco | Image | A robotic arm grips the can and pours the water into a cup |
WaterMaze | DeepMind Lab | Image | Get to a target position described by an image |
Seaquest | Atari | Image | Get to a target position described by an image |
Montezuma’s Revenge | Atari | Image | Get to a target position described by an image |
Reacher | DeepMind Control Suit | Image | A two-link planar reacher poses an aimed shape |
Manipulator | DeepMind Control Suit | Image | A 3-DoF finger robot rotates the body into a target orientation |
Finger | DeepMind Control Suit | Image | A planar manipulator brings an object in a target location |
Place Object in Tray | PyBullet | Image | A robotic arm puts toys into a tray |
Opening Drawer | PyBullet | Image | A robotic arm opens the drawer |
PickandPlace | PyBullet | Image | A robotic arm first picks up the block then places it to a target position |
Five Object Manipulation | CLEVR-Robot/Mujoco | Image/Language | A point mass agent arranges, orders and sorts 5 fixed objects |
Diverse Object Manipulation | CLEVR-Robot/Mujoco | Image/Language | A point mass agent orders objects in different shapes and colors |
Playground | Playground | Image/Language | A 2D hand moves and grasps/releases different objects |
VizDoom | ViZDoom | Image/Language | AI bots reach specific place in the first person shooter 3D game Doom |