tianhongdai / div-hindsight

This is the official code of our paper "Diversity-based Trajectory and Goal Selection with Hindsight Experience Relay" [PRICAI 2021].

Diversity-based Trajectory and Goal Selection with Hindsight Experience Replay

This is the official implementation of our paper "Diversity-based Trajectory and Goal Selection with Hindsight Experience Replay" [PRICAI 2021][Arxiv][Slides].

Pre-requisites

• python==3.6.8
• gym==0.12.5
• mujoco-py==2.0.2.0
• tensorflow==1.13.1

Installation

1. Install the OpenMPI:

sudo apt-get install openmpi-bin openmpi-doc libopenmpi-dev

2. Install the required python packages.
3. Install baseline:

pip install -e .

Usage

Enter to the specific location:

cd ./baselines/her/experiment

The scripts for FetchEnv:

python train.py --env_name FetchPush-v1 --clip_div=0.001 --use_kdpp True --prioritization diversity --seed 0 --n_epochs 50 --num_cpu 16 --logdir logs/fetchpushv1/cpu16ep50/alg=DDPG+DTGSH=/r0 --logging True
python train.py --env_name FetchPickAndPlace-v1 --clip_div=0.001 --use_kdpp True --prioritization diversity --seed 0 --n_epochs 50 --num_cpu 16 --logdir logs/fetchpickv1/cpu16ep50/alg=DDPG+DTGSH=/r0 --logging True

The scripts for HandEnv:

python train.py --env_name HandManipulateEggFull-v0 --use_kdpp True  --prioritization diversity --seed 0 --goal_type full --sigma 0.1 --n_epochs 50 --num_cpu 16 --logdir logs/handegg1/cpu16ep50/alg=DDPG+DTGSH=/r0 --logging True
python train.py --env_name HandManipulateBlockRotateXYZ-v0 --use_kdpp True  --prioritization diversity --seed 0 --goal_type rotate --sigma 0.1 --n_epochs 50 --num_cpu 16 --logdir logs/handblock1/cpu16ep50/alg=DDPG+DTGSH=/r0 --logging True
python train.py --env_name HandManipulatePenRotate-v0 --use_kdpp True --prioritization diversity --seed 0 --goal_type rotate --sigma 0.1 --n_epochs 50 --num_cpu 16 --logdir logs/handpen1/cpu16ep50/alg=DDPG+DTGSH=/r0 --logging True

Acknowledgements

• @ruizhaogit for Priority-based Experience Replay
• @openai for Baselines
• @satwik77 for Original DPP Implementation

BibTex

To cite this code for publications - please use:

@inproceedings{dai2021her
	author={Dai, Tianhong and Liu, Hengyan and Arulkumaran, Kai and Ren, Guangyu and Bharath, Anil Anthony},
	title={Diversity-Based Trajectory and Goal Selection with Hindsight Experience Replay},
	booktitle={Pacific Rim International Conference on Artificial Intelligence},
	year={2021},
	pages={32--45},
	publisher={Springer},
}

div-hindsight's Issues

error when running the code

hello,
when I running the code, it reports an error like the following shows:

marg= lmbda[i-1] * E[rem-1,i-1]/(E[rem,i]+1e-100)
IndexError: index -101 is out of bounds for axis 0 with size 100