This is modified version of the PyTorch implementation of:
Multi-skill Mobile Manipulation for Object Rearrangement
Jiayuan Gu, Devendra Singh Chaplot, Hao Su, Jitendra Malik
UC San Diego, Meta AI Research, UC Berkeley
Project website: https://sites.google.com/view/hab-m3
set_table_teaser.mp4
Table of Contents
conda create -n habitat-env python=3.10 -y
conda activate habitat-env
conda install cmake=3.14.0 patchelf ninja -y
This will install habitat sim, habitat lab, and this package. It will also download the necessary datasets.
cd hab-mobile-manipulation
./install.sh
Interactively play the task with the default config:
conda activate habitat-env
python habitat_extensions/tasks/rearrange/play.py
Use i/j/k/l
to move the robot end-effector, and w/a/s/d
to move the robot base. Use f/g
to grasp or release an object.
Use a specific task config:
python habitat_extensions/tasks/rearrange/play.py --cfg configs/rearrange/tasks/pick_v1.yaml
Or use a specific RL config:
python habitat_extensions/tasks/rearrange/play.py --cfg configs/rearrange/skills/tidy_house/pick_v1_joint_SCR.yaml
Pretrained skills can be downloaded via:
gdown https://drive.google.com/drive/folders/1u7DAd25PE818wjg-MxDKJ7y5n8GQtfrz -O data/results/rearrange/skills --folder
If this fails, you can manually add to the data/
folder the pretrained models from here. Simply download and unzip the results/
folder into data/
.
# Evaluate the latest checkpoint of a skill saved at "data/results/rearrange/skills/tidy_house/pick_v1_joint_SCR"
python mobile_manipulation/run_ppo.py --cfg configs/rearrange/skills/tidy_house/pick_v1_joint_SCR.yaml --run-type eval
# Evaluate the latest checkpoint of a skill saved at "data/results/rearrange/skills/tidy_house/pick_v1_joint_SCR/seed=100"
python mobile_manipulation/run_ppo.py --cfg configs/rearrange/skills/tidy_house/pick_v1_joint_SCR.yaml --run-type eval --run-type eval PREFIX seed=100
# Interactively visualize results
python mobile_manipulation/eval_composite.py --cfg configs/rearrange/composite/tidy_house/mr.yaml --viewer --render-info
# Save results
# "--no-rgb" can be passed to the script to accelerate if rgb is not used.
python mobile_manipulation/eval_composite.py --cfg configs/rearrange/composite/tidy_house/mr.yaml --save-log
# Save videos
python mobile_manipulation/eval_composite.py --cfg configs/rearrange/composite/tidy_house/mr.yaml --save-video all
We provide a script to evaluate a HAB task for multiple runs (different training and evaluation seeds).
BG=1 bash scripts/eval_composite.sh configs/rearrange/composite/tidy_house/mr.yaml
To collect evaluation results of multiple runs:
# By default, there will also be a timestamp in the path
python scripts/collect_results.py -d data/results/rearrange/composite/tidy_house/mr
Train an individual skill (e.g., Pick for TidyHouse):
# The result will be saved at "data/results/rearrange/skills/tidy_house/pick_v1_joint_SCR".
python mobile_manipulation/run_ppo.py --cfg configs/rearrange/skills/tidy_house/pick_v1_joint_SCR.yaml --run-type train
# Specify a prefix and a random seed for the experiment.
# The result will be saved at "data/results/rearrange/skills/tidy_house/pick_v1_joint_SCR/seed=101"
python mobile_manipulation/run_ppo.py --cfg configs/rearrange/skills/tidy_house/pick_v1_joint_SCR.yaml --run-type train PREFIX seed=101 TASK_CONFIG.SEED 101
Many thanks to Jiayuan Gu, Devendra Singh Chaplot, Hao Su, Jitendra Malik, authors of the original repository.