okstill / learninghumanoidwalking

Training a humanoid robot for locomotion using Reinforcement Learning

LearningHumanoidWalking

Code for the papers:

• Learning Bipedal Walking On Planned Footsteps For Humanoid Robots (Humanoids2022)
  Rohan P. Singh, Mehdi Benallegue, Mitsuharu Morisawa, Rafael Cisneros, Fumio Kanehiro

• Learning Bipedal Walking for Humanoids with Current Feedback (arxiv)
  Rohan P. Singh, Zhaoming Xie, Pierre Gergondet, Fumio Kanehiro
  (WIP on branch topic/omnidirectional-walk)

Code structure:

A rough outline for the repository that might be useful for adding your own robot:

LearningHumanoidWalking/
├── envs/                <-- Actions and observation space, PD gains, simulation step, control decimation, init, ...
├── tasks/               <-- Reward function, termination conditions, and more...
├── rl/                  <-- Code for PPO, actor/critic networks, observation normalization process...
├── models/              <-- MuJoCo model files: XMLs/meshes/textures
├── trained/             <-- Contains pretrained model for JVRC
└── scripts/             <-- Utility scripts, etc.

Requirements:

• Python version: 3.7.11
• Pytorch
• pip install:
  • mujoco==2.2.0
  • mujoco-python-viewer
  • ray==1.9.2
  • transforms3d
  • matplotlib
  • scipy

Usage:

Environment names supported:

Task Description	Environment name
Basic Walking Task	'jvrc_walk'
Stepping Task (using footsteps)	'jvrc_step'

To train:

$ python run_experiment.py train --logdir <path_to_exp_dir> --num_procs <num_of_cpu_procs> --env <name_of_environment>

To play:

We need to write a script specific to each environment.
For example, debug_stepper.py can be used with the jvrc_step environment.

$ PYTHONPATH=.:$PYTHONPATH python scripts/debug_stepper.py --path <path_to_exp_dir>

What you should see:

Ascending stairs:

Descending stairs:

Walking on curves:

Citation

If you find this work useful in your own research:

@inproceedings{singh2022learning,
  title={Learning Bipedal Walking On Planned Footsteps For Humanoid Robots},
  author={Singh, Rohan P and Benallegue, Mehdi and Morisawa, Mitsuharu and Cisneros, Rafael and Kanehiro, Fumio},
  booktitle={2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids)},
  pages={686--693},
  year={2022},
  organization={IEEE}
}