openhumanoids / oh-distro

An integrated humanoid control, planning and perception system. Developed by MIT and the University of Edinburgh for the Boston Dynamics Atlas and the NASA Valkyrie humanoid robots

Warning: Don't expect this to work!

We're releasing most of the source code from the MIT DRC project in the hope that it will benefit the robotics community. But there are parts of this software, like the Boston Dynamics Atlas software interface, which we are not allowed to release publicly. As a result, this public repo is incomplete. Some of the submodules and external projects are private, and you won't be able to access them unless you're a member of the team. Sorry!

We're actively working on making this a project that can be used by people outside the group, but for now, you should consider it a collection of (potentially) interesting code, not a functional application.

The core algorithms and tools, however, live in their own projects which are much better supported:

• Drake (planning, control, simulation, optimization): http://drake.mit.edu
• Pronto (state estimation): https://github.com/mitdrc/pronto
• Director (user interface): https://github.com/RobotLocomotion/director

OpenHumanoids

Table of Contents

• Warning: Don't expect this to work!
• OpenHumanoids
  • Introduction
    • Background
    • System Requirements
  • Download Instructions
    • Install Git
    • Getting Access
    • Download the source code
  • Dependencies
    • Required Packages
    • LCM Dependency
    • Install Matlab
    • Instructions for MOSEK
  • Build Instructions
    • Environment Setup
    • Matlab Environment Setup
    • Compiling
    • Instructions for GUROBI
  • ROS
  • Using system-installed PCL and OpenCV
  • Using externally installed Gurobi
  • Build Issues FAQ

Introduction

OpenHumanoids is the software platform developed by the MIT DRC team and used to compete in successive phases of the DARPA Robotics Challenge with the Boston Dynamics Atlas robot. It also full supports the NASA Valkyrie and several fixed base arms.

This README describes how to download and build the OpenHumanoids source code and how to satisfy 3rd party dependencies.

Background

MIT DRC youtube video

System Requirements

These instructions are written for Ubuntu 14.04 64-bit.

Download Instructions

Install Git

The DRC source code is stored in a Git repository. To download the source code you may need to first install Git on your system:

sudo apt-get install git gitk git-gui

Getting Access

You may need permission to access the git repository hosted on GitHub. To do so, create a GitHub account at GitHub.com if you don't already have one.

Next, add your public SSH key to your GitHub account so that you can easily push and pull over SSH. Read the generating ssh keys article for instructions to generate and link an ssh key to your account.

Download the source code

Download the repository with the git clone command and cd into the distro:

git clone [email protected]:openhumanoids/oh-distro.git && cd oh-distro

If you are not a member of the OpenHumanoids organization, please deinit one private submodule or else the following command will fail:

git submodule deinit catkin_ws/src/exotica-dev && git rm catkin_ws/src/exotica-dev

Initialize the submodules (Drake, director, pronto):

git submodule update --init --recursive

Add the sandbox remote. The sandbox is the location where branches can be shared.

git remote add sandbox [email protected]:oh-dev/oh-distro.git
git fetch sandbox

Dependencies

Required Packages

There are required 3rd party dependencies that can be satisfied by installing packages on Ubuntu. Install with the following commands:

sudo apt-get update

sudo apt-get install build-essential cmake debhelper freeglut3-dev gtk-doc-tools libboost-filesystem-dev libboost-iostreams-dev libboost-program-options-dev libboost-random-dev libboost-regex-dev libboost-signals-dev libboost-system-dev libboost-thread-dev libcurl4-openssl-dev libfreeimage-dev libglew-dev libgtkmm-2.4-dev libltdl-dev libgsl0-dev libportmidi-dev libprotobuf-dev libprotoc-dev libqt4-dev libqwt-dev libtar-dev libtbb-dev libtinyxml-dev libxml2-dev ncurses-dev pkg-config protobuf-compiler python-matplotlib libvtk5.8 libvtk5-dev libvtk5-qt4-dev libqhull-dev python-pygame doxygen mercurial libglib2.0-dev openjdk-6-jdk python-dev gfortran f2c libf2c2-dev spacenavd libspnav-dev python-numpy python-scipy python-yaml python-vtk python-pip libgmp3-dev libblas-dev liblapack-dev libv4l-dev subversion libxmu-dev libusb-1.0-0-dev python-pymodbus graphviz curl libwww-perl libterm-readkey-perl libx264-dev libopenni-dev swig libqglviewer-dev libsuitesparse-dev libsdl1.2-dev

LCM Dependency

LCM (v1.1.0) is a required dependency which must be installed from source. It can be retrieved from http://lcm-proj.github.io/

wget https://github.com/lcm-proj/lcm/releases/download/v1.1.0/lcm-1.1.0.zip
unzip lcm-1.1.0.zip
cd lcm-1.1.0
./configure
make
sudo make install

LCM v1.1.1 is known to be bad. Do not try to use it.

Install Matlab

Download Matlab r2014a from Mathworks.com. Unzip the file you just downloaded (e.g., unzip ./R2014a-linux64.zip) cd into the resulting directory sudo ./install When prompted for how to install, choose "Log in with a MathWorks Account."

Newer versions of Matlab are known to not play nicely with our code.

Choose a "Typical" install and click next through the rest of the process. You will need to enter your Mathworks username and password during the install process, and you should see a single license that you can use for the install (this comes from a lookup of the activation key). You should have a functional MATLAB in /usr/local/MATLAB/R2014a/bin now. You can either add this directory to your PATH environment variable (e.g. in ~/.bashrc) or you can make a symlink in /usr/local/bin/ that points to the MATLAB binary - sudo ln -s /usr/local/MATLAB/R2014a/bin/matlab /usr/local/bin/matlab. If you put it in .bashrc, you'll need to source that file before matlab will be in your path (or, just start a new shell)

After installing MATLAB, two of the symlinks for libraries need to be changed:

cd /usr/local/MATLAB/R2014a/sys/os/glnxa64
ls -l

The symbolic links for libstdc++.so.6 and libgfortran.so.3 should point to versions in /usr/lib, not local ones.

Before changing this libraries, first make sure g++ 4.4 is installed:

sudo apt-get install g++-4.4

Now, modify the symlinks:

sudo rm libgfortran.so.3
sudo ln -s /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0 libgfortran.so.3
sudo rm libstdc++.so.6
sudo ln -s /usr/lib/gcc/x86_64-linux-gnu/4.4/libstdc++.so libstdc++.so.6

Instructions for MOSEK

Mosek is a solver used in the footstep planner. Obtain an academic licence from http://license.mosek.com/academic Check your email and place your license in ~/mosek/mosek.lic The Mosek code is checked out as part of the project externasl

Build Instructions

Environment Setup

The behavior of certain build steps can be affected by environment variables, so you should setup your environment before starting the build. The environment is setup by sourcing the file oh-distro/software/config/drc_environment.sh. Typically, users will source this file automatically in their ~/.bashrc file by adding this line to ~/.bashrc:

source /path-to/oh-distro/software/config/drc_environment.sh

If you have already done this, make sure your ~/.bashrc contains the correct path to the drc_environment.sh file in the oh-distro source code directory that you just cloned with git.

Matlab Environment Setup

Create a file ~/Documents/MATLAB/startup.m that contains the line:

run([getenv('DRC_BASE'), '/software/build/config/drc_control_setup.m'])

Compiling

Make sure you have sourced the drc_environment.sh file to setup the DRC environment prior to building. If you did not source the file automatically in ~/.bashrc, then do so now with the following command:

cd oh-distro
source software/config/drc_environment.sh

If you do not have access to private external submodules such as Gurobi, Snopt, or the Atlas drivers, you need to turn off BUILD_PRIVATE_EXTERNALS:

cd oh-distro/software/externals
mkdir pod-build && cd pod-build
cmake .. -DBUILD_PRIVATE_EXTERNALS:BOOL=OFF
cd ..
make -j
cd ..
make -j

Please make sure to install Gurobi and Snopt manually.

If you are a member of the OpenHumanoids organization, run make to build externals and then the main codebase:

cd software/externals
make -j
cd ..
make -j

Instructions for GUROBI

Gurobi is a solver used in our walking controller. Install its dependencies with the following commands:

apt-get install curl libwww-perl libterm-readkey-perl

Then generate an academic licence: First make an account http://www.gurobi.com/download/licenses/free-academic , then use the Gurobi key client (grbgetkey) to store the license on your machine. Place it in the suggested location (~/gurobi.lic)

The grbgetkey module is built as part of the externals.

Note that the tarball for Gurobi is part of our tree and the gurobi pod uses it to avoid needing to download it from Gurobi.

ROS

ROS is not required per se. If you would like to use this distribution in conjunction with IHMC's SCS, your own controllers for Valkyrie, or to use EXOTica for planning and optimization, please install ROS Indigo including MoveIt and ROS-Control. Valkyrie uses ROS-Control for the Hardware API and our LCM2ROSControl translator package hence requires ROS Control.

sudo apt-get install ros-indigo-desktop-full ros-indigo-moveit-full ros-indigo-ros-control

Compile catkin workspace:

cd $DRC_BASE/catkin_ws
catkin_make -DCMAKE_BUILD_TYPE=RelWithDebInfo

Before you run any ROS code from the catkin workspace, source the setup script:

source catkin_ws/devel/setup.bash

Using system-installed PCL and OpenCV

PCL (version 1.7.1) and OpenCV (~2.4.12.2) can also be system-installed instead of being built from source in the externals build step resulting in considerable build time savings. In order to do this:

Add a PPA and install PCL:

sudo add-apt-repository -y ppa:v-launchpad-jochen-sprickerhof-de/pcl
sudo apt-get update
sudo apt-get install -y libpcl-1.7-all

Use e.g. our in-house packaged version of OpenCV 2.4.12.3 that matches the one being built from source as part of the externals build step:

wget http://terminator.robots.inf.ed.ac.uk/public/opencv_2.4.12.2.deb
sudo dpkg -i opencv_2.4.12.2.deb

Note: Using the in-house packaged version of OpenCV requires libdc1394-22-dev to be installed (sudo apt-get install libdc1394-22-dev).

These options are disabled by default on purpose. Then, to make use of the system-installed versions, configure the externals build with the corresponding flags:

cd software/externals
mkdir pod-build && cd pod-build
cmake .. -DUSE_SYSTEM_PCL:BOOL=ON -DUSE_SYSTEM_OPENCV:BOOL=ON

Using externally installed Gurobi

If installing your own version of Gurobi, please add the following to your .bashrc file before compiling oh-distro (and after sourcing drc_environment.sh since these variables will also be set when sourcing our drc_environment config):

export GUROBI_HOME=[PATH_TO]/gurobi/gurobi562/linux64
export PATH=$PATH:$GUROBI_HOME/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$GUROBI_HOME/lib
export GRB_LICENSE_FILE=$HOME/gurobi.lic

Build Issues FAQ

ISSUE: make in externals failed:

• REASON: A submodule has been updated
• RESOLUTION:
  • retry (make -j 1) and see which module failed
  • remove the relevent module from pod build: rm pod-build/src/[module] pod-build/tmp/[module]
  • continue making externals

ISSUE: drc_lcmtypes fails to build showing something like:

• .../oh-distro/software/drc_lcmtypes/lcmtypes/drc_robot_command_t.lcm: No such file or directory
• make[3]: *** [CMakeFiles/lcmgen_c] Error 255
• REASON: An LCM type has been removed or added.
• RESOLUTION:
  • make clean in drc_lcmtypes and then continue

ISSUE: drake won't build.

• REASON: Drake is specially configured to build with Open Humanoids. Never make in software/drake, always make in software/drake/drake/
• RESOLUTION: Make a clean build of drake:

cd <path-to>/oh-distro/software
rm drake
cd externals
rm pod-build/src/drake-cmake-* pod-build/tmp/drake-cmake-* -Rf
git submodule update --init --recursive
cd externals
make -j 1
cd software/drake/drake
make -j

oh-distro's Issues

Building w/o private externals access

Could not build the controls directory without SNOPT.

I downloaded Gurobi outside of the OH build process, but there were no instructions on how to link the compiler to my external Gurobi installation.

Could not run build Director without the Controls directory and some Atlas drivers.

Spoke to Russ about setting up OpenHumanoids with a precompiled binary for SNOPT.

remove hard coded joints names

These two modules should be removed:
oh-distro/software/pronto/pronto-utils/src/pronto_joint_tools
oh-distro/software/utils/drc_utils/src/joint_utils
... basically just a header with the joint names.

They are used in a few places but aren't needed. They exist from when ATLAS_STATE (now CORE_ROBOT_STATE) didn't have joint names in it.

develop State Estimation message passing for Valkyrie

@gizatt : I've summarised the messaging we are looking to put in place here:
https://github.com/openhumanoids/oh-distro/wiki/State-Estimation-Messaging

I'll try to run your translator while manually moving the robot on the winch to get a log to patch all this up.

This is also related ( @raluca-scona ):
https://github.com/openhumanoids/main-distro/issues/2285

Optimise kinematic calibration

We did some static tests on Friday with the robot attached to the NASA Gantry:

This is what the head-to-hand calibration looks like:

Note that the lower arm was unpowered and the forearm yaw was aligned by eye and the fingers were not reporting their states. this shows the default finger positions. (the URDF/default finger positions make a 'Vulcan salute'). I think this is very promising.

But running some optimisation (sic) is likely help. An initial target is to achieve repeatable sub 1cm accuracy

Controller creates NaN when moving into single foot balancing

The controller segfaults when transitioning into single foot balancing. The video should explain it but:

• move mass onto one foot, works fine
• remove other foot from control authority, robot falls. InstQP reports a NaN
  This used to work before the recent upgrade of drake in January.

@tkoolen or @rdeits : could you have a look?
I guess the intention is to replace this with a c++ alternative.

make an ihmc panel to choose control modes

choose either whole body control, left arm, right arm or both

DRCPlanner with Valkyrie not working

After updating to drake master:

>> addpath_control;
>> BasePlanner.withValkyrie(2);
Error using Valkyrie
Abstract classes cannot be instantiated.  Class 'Valkyrie' inherits abstract methods or properties but does not implement them.  See the list of methods and properties that 'Valkyrie' must implement if you do not intend the class to be
abstract.

Error in DRCPlanner.constructValkyrie (line 38)
      r = Valkyrie([], options);

Error in BasePlanner.withValkyrie (line 8)
      obj = BasePlanner(BasePlanner.constructValkyrie(varargin{:}));

Investigate old dependencies

When copy-pasting the dependency list on a fresh install of trusty-desktop, unity-control-center gets damaged (via libcheese-gtk). Initial investigation narrowed it down to freeglut3-dev being the first package that exhibits the issue (i.e. cannot install all dependencies in one go). [This has now shown up on 3+ machines here, and does not show up on the Travis servers, hence it must be a desktop distribution issue]

Running

sudo apt-get install  libglew-dev libcheese7 libcheese-gtk23 libclutter-gst-2.0-0 libcogl15 libclutter-gtk-1.0-0 libclutter-1.0-0  xserver-xorg-input-all

before the dependency list fixes the issue for now. May need further investigation which package actually causes the issue, and which packages in our dependency list are unneeded/can be dropped/need to be upgraded.

Complete Octomap support

• render using latest version of octomap in director
• add support and controls to UI e.g. change color modes
• switch remaining uses from octomap_raw_t to drc_map_octree_t
• have the map server renderer use new rendering code, but also be able to access polydata
• Have map server publish color octrees. (Will need to check on ROS/Exotica side)

Issue: Drawing only a portion of the height of the octree is not available through the Octomap rendering code. (e.g. to remove the ceiling of an octree). It was available previously with the gtk version

MAP_DEPTH - heightmaps, depthmaps
MAP_OCTREE - octree + transform map_octree_t
MAP_CLOUD - point cloud

POINTS_COLLECTION
OBJECT_COLLECTION

OCTOMAP*** - octree + transform octomap_raw_t

General publishing of point clouds to UI doesn't exist
MAP_OCTREE and OCTOMAP render different things

• different messages

Future of control directory

@rdeits has been pulling the InstQP into drake and making it C++. Is there a future for the 'control' directory? A lot of its is stale matlab and quite a bit of it hasn't been touched since the Trials.

We had talked about this briefly at the DRC. @rdeits : Is it possible to simple ditch it going forward?

For example @MarcoCar has started (in his branch) a separate directory for motion planning (using Drake) which never had a specific home.

Atlas lcmtypes error when building w/o atlas_drivers external

I am trying to build OH without access to the atlas_drivers external (but have access to SNOPT and built gurobi on its own) and am getting the following errors:

[ 14%] Building CXX object src/CMakeFiles/AtlasFallDetector.dir/AtlasFallDetector.cpp.o
In file included from /home/rc/oh-distro/software/control/src/AtlasFallDetector.cpp:1:0:
/home/rc/oh-distro/software/control/src/AtlasFallDetector.hpp:12:39: fatal error: lcmtypes/atlas/status_t.hpp: No such file or directory
#include "lcmtypes/atlas/status_t.hpp"
^
compilation terminated.
make[4]: *** [src/CMakeFiles/AtlasFallDetector.dir/AtlasFallDetector.cpp.o] Error 1
make[3]: *** [src/CMakeFiles/AtlasFallDetector.dir/all] Error 2
make[2]: *** [all] Error 2
make[1]: *** [all] Error 2
make: *** [all] Error 2

I tried cd'ing into drc_lcmtypes, doing a make clean, then make again in /software. That didn't work.

Occ-map compilation error (using system-installed OpenCV)

During compilation, cloning OpenCV would freeze and quit after 3 attempts. Instead I used the system-installed instructions for OpenCV. When compiling, I got the following errors:

"Linking CXX executable ../../bin/occ-map-img-to-pixmap
/usr/bin/ld: warning: libdc1394.so.22, needed by /usr/local/lib/libopencv_highgui.so, not found (try using -rpath or -rpath-link)
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_video_set_transmission' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_feature_set_value'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_video_set_iso_speed' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_feature_set_absolute_control'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_feature_set_power' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_feature_get_value'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_get_color_coding_from_video_mode' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_camera_free'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_convert_frames' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_camera_free_list'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_capture_is_frame_corrupt' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_video_set_operation_mode'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_video_get_supported_modes' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_debayer_frames'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_set_control_registers' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_free'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_video_get_mode' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_camera_new'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_video_get_supported_framerates' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_feature_get_all'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_get_image_size_from_video_mode' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_feature_whitebalance_set_value'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_video_set_mode' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_capture_dequeue'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_deinterlace_stereo' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_capture_enqueue'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_capture_stop' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_capture_setup'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_feature_set_mode' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_get_control_registers'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_capture_get_fileno' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_video_set_framerate'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_get_color_coding_bit_size' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_feature_whitebalance_get_value'
/usr/local/lib/libopencv_highgui.so: undefined reference to dc1394_new' /usr/local/lib/libopencv_highgui.so: undefined reference todc1394_camera_enumerate'
collect2: error: ld returned 1 exit status
make[7]: *** [bin/occ-map-img-to-pixmap] Error 1
make[6]: *** [src/map_server/CMakeFiles/occ-map-img-to-pixmap.dir/all] Error 2
make[5]: *** [all] Error 2
make[4]: *** [all] Error 2
make[3]: *** [src/occ-map-stamp/occ-map-build] Error 2
make[2]: *** [CMakeFiles/occ-map.dir/all] Error 2
make[1]: *** [all] Error 2
make: *** [all] Error 2
"

Broken tests after switching to Valkyrie default

Two tests have been broken after switching to Valkyrie as the default robot:

testTelopPanel.py
http://kobol.csail.mit.edu/cd/testDetails.php?test=3491786&build=19960

testContinuousWalking.py
http://kobol.csail.mit.edu/cd/testDetails.php?test=3491780&build=19960

Either we should add args to these tests so that they use Atlas v5, or we should update the tests to correctly use the Valkyrie model. Even better, adapt the tests so they pass with Atlas or Valkyrie, then run the tests in both modes!

Moving development to oh-distro-private

Previously Pat and us folks in Edinburgh discussed moving back to private development.

I have just set oh-distro to be read only. Please move your pull requests and branches to:
https://github.com/openhumanoids/oh-distro-private

The working assumption is to push master of private to public occasionally.

Redesign drc_robot_command_t

As discussed with @gizatt , we will re work this to be a set of vectors of properties
rather than a vector of sets of properties - as we previously used to do.
(This fits with current tools and ROS conventions)

I'll work on this.

"Public" build of oh-distro

The idea is to have the public (i.e. non OH organisation members) be able to build oh-distro without access to private externals, with no guarantee of functionality though.

The steps hereto are:

• disable matlab-dependent parts when matlab is not present (#37)
• disable atlas-collection by default (follow-up/last step in #30)
• resolve flycapture, e.g. either move to atlas-collection or disable by default in drivers/tobuild.txt
• skip ptgrey if flycapture is not found, e.g. using cmake if-not-found-return or also move to atlas-collection/flycapture

Any thoughts on the last two points @patmarion, @mauricefallon ?

State estimation on Valkyrie

We would like to have a magnetometer field within the val_hardware_msgs/valImuSensor message type.

Encapsulate Atlas specific API and types

I intend to gather up all Atlas specific code into a single place such that:

• software builds without drc_atlas lcm types. There are 30 of these currently.
• software builds without BDI proprietary library
• this will also simplify the dependency tree
• move us a couple of steps closer to full open source (i.e. checkout and build)

The steps:

• Fish out the 30 LCM types into a pod
• Move all dependent code into a single place: state-sync, bdi-footstep-translator.
• Fix some minor dependencies on these types e.g. signal scope and the fall detector
• Gather the dependent code and types with the driver (and disable by default)

Then during Valkyrie development, we will avoid the intermingling of our software and the manufacturer's API and proprietary files.

Add ability to have different robot versions co-exist

State estimation in the Val SCS simulator is broekn

Currently pronto isn't properly working with simulated sensor signals from Val. (Atlas mean while is perfect).

The robot is rotating strangely. Likely this is because SCS uses a different urdf than oh-distro and the IMUs are in different places.

Just letting you know @raluca-scona , no need to look into this now