mees / generalize_spatial_relations

[IROS 2017] Code for the paper "Metric Learning for Generalizing Spatial Relations to New Objects"

Metric Learning for Generalizing Spatial Relations to New Objects

This repository implements the code for describing and generalizing spatial relations of our paper published at IROS 2017. More information at our project page.

Reference

If you find the code helpful please consider citing our work

@INPROCEEDINGS{mees17iros,
  author = {Oier Mees and Nichola Abdo and Mladen Mazuran and Wolfram Burgard},
  title = {Metric Learning for Generalizing Spatial Relations to New Objects},
  booktitle = {Proceedings of the International Conference on Intelligent Robots and Systems (IROS)},
  year = 2017,
  address = {Vancouver, Canada},
  url = {http://ais.informatik.uni-freiburg.de/publications/papers/mees17iros.pdf},
}

Dataset

The Freiburg Spatial Relations dataset features 546 scenes each containing two out of 25 household objects. The depicted spatial relations can roughly be described as on top, on top on the corner, inside, inside and inclined, next to, and inclined. The dataset contains the 25 object models as textured .obj and .dae files, a low resolution .dae version for visualization in rviz, a scene description file containing the translation and rotation of the objects for each scene, a file with labels for each scene, the 15 splits used for cross validation, and a bash script to convert the models to pointclouds.

Installation

Install Glog, PCL, Eigen3 and OctoMap.

• Download the dataset and extract it such that the structure looks like generalize_spatial_relations/data/object_models/

• Navigate to the relations dataset and create the point clouds from the .obj files (needs pcl): cd scripts; ./create_uniform_pcd.sh

• Put your directories path here.

• Compile the c++ code:

  mkdir build
  cd build
  cmake ..
  make -j4
  

Experiments

To generalize relation from one or multiple scenes to another, take a look at imitateRelationsFast.cpp. It uses a precomputed LMNN embedding to compute distances between spatial relations. For example to generalize the scenes spearmint_salz_2662 and pot4_milkCarton_6394 with a box object as base and a cereal box as the relative object:

./imitateRelationFast -b SmallBoxBase -o muesli2 -i 8868 -1 spearmint_salz_2662 -2 pot4_milkCarton_6394

If you are just interested in computing our spatial relation feature descriptor given two object clouds, take a look here.

Notes

If you want to learn the features describing pairwise object relations with deep learning and optimize the generalization experiments on a continuous level check out the repo of our follow-up work here.

License

For academic usage, the code is released under the GPLv3 license. For any commercial purpose, please contact the authors.