ethz-asl / scenenet_to_rosbag

Dataset tools for converting the SceneNet RGB-D dataset raw sequence data to a ROS bag.

Interfacing SceneNet and ROS

Dataset tools for working with the SceneNet RGB-D dataset and converting its raw trajectory data to a ROS bag.

How to use these tools

1. Clone this and the pySceneNetRGBD repositories to the src folder of your catkin workspace, build your workspace and source it.

   cd <catkin_ws>/src
   git clone [email protected]:ethz-asl/scenenet_ros_tools.git
   git clone [email protected]:jmccormac/pySceneNetRGBD.git
   catkin build
   source <catkin_ws>/devel/setup.bash

2. Download the SceneNet training (263GB split into 17 tarballs of 16GB each) and/or validation set (15GB) with the respective protobuf files to the data directory of the pySceneNetRGBD folder, then run make in the root pySceneNetRGBD folder to generate the protobuf description.

   cd pySceneNetRGBD
   mkdir data && cd data
   
   # Training set
   wget https://www.doc.ic.ac.uk/~bjm113/scenenet_data/train_protobufs.tar.gz train_protobufs.tar.gz
   tar -xvzf train_protobufs.tar.gz --strip=1
   wget https://www.doc.ic.ac.uk/~bjm113/scenenet_data/train_split/train_0.tar.gz train_0.tar.gz
   tar -xvzf train_0.tar.gz
   wget https://www.doc.ic.ac.uk/~bjm113/scenenet_data/train_split/train_1.tar.gz train_1.tar.gz
   tar -xvzf train_1.tar.gz
   ...
   
   # Validation set
   wget http://www.doc.ic.ac.uk/~bjm113/scenenet_data/scenenet_rgbd_val.pb scenenet_rgbd_val.pb
   wget http://www.doc.ic.ac.uk/~bjm113/scenenet_data/SceneNetRGBD-val.tar.gz SceneNetRGBD-val.tar.gz
   tar -xvzf SceneNetRGBD-val.tar.gz
   
   cd .. && make

3. Make the Python script executable and run it as a ROS node to convert data from a SceneNet trajectory to a rosbag. The rosbag will contain a sequence of RGB and depth images, ground truth 2D instance label images, and relative transforms. Optionally, it can contain colorized ground truth 2D instance label images, colored pointclouds of the scene, and colored pointclouds of ground truth instance segments.

   cd ../scenenet_ros_tools && chmod +x nodes/scenenet_to_rosbag.py
   rosrun scenenet_ros_tools scenenet_to_rosbag.py --scenenet-path PATH/TO/pySceneNetRGBD --dataset-type DATASET_TYPE --trajectory INDEX  [--train-set-split N] [--limit NUM] [--output-bag NAME]

   Example for a trajectory from the training set:

   rosrun scenenet_ros_tools  scenenet_to_rosbag.py --scenenet-path ../pySceneNetRGBD/ --dataset-type train --train-set-split 0 --trajectory 1  --output-bag scenenet_train_0_traj_1.bag

   Example for a trajectory from the validation set:

   rosrun scenenet_ros_tools  scenenet_to_rosbag.py --scenenet-path ../pySceneNetRGBD/ --dataset-type val --trajectory 1 --output-bag scenenet_val_traj_1.bag

   The output bag contains the following topics:

   # RGB and depth images
   /camera/rgb/camera_info         : sensor_msgs/CameraInfo
   /camera/rgb/image_raw           : sensor_msgs/Image
   /camera/depth/camera_info       : sensor_msgs/CameraInfo
   /camera/depth/image_raw         : sensor_msgs/Image        
   
   # Ground truth 2D instance segmentation image
   /camera/instances/image_raw     : sensor_msgs/Image
   
   # Ground truth colorized 2D instance segmentation image [Disabled by default]
   /camera/instances/image_rgb     : sensor_msgs/Image
   
   # Colored pointclouds of ground truth instance segments [Disabled by default]
   /scenenet_node/object_segment   : sensor_msgs/PointCloud2
   
   # Colored pointcloud of the scene                       [Disabled by default]
   /scenenet_node/scene            : sensor_msgs/PointCloud2
   
   # Transform from /scenenet_camera_frame to /world
   /tf                             : tf/tfMessage

scenenet_to_rosbag's Issues

Create script that sets everything up

• Automatically download the pySceneNetRGBD
• Depend on https://github.com/ethz-asl/protobuf_catkin
• Add script input to select various datasets to download

fix repository folder structure

Add arguments to avoid hard-coding of protobuf_path and data_root_path

Would it be possible to add two arguments to avoid hard-coding the path to the protobuf file and the type of dataset ('train' or 'val')? Something like francescomilano172@0a18291 (forked repo)? I find this handy when generating data automatically from a script.

Some updates?

Thanks for your work.
But I met some problems to download the file.
When I run

wget http://www.doc.ic.ac.uk/~ahanda/train_protobufs.tar.gz train_protobufs.tar.gz

The error is

URL transformed to HTTPS due to an HSTS policy
--2020-06-13 11:39:00--  https://www.doc.ic.ac.uk/~ahanda/train_protobufs.tar.gz
Resolving www.doc.ic.ac.uk (www.doc.ic.ac.uk)... 146.169.13.6
Connecting to www.doc.ic.ac.uk (www.doc.ic.ac.uk)|146.169.13.6|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2020-06-13 11:39:02 ERROR 403: Forbidden.

--2020-06-13 11:39:02--  http://train_protobufs.tar.gz/
Resolving train_protobufs.tar.gz (train_protobufs.tar.gz)... failed: Name or service not known.
wget: unable to resolve host address ‘train_protobufs.tar.gz’

Then I go to the scenenet website to download train_protobufs.tar.gz manually then I tried

tar -xvzf train_protobufs.tar.gz --strip=1

Then errors again

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

According to the error, the file is not in gzip format. Then I use

tar xvf train_protobufs.tar.gz --strip=1

to uncompress it but here I am not sure if can yield the same thing as your original command tar xvzf ... because I don't the the meaning of --strip==1

Instance publishing

Here why not publish instance value directly ? What's the purpose of using mono_to_rgb ?

if (publish_instances):
     # Publish the instance data.
     color_instance_image = mono_to_rgb(instance_image)
     color_instance_msg = cvbridge.cv2_to_imgmsg(
         color_instance_image, "bgr8")
     color_instance_msg.header = header
     output_bag.write('/camera/instances/image_raw', color_instance_msg,
                      timestamp)