python-repository-hub / turtlebot3_slam_3d

Turtlebot3 3D-SLAM demo using RTAB-Map with Jetson TX2 and ZED Mini

Turtlebot3 3D-SLAM using RTAB-Map with Jetson TX2

Sample repository for creating a three dimensional map of the environment in real-time and navigating through it. Object detection using YOLO is also performed, showing how neural networks can be used to take advantage of the image database stored by RTAB-Map and use it to e.g. localize objects in the map.

Screencast:

Click for better quality.

Objects found by the robot are mapped as below:

Quick Start:

mkdir catkin_ws/src -p
cd catkin_ws
wstool init src
wstool merge -t src https://raw.githubusercontent.com/ROBOTIS-JAPAN-GIT/turtlebot3_slam_3d/master/.rosinstall
wstool update -t src
rosdep install -y -r --from-paths src --ignore-src
catkin build turtlebot3_slam_3d

Launching Demo Program

Download one of the bag files:

# Small version (300MB)
https://drive.google.com/a/robotis.com/uc?export=download&confirm=SLgY&id=1sfMhQV5ipJm0ghrvQ8HpOw2tTr179aiP

# Full version (1.2GB)
https://drive.google.com/a/robotis.com/uc?export=download&confirm=BkOw&id=1BUQdcuxshEC-W6O9Jkel6sUP9-FxkLca

Setup the environment with:

cd catkin_ws
source devel/setup.bash
export ROS_PACKAGE_PATH=$ROS_PACKAGE_PATH:$(pwd)/src

Then, launch the sample mapping demo with the following. It takes a while to load the nodes and to play the bag file.

roslaunch turtlebot3_slam_3d demo_bag.launch bag_file:=/absolute/path/to/bag_file.bag

Default bag_file path is set to turtlebot3_slam_3d/rtab_bag.bag.

Map is saved to ~/.ros/rtabmap.db as default setting.

Generate Semantic Map

Semantic mapping can be generated using the code in the scripts directory. We perform this procedure offline to save calculation time during execution and to make the clustering algorithm easier.

To generate a detection database from a bag file:

1. Start the bag file:

roslaunch turtlebot3_slam_3d demo_bag.launch bag_file:=/absolute/path/to/bag_file.bag

2. Wait for the bag to load and for the robot to be displayed properly.
3. Run the collector script. This will collect detection results and save them to files in the current directory (detections_raw.db and detections_dbscan.db).

rosrun turtlebot3_slam_3d detection_collector.py 

4. Wait until the rosbag play has finished and all detections have been collected.

To display the detected objects in the map, start the bag file with publish_detection:=true.

$ roslaunch turtlebot3_slam_3d demo_bag.launch publish_detection:=true detection_db:=/absolute/path/to/detections_dbscan.db

Run with Turtlebot3

Hardware Components:

	Part name	Quantity
Main Components	Waffle Pi	1
	Jetson TX2	1
	ZED Mini or RealSense D435	1
Supplementary Items	19V Battery for TX2	1
	USB Hub (micro-B to Type A)	1
Camera Bracket	ZED Mini Camera Bracket	1
	M2x4mm	2
	M3x8mm	2
Chassis Parts	Waffle Plate	3
	Plate Support M3x45mm	11
	M3x8mm	22

Software Components:

ZED Mini Software Setup

Download ZED SDK for Jetson TX2 from https://www.stereolabs.com/developers/. Then, build the ROS wrapper with:

cd catkin_ws
catkin build zed_wrapper

Intel RealSense D435 Software Setup

Run the installation script from jetsonhacks to install the RealSense SDK on Jetson TX2:

cd catkin_ws/src/buildLibrealsense2TX
./installLibrealsense.sh 

Then, build the ROS wrapper with:

cd catkin_ws
catkin build realsense2_camera

Launch Files:

First, setup the environment:

cd catkin_ws
source devel/setup.bash
export ROS_PACKAGE_PATH=$ROS_PACKAGE_PATH:$(pwd)/src

To create your own map, run:

roslaunch turtlebot3_slam_3d turtlebot3_slam_3d.launch

To navigate through your map use:

roslaunch turtlebot3_slam_3d turtlebot3_zed_bringup.launch
roslaunch turtlebot3_slam_3d rtabmap.launch localization:=true

With Intel RealSense D435, use the following instead:

roslaunch turtlebot3_slam_3d turtlebot3_slam_3d.launch use_zed:=false

roslaunch turtlebot3_slam_3d turtlebot3_d435_bringup.launch
roslaunch turtlebot3_slam_3d rtabmap.launch localization:=true use_zed:=false