bosshuan / sit-dataset

SiT Dataset: Socially Interactive Pedestrian Trajectory Dataset for Social Navigation Robots

SiT Dataset: Socially Interactive Pedestrian Trajectory Dataset for Social Navigation Robots

Example of SiT dataset

Updates

• [2023-05-29] We opened SiT Dataset Github.
• [2023-05-29] We opened SiT Dataset Website.
• [2023-06-18] Semantic map data of SiT Dataset released on Github.

Upcomings

• [2023-06] SiT Mini-Dataset release on public.
• [2023-06] Dockerfiles for each perception task release on Dockerhub.
• [2023-06] SiT benchmark and devkit for each perception task release.
• [2023-06] SiT Mini-Dataset Rosbag files release on public.
• [2023-09] SiT Full dataset with rosbag files release on public.
• [2023-09] Pretrained models for 3D object detection and Trajectory prediction release on public.
• [2023-10] Dockerfiles for each perception task release.
• [2024-01] SiT End-to-End pedestrain trajectory prediction challenge starts on Eval AI.
• [2023-07] Pretrained models for 3D object detection.

Overview

Our Social Interactive Trajectory (SiT) dataset is a unique collection of pedestrian trajectories for designing advanced social navigation robots. It includes a range of sensor data, annotations, and offers a unique perspective from a robot navigating crowded environments, capturing dynamic human-robot interactions. It's meticulously organized for training and evaluating models across tasks like 3D detection, 3D multi-object tracking, and trajectory prediction, providing an end-to-end modular approach. It includes a comprehensive benchmark and exhibits the performance of several baseline models. This dataset is a valuable resource for future pedestrian trajectory prediction research, supporting the development of safe and agile social navigation robots.

Robot Platform & Sensor Setup

• Ubuntu 18.04 LTS
• ROS Melodic
• Clearpath Husky UGV
• Velodyne VLP-16 * 2
• RGB Camera Basler a2A1920-51gc PRO GigE * 5
• MTi-680G IMU & GPS * 1
• VectorNAV VN-100 IMU * 1

Benchmarks

We provide benchmarks and pretrained models for 3D pedestrian detection, 3D Multi-Object Tracking, Pedestrian Trajectory Prediction and End-to-End Motion Forecasting.

3D Object Detection

Methods	Modality	mAP ↑	AP(0.25) ↑	AP(0.5) ↑	AP(1.0) ↑	AP(2.0) ↑	Pretrained
FCOS3D	Camera	0.131	0.054	0.147	0.162	0.162	TBD
PointPillars	LiDAR	0.319	0.202	0.316	0.346	0.414	TBD
CenterPoint-P	LiDAR	0.382	0.233	0.388	0.424	0.482	TBD
CenterPoint-V	LiDAR	0.514	0.352	0.522	0.556	0.620	TBD
Transfusion-P	Fusion	0.396	0.213	0.371	0.451	0.551	TBD
Transfusion-V	Fusion	0.533	0.360	0.512	0.587	0.672	TBD

3D Multi-Object Trajectory Tracking

Method	sAMOTA↑	AMOTA↑	AMOTP(m)↓	MOTA↑	MOTP(m)↓	IDS↓
PointPillars + AB3DMOT	0.3679	0.0826	0.5125	0.2073	0.9702	1048
Centerpoint Detector + AB3DMOT	0.4626	0.1159	0.3757	0.3438	0.8360	554
Centerpoint Tracker	0.7244	0.2793	0.2611	0.5150	0.4274	1136

Pedestrian Trajectory Prediction

Name	Map	ADE5 ↓	FDE5 ↓	ADE20 ↓	FDE20 ↓	Pretrained
Social-LSTM	X	1.336	2.554	1.319	2.519	TBD
Y-NET	X	1.188	2.427	0.640	1.547	TBD
Y-NET	O	1.036	2.306	0.596	1.370	TBD
NSP-SFM	X	1.036	1.947	0.529	0.936	TBD
NSP-SFM	O	0.808	1.549	0.443	0.807	TBD

End-to-End Pedestrian Motion Forecasting

Method	mAP ↑	mAPf ↑	ADE5 ↓	FDE5 ↓	Pretrained
Fast and Furious	0.490	0.079	1.915	3.273	TBD
FutureDet-P	0.209	0.037	2.532	4.537	TBD
FutureDet-V	0.408	0.053	2.416	4.409	TBD

Download Dataset

• Download SiT Mini dataset from below Google Drive link.
  Click Download link.
  

• Full dataset and Rosbag files will be uploaded(TBD).

ROS Bag Raw Data

ROS bagfiles include below sensor data:

Topic Name	Message Type	Message Descriptison
/29_camera/pylon_camera_node/ image_raw/compressed	sensor_msgs/CompressedImage	Compressed Bayer Image by Basler a2A1920-51gv PRO GigE
/41_camera/pylon_camera_node/ image_raw/compressed	sensor_msgs/CompressedImage	Compressed Bayer Image by Basler a2A1920-51gv PRO GigE
/46_camera/pylon_camera_node/ image_raw/compressed	sensor_msgs/CompressedImage	Compressed Bayer Image by Basler a2A1920-51gv PRO GigE
/47_camera/pylon_camera_node/ image_raw/compressed	sensor_msgs/CompressedImage	Compressed Bayer Image by Basler a2A1920-51gv PRO GigE
/65_camera/pylon_camera_node/ image_raw/compressed	sensor_msgs/CompressedImage	Compressed Bayer Image by Basler a2A1920-51gv PRO GigE
/bottom/velodyne_points	sensor_msgs/PointCloud2	Pointcloud by Velodyne VLP-16
/top/velodyne_points	sensor_msgs/PointCloud2	Pointcloud by Velodyne VLP-16
/vn100/vectornav/IMU	sensor_msgs/Imu	VN-100 IMU
/xsens/filter/position_interpolated	geometry_msgs/Vector3Stamped	Interpolated GNSS data to the timestep of top velodyne
/xsens/filter/positionlla	geometry_msgs/Vector3Stamped	GNSS by MTi-680
/xsens/imu/data	sensor_msgs/Imu	IMU by MTi-680
/xsens/imu_interpolated	sensor_msgs/Imu	Interpolated IMU data to the timestep of top velodyne

License

The SiT dataset is published under the CC BY-NC-ND License 4.0, and all codes are published under the Apache License 2.0.

Acknowledgement

The SiT dataset is contributed by Jongwook Bae, Jungho Kim, Junyong Yun, Changwon Kang, Junho Lee, Jeongseon Choi, Chanhyeok Kim, Jungwook Choi, advised by Jun-Won Choi.

We thank the maintainers of the following projects that enable us to develop SiT Dataset: MMDetection by MMLAB