yukunxia / superpoint-stereo-visual-odometry

Deployment and evaluation code for a SuperPoint-based Stereo Visual Odometry

SuperPoint Stereo Visual Odometry

This repository stores the evaluation and deployment code of our On-Device Machine Learning course project. Team members are Yukun Xia, and Yuqing Qin. The goal of this project is to explore the relationships between the performance and latency/energy efficiency of different visual odometry front ends, and benchmark them on a Jetson Nano. We retrained SuperPoint models with different backbones, as well as changed the configurations, eg. input resolution and quantization. We tested 78 configurations on KITTI dataset, and our temporary conclusion is that, for an edge device with a basic DL accelerator, the performance, latency and energy efficiency of a SuperPoint model can be competitive against classic methods, eg. ORB and SIFT. But still, we need more data and more rigorous tests.

For more details, please read our project report pdf under the root folder.

Dependencies

• ROS
• OpenCV 4.5.4
• Ceres
• Eigen3 3.3
• CUDA
• TensorRT 8.2

System Diagram

Usage

Basic

• Download rosbags or create rosbags from KITTI dataset via kitti2bag
• catkin_make
• source devel/setup.bash or source devel/setup.zsh
• rosrun odml_visual_odometry engine_generation.py
• roslaunch odml_data_processing main.launch
• rostopic pub /kitti_loader_action_server/goal odml_data_processing/kitti_data_loaderActionGoal "header: seq: 0 stamp: secs: 0 nsecs: 0 frame_id: '' goal_id: stamp: secs: 0 nsecs: 0 id: '' goal: kitti_eval_id: 4 description: ''"

Evaluate over all configurations

please checkout the long_term_eval branch

superpoint-stereo-visual-odometry's Issues

can realize superglue for matching keypoints?

hi,professor:
i extract your superpoint code, then i run on jetson tx2, next step matching, can you give an examples use superglue with tensorRT? please help!

I want to learn from you about model lightweight

Your work is so great. Thank you for your work. Will your model conversion code be open source? I just came into contact with lightweight and want to learn from you.