cedricxie / second_ros

Run SECOND Detector as a ROS Node with KITTI Dataset

Run SECOND as a ROS Node

This project is still work in progress. Feel free to make pull requests and improve the performance together.

It is mostly based on SECOND detector and Voxelnet ROS Implementation. Special thanks to @traveller59, @tigerk0430 and @qianguih!

Instructions

Make sure to modify the paths in the scripts based on your setting accordingly.

Step 1: build package

mkdir catkin_ws && cd catkin_ws
git clone https://github.com/cedricxie/second_ros
catkin_make && source devel/setup.bash

Step 2: create environment for SECOND

Follow installation and dataset preparation instruction at https://github.com/traveller59/second.pytorch. Miniconda is used to configure the dependencies. Notice that if you have rospy installed in Python 3, you can still use rosrun to launch SECOND as a ROS node.

conda env create -n py36 -f environment.yml
conda activate py36
source add_paths.sh
pip install --user rospkg catkin_pkg

Step 3: launch SECOND as a ROS node

Play a bag from KITTI and see the result!

roslaunch second_ros second_kitt.launch

Video

Check out the SECOND in Rviz at link. The ROS bag is from KITTI raw dataset 2011_09_26_drive_0005.

second_ros's Issues

The ROS node error when inference rosbag data

the second node can launch with no error, But when the rosbag was played, error occured like below.

Ubuntu 16.04 LTS
ROS-kinect
python 3.6.7
pytorch 0.4.1

the rosbag data is convert from raw data in kitti 2011.09.26 0005 by kitti2bag tool.
hopefully for any advice, thanks !

Video is not available

Hi,

The video link shared at readme doesn't show properly. Could you please reupload?

I have a question

Can I take a bag file from another sensor as a subscriber and process it? Or is it only possible to run kitti raw data files?

Update second_ros to use newest version of second.pytorch?

Hi @cedricxie

thx for providing this cool node-functionality for second.pytorch.

I was wondering about the steps that would be necessary to adapt your repo to work with the newest version of second.pytorch. Could you provide some info on that? Like: Is it possible, what are critical steps, things to look out for?

The reason for this is that it would be really cool if one could use all the other networks that are now available within the framework of second.pytorch (i.e. PointPillars and maybe even TANet, which builds upon the second.pytorch framework).
Cheers

Error during loading pre-trained net

The second node and rosbag launched successfully. But the pre-trained net can't be loaded.

The " ~/.kittiviewerrc " file path can't find

where to get this file "~/.kittiviewerrc "?
the code location is in second_ros.py -- line 167
self.json_setting = Settings(str('/home/aisimba/' + ".kittiviewerrc"))
thanks !

save_model_dir

why "save_model_dir" is not used in second_ros.py but it still in this file

Fix the bad detection visualization issue (even using good model)?

Hi, Awesome work, thanks a lot.
I trained the car model and achieved about the same level accuracy as the SECOND original piece, however, when I run the inference and visualize in ROS, the result seems quite messy. It seems not only the detection is not accurate but also some out of sync issue. The testing data frames are from the rosbag converted using the kitti raw data 2011_09_26_drive_0005.
You can play the rosbag I recorded to see my visualization result in ROS: link
May I know if you experienced the same problem? I'm so grateful if you could give some hint to figure out where possibly goes wrong. Thanks a lot!

Which version of SECOND package is used?

Hi,
so I followed the installation instructions for this repo which leads to instructions of other git repos. Namely second.pytorch and spconv. The installation process was not successful and it took me a while to realize that this repo uses an older version of second.pytorch(I am guessing v1.0) and therefore the installation-instructions that I used were not the correct ones.

@cedricxie could you please provide the versions of these used tools and update the readme.md to make installation easier. thx.

Help！2d IOU error

in https://github.com/cedricxie/second_ros/blob/master/src/second_ros/second/second/core/non_max_suppression/nms_gpu.py
I test the following code

if __name__ == "__main__":
    import numpy as np
    box1 = np.array([[0,0,20,20,-0.1]])
    box2 = np.array([[0,0,20,20,-0.1]])
    print(rotate_iou_gpu(box1, box2))

Obviously it should print 1,but terminal shows

Is this something I'm using or is there something wrong with the code itself?I look forward to receiving your reply

How to detect object for all 360 degree?

Hi,

Current development is only limited to forward FOV range. But how to detect object considering all 360 degree. Kindly provide suggestions if you have implemnted.