JS3C-Net
Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion (AAAI2021)
This repository is for JS3C-Net introduced in the following AAAI-2021 paper [arxiv paper]
Xu Yan, Jiantao Gao, Jie Li, Ruimao Zhang, Zhen Li*, Rui Huang and Shuguang Cui, "Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion".
If you find our work useful in your research, please consider citing:
@InProceedings{yan2021sparse,
title={Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion},
author={Yan, Xu and Gao, Jiantao and Li, Jie and Zhang, Ruimao, and Li, Zhen and Huang, Rui and Cui, Shuguang},
journal={AAAI Conference on Artificial Intelligence ({AAAI})},
year={2021}
}
Getting Started
Set up
Clone the repository:
git clone https://github.com/yanx27/JS3C-Net.git
Installation instructions for Ubuntu 16.04:
- Make sure CUDA and cuDNN are installed. Only this configurations has been tested:
- Python 3.6.9, Pytorch 1.3.1, CUDA 10.1;
- Compile the customized operators by
sh complile.sh
in/lib
. - Install spconv1.0 in
/lib/spconv
. We use the same version with PointGroup, you can install it according to the instruction. Higher version spconv may cause issues.
Data Preparation
- SemanticKITTI and SemanticPOSS datasets can be found in semantickitti-page and semanticposs-page.
- Download the files related to semantic segmentation and extract everything into the same folder.
- Use voxelizer generate ground truths of semantic scene completion, where following parameters are used. We provide pre-processed SemanticPOSS SSC labels here.
min range: 2.5
max range: 70
future scans: 70
min extent: [0, -25.6, -2]
max extent: [51.2, 25.6, 4.4]
voxel size: 0.2
- Finally, the dataset folder should be organized as follows.
SemanticKITTI(POSS)
├── dataset
│ ├── sequences
│ │ ├── 00
│ │ │ ├── labels
│ │ │ ├── velodyne
│ │ │ ├── voxels
│ │ │ ├── [OTHER FILES OR FOLDERS]
│ │ ├── 01
│ │ ├── ... ...
- Note that the data for official SemanticKITTI SSC benchmark only contains 1/5 of the whole sequence and they provide all extracted SSC data for the training set here.
SemanticKITTI
Training
Run the following command to start the training. Output (logs) will be redirected to ./logs/JS3C-Net-kitti/
. You can ignore this step if you want to use our pretrained model in ./logs/JS3C-Net-kitti/
.
$ python train.py --gpu 0 --log_dir JS3C-Net-kitti --config opt/JS3C_default_kitti.yaml
Evaluation Semantic Segmentation
Run the following command to evaluate model on evaluation or test dataset
$ python test_kitti_segment.py --log_dir JS3C-Net-kitti --gpu 0 --dataset [val/test]
Evaluation Semantic Scene Completion
Run the following command to evaluate model on evaluation or test dataset
$ python test_kitti_ssc.py --log_dir JS3C-Net-kitti --gpu 0 --dataset [val/test]
SemanticPOSS
Results on SemanticPOSS can be easily obtained by
$ python train.py --gpu 0 --log_dir JS3C-Net-POSS --config opt/JS3C_default_POSS.yaml
$ python test_poss_segment.py --gpu 0 --log_dir JS3C-Net-POSS
Pretrained Model
We trained our model on a single Nvidia Tesla V100 GPU with batch size 6. If you want to train on the TITAN GPU, you can choose batch size as 2. Please modify dataset_dir
in args.txt
to your path.
Model | #Param | Segmentation | Completion | Checkpoint |
---|---|---|---|---|
JS3C-Net | 2.69M | 66.0 | 56.6 | 18.5MB |
Results on SemanticKITTI Benchmark
Quantitative results on SemanticKITTI Benchmark at the submisison time.
Acknowledgement
This project is not possible without multiple great opensourced codebases.
License
This repository is released under MIT License (see LICENSE file for details).