This repository contains the PyTorch implementation for PoinTr, which was adapted for a study on point cloud completion of terrestrial laserscanning data of individual trees.
The folder tree_workflow contains jupyter notebooks for:
- Creating training samples from individual tree point clouds (make_samples_for_treePoinTr.ipynb)
- Applying the completion approach to individual trees (whole_tree_completion_workflow.ipynb)
The data used in the study and the models trained in the study can be downloaded from Zenodo: doi.org/10.5281/zenodo.13303159
For installation and usage, please follow the original instructions below, provided by the authors of PoinTr.
Created by Xumin Yu*, Yongming Rao*, Ziyi Wang, Zuyan Liu, Jiwen Lu, Jie Zhou
[arXiv] [Video] [Dataset] [Models] [supp]
This repository contains PyTorch implementation for PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers (ICCV 2021 Oral Presentation).
PoinTr is a transformer-based model for point cloud completion. By representing the point cloud as a set of unordered groups of points with position embeddings, we convert the point cloud to a sequence of point proxies and employ a transformer encoder-decoder architecture for generation. We also propose two more challenging benchmarks ShapeNet-55/34 with more diverse incomplete point clouds that can better reflect the real-world scenarios to promote future research.
- 2023-9-2 AdaPoinTr accepted by T-PAMI, Projected-ShapeNet dataset see here
- 2023-1-11 Release AdaPoinTr (PoinTr + Adaptive Denoising Queries), achieving SOTA performance on various benchmarks. Arxiv.
- 2022-06-01 Implement SnowFlakeNet.
- 2021-10-07 Our solution based on PoinTr wins the Championship on MVP Completion Challenge (ICCV Workshop 2021). The code will come soon.
- 2021-09-09 Fix a bug in
datasets/PCNDataset.py
(#27), and update the performance of PoinTr on PCN benchmark (CD from 8.38 to 7.26).
We provide pretrained PoinTr models:
dataset | url | performance |
---|---|---|
ShapeNet-55 | [Tsinghua Cloud] / [Google Drive] / [BaiDuYun] (code:erdh) | CD = 1.09e-3 |
ShapeNet-34 | [Tsinghua Cloud] / [Google Drive] / [BaiDuYun] (code:atbb ) | CD = 2.05e-3 |
PCN | [Tsinghua Cloud] / [Google Drive] / [BaiDuYun] (code:9g79) | CD = 8.38e-3 |
PCN_new | [Tsinghua Cloud] / [Google Drive] / [BaiDuYun] (code:aru3 ) | CD = 7.26e-3 |
KITTI | [Tsinghua Cloud] / [Google Drive] / [BaiDuYun] (code:99om) | MMD = 5.04e-4 |
We provide pretrained AdaPoinTr models (coming soon):
dataset | url | performance |
---|---|---|
ShapeNet-55 | Tsinghua Cloud / Google Drive / BaiDuYun | CD = 0.81e-3 |
ShapeNet-34 | Tsinghua Cloud / Google Drive / BaiDuYun | CD = 1.23e-3 |
Projected_ShapeNet-55 | Tsinghua Cloud / Google Drive / [BaiDuYun](code:dycc) | CD = 9.58e-3 |
Projected_ShapeNet-34 | Tsinghua Cloud / Google Drive / [BaiDuYun](code:dycc) | CD = 9.12e-3 |
PCN | [Tsinghua Cloud] / [Google Drive] / [BaiDuYun](code:rc7p) | CD = 6.53e-3 |
- PyTorch >= 1.7.0
- python >= 3.7
- CUDA >= 9.0
- GCC >= 4.9
- torchvision
- timm
- open3d
- tensorboardX
pip install -r requirements.txt
NOTE: PyTorch >= 1.7 and GCC >= 4.9 are required.
# Chamfer Distance
bash install.sh
The solution for a common bug in chamfer distance installation can be found in Issue #6
# PointNet++
pip install "git+https://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops&subdirectory=pointnet2_ops_lib"
# GPU kNN
pip install --upgrade https://github.com/unlimblue/KNN_CUDA/releases/download/0.2/KNN_CUDA-0.2-py3-none-any.whl
Note: If you still get ModuleNotFoundError: No module named 'gridding'
or something similar then run these steps
1. cd into extensions/Module (eg extensions/gridding)
2. run `python setup.py install`
That will fix the ModuleNotFoundError
.
The details of our new ShapeNet-55/34 datasets and other existing datasets can be found in DATASET.md.
To inference sample(s) with pretrained model
python tools/inference.py \
${POINTR_CONFIG_FILE} ${POINTR_CHECKPOINT_FILE} \
[--pc_root <path> or --pc <file>] \
[--save_vis_img] \
[--out_pc_root <dir>] \
For example, inference all samples under demo/
and save the results under inference_result/
python tools/inference.py \
cfgs/PCN_models/AdaPoinTr.yaml ckpts/AdaPoinTr_PCN.pth \
--pc_root demo/ \
--save_vis_img \
--out_pc_root inference_result/ \
To evaluate a pre-trained PoinTr model on the Three Dataset with single GPU, run:
bash ./scripts/test.sh <GPU_IDS> \
--ckpts <path> \
--config <config> \
--exp_name <name> \
[--mode <easy/median/hard>]
Test the PoinTr (AdaPoinTr) pretrained model on the PCN benchmark or Projected_ShapeNet:
bash ./scripts/test.sh 0 \
--ckpts ./pretrained/PoinTr_PCN.pth \
--config ./cfgs/PCN_models/PoinTr.yaml \
--exp_name example
bash ./scripts/test.sh 0 \
--ckpts ./pretrained/PoinTr_ps55.pth \
--config ./cfgs/Projected_ShapeNet55_models/AdaPoinTr.yaml \
--exp_name example
Test the PoinTr pretrained model on ShapeNet55 benchmark (easy mode):
bash ./scripts/test.sh 0 \
--ckpts ./pretrained/PoinTr_ShapeNet55.pth \
--config ./cfgs/ShapeNet55_models/PoinTr.yaml \
--mode easy \
--exp_name example
Test the PoinTr pretrained model on the KITTI benchmark:
bash ./scripts/test.sh 0 \
--ckpts ./pretrained/PoinTr_KITTI.pth \
--config ./cfgs/KITTI_models/PoinTr.yaml \
--exp_name example
CUDA_VISIBLE_DEVICES=0 python KITTI_metric.py \
--vis <visualization_path>
To train a point cloud completion model from scratch, run:
# Use DistributedDataParallel (DDP)
bash ./scripts/dist_train.sh <NUM_GPU> <port> \
--config <config> \
--exp_name <name> \
[--resume] \
[--start_ckpts <path>] \
[--val_freq <int>]
# or just use DataParallel (DP)
bash ./scripts/train.sh <GPUIDS> \
--config <config> \
--exp_name <name> \
[--resume] \
[--start_ckpts <path>] \
[--val_freq <int>]
Train a PoinTr model on PCN benchmark with 2 gpus:
CUDA_VISIBLE_DEVICES=0,1 bash ./scripts/dist_train.sh 2 13232 \
--config ./cfgs/PCN_models/PoinTr.yaml \
--exp_name example
Resume a checkpoint:
CUDA_VISIBLE_DEVICES=0,1 bash ./scripts/dist_train.sh 2 13232 \
--config ./cfgs/PCN_models/PoinTr.yaml \
--exp_name example --resume
Finetune a PoinTr on PCNCars
CUDA_VISIBLE_DEVICES=0,1 bash ./scripts/dist_train.sh 2 13232 \
--config ./cfgs/KITTI_models/PoinTr.yaml \
--exp_name example \
--start_ckpts ./weight.pth
Train a PoinTr model with a single GPU:
bash ./scripts/train.sh 0 \
--config ./cfgs/KITTI_models/PoinTr.yaml \
--exp_name example
We also provide the Pytorch implementation of several baseline models including GRNet, PCN, TopNet and FoldingNet. For example, to train a GRNet model on ShapeNet-55, run:
CUDA_VISIBLE_DEVICES=0,1 bash ./scripts/dist_train.sh 2 13232 \
--config ./cfgs/ShapeNet55_models/GRNet.yaml \
--exp_name example
MIT License
Our code is inspired by GRNet and mmdetection3d.
If you find our work useful in your research, please consider citing:
@inproceedings{yu2021pointr,
title={PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers},
author={Yu, Xumin and Rao, Yongming and Wang, Ziyi and Liu, Zuyan and Lu, Jiwen and Zhou, Jie},
booktitle={ICCV},
year={2021}
}