eliphatfs / skeletonmerger

Code repository for paper `Skeleton Merger: an Unsupervised Aligned Keypoint Detector`.

Skeleton Merger

Skeleton Merger, an Unsupervised Aligned Keypoint Detector. The paper is available at https://arxiv.org/abs/2103.10814.

Update Aug. 6th: The point cloud visualizer is now released! See https://github.com/eliphatfs/PointCloudVisualizer.

A map of the repository

• The merger/pointnetpp folder contains the Pytorch Implementation of PointNet and PointNet++ repository with some minor changes. It is adapted to make compatible relative imports.
• The merger/composed_chamfer.py file contains an efficient implementation of proposed Composite Chamfer Distance (CCD).
• The merger/data_flower.py file is for data loading and preprocessing.
• The merger/merger_net.py file contains the Skeleton Merger implementation.
• The root folder contains several scripts for training and testing.

You can find a pre-trained model on chairs from ShapeNetCore here. Notice that axis order (e.g., gravity axis may be either y or z) and scaling may vary between datasets, so it is recommended to train a model locally from scratch if you need to use Skeleton Merger. It's fast! Skeleton Merger usually gives reasonable results within 5-10 epochs, which only takes minutes on ShapeNetCore-scale datasets with a GTX 1080. (For full power of the model you still need to train for 50-100 epochs and do some epoch selection by validation error or by the downstream task.)

Usage of script files

Usage of the script files, together with a brief description of data format, are available through the -h command line option.

Dataset

The ShapeNetCore.v2 dataset used in the paper is available from the Point Cloud Datasets repository.

skeletonmerger's Issues

GPU需求和训练时间

我想了解一下训练这个对准关键点检测器一般需要显存多少的GPU以及需要训练多长时间？

The subclass id of Airplanes and Guitars ?

Hi, nice job!

I have noticed that you changed the subclass id of chair category(0 to 14). So , would you please tell me the subclass id of Airplanes and Guitars?

Thank you very much!

The results

How to train on Kitti

Please help on how to train this on kitti

The error of predictor_keypointnet.py

Hi, nice job

I am very sorry to bother you, when I ran the predictor_keypointnet.py on the guitar category, I met the following error:
99%|█████████▉| 696/704 [03:42<00:02, 3.12it/s]
Traceback (most recent call last):
recon, key_points, kpa, emb, null_activation = net(torch.Tensor(np.array(Q)).to(ns.device))
x = batch_norm(x) merger_net.py", line 35, in forward
ValueError: expected 2D or 3D input (got 1D input)

Thank you very much!

annotation-json and pcd files for predictor_keypoint.py

Where can I get the annotation json and pcd file to predict keypoint?
I only have the shapenetcorev2 dataset ,only .h5 files and some json files.
Thanks!

No such file or directory: '../keypointnet/annotations/chair.json' ，‘../keypointnet/pcd’

How to use it in shapenetcorev2 data set? How to modify predictor_keypointnet.py and eval_keypointnet.py?
I'm stuck here, please help TAT

The error about datasets

Sorry to bother you , i ues shapenetcorev2 dataset, but i don't know how can i divide the dataset just like you? can you help me ?

Want a pretrained model

I trained your model locally with default settings, however, the result is not as good as shown in the paper. I write this issue to ask for your help. Could you please provide me with your training settings( epoch, batch num, optimizer). Also, I would be very glad to get a pretrained model from you. Best wishes! 😄

Error while training the model using ShapeNetCore.v2

Hello,

I downloaded the "ShapeNetCore.v2" from https://github.com/AnTao97/PointCloudDatasets, as mentioned in the readme.
I try to train the model using following configurations:
python train.py -t '/home/username/datasets/shapenetcorev2_hdf5_2048' -v '/home/username/datasets/shapenetcorev2_hdf5_2048/val' -c 1 -m airplane-merger.pt -d gpu

And, I get the following error messages:
Traceback (most recent call last):
File "/home/projects/SkeletonMerger/train.py", line 79, in
x, xl = all_h5(DATASET, True, True, subclasses=(ns.subclass,), sample=None) # n x 2048 x 3
File "/home/projects/SkeletonMerger/merger/data_flower.py", line 36, in all_h5
xy = tuple(lazy)
File "/home/projects/SkeletonMerger/merger/data_flower.py", line 33, in
lazy = map(lambda x: load_h5(x, normalize, include_label),
File "/home/projects/SkeletonMerger/merger/data_flower.py", line 14, in load_h5
f = h5py.File(h5_filename, 'r')
File "/home/anaconda3/envs/pytorchworkshop/lib/python3.9/site-packages/h5py/_hl/files.py", line 455, in init
fid = make_fid(name, mode, userblock_size,
File "/home/anaconda3/envs/pytorchworkshop/lib/python3.9/site-packages/h5py/_hl/files.py", line 199, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 100, in h5py.h5f.open
OSError: Unable to open file (file signature not found)

It seems there is some issue in reading the h5 file. Please let me know.