yochengliu / relation-shape-cnn

Relation-Shape Convolutional Neural Network for Point Cloud Analysis (CVPR 2019 Oral & Best paper finalist)

Relation-Shape Convolutional Neural Network for Point Cloud Analysis

This repository contains the author's implementation in Pytorch for the paper:

Relation-Shape Convolutional Neural Network for Point Cloud Analysis [arXiv] [CVF]
Yongcheng Liu, Bin Fan, Shiming Xiang and Chunhong Pan
CVPR 2019 Oral & Best paper finalist     Project Page: https://yochengliu.github.io/Relation-Shape-CNN/

Citation

If our paper is helpful for your research, please consider citing:

        @inproceedings{liu2019rscnn,   
            author = {Yongcheng Liu and    
                            Bin Fan and    
                      Shiming Xiang and   
                           Chunhong Pan},   
            title = {Relation-Shape Convolutional Neural Network for Point Cloud Analysis},   
            booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},    
            pages = {8895--8904},  
            year = {2019}   
        }   

Usage: Preparation

Requirement

• Ubuntu 14.04
• Python 3 (recommend Anaconda3)
• Pytorch 0.3.*/0.4.*
• CMake > 2.8
• CUDA 8.0 + cuDNN 5.1

Building Kernel

git clone https://github.com/Yochengliu/Relation-Shape-CNN.git 
cd Relation-Shape-CNN

• mkdir build && cd build
• cmake .. && make

Dataset

Shape Classification

Download and unzip ModelNet40 (415M). Replace $data_root$ in cfgs/config_*_cls.yaml with the dataset parent path.

ShapeNet Part Segmentation

Download and unzip ShapeNet Part (674M). Replace $data_root$ in cfgs/config_*_partseg.yaml with the dataset path.

Usage: Training

Shape Classification

sh train_cls.sh

You can modify relation_prior in cfgs/config_*_cls.yaml. We have trained a Single-Scale-Neighborhood classification model in cls folder, whose accuracy is 92.38%.

Shape Part Segmentation

sh train_partseg.sh

We have trained a Multi-Scale-Neighborhood part segmentation model in seg folder, whose class mIoU and instance mIoU is 84.18% and 85.81% respectively.

Usage: Evaluation

Shape Classification

Voting script: voting_evaluate_cls.py

You can use our model cls/model_cls_ssn_iter_16218_acc_0.923825.pth as the checkpoint in config_ssn_cls.yaml, and after this voting you will get an accuracy of 92.71% if all things go right.

Shape Part Segmentation

Voting script: voting_evaluate_partseg.py

You can use our model seg/model_seg_msn_iter_57585_ins_0.858054_cls_0.841787.pth as the checkpoint in config_msn_partseg.yaml.

License

The code is released under MIT License (see LICENSE file for details).

Acknowledgement

The code is heavily borrowed from Pointnet2_PyTorch.

Contact

If you have some ideas or questions about our research to share with us, please contact [email protected]

relation-shape-cnn's Issues

ImportError: torch.utils.ffi is deprecated. Please use cpp extensions instead.

Hi,

Thanks for sharing your code. Really useful.

I am getting "ImportError: torch.utils.ffi is deprecated. Please use cpp extensions instead." while importing from models import RSCNN_SSN_Cls as RSCNN_SSN

Please help

Please point out in code where you are doing "random picking in the ball"

Great work BTW!!!
I see in the paper you mention about "random picking in the ball"
I know the ball query code is in ball_query_gpu.cu file

Please point out the implementation that does "random" picking in the ball.

Dose RSConvLayer build with shared weight ?

In paper, "To capture more sufficient geometric relation, we force RSCNN to learn over three-scale neighborhoods centered on
a sampled point with a shared weight. This is different from multi-scale grouping (MSG) [28] that learns multiscale features using multiple groups of weight."
But in release code and parameters, I found the RSConv.weight was not shared in different scale pointclouds.

how to apply this structure to semantic segmentation

hi, thanks for your release code and i learned a lot from it. now i have some question,that how can i use this RS-CNN to sovle the problem of semantic segmentation?
can the part-seg code also use in semantic seg ?
im sorry to bother you.thanks

请问prior relation在哪个代码文件中进行计算？

我看了各个文件的代码，但没有找到 10维的数据输入的计算过程，在trian文件中，只是先读了数据，然后将数据放入model中，可是初始数据只有3维，生成10维输入的相应代码在哪个文件中呢？期待您的回复！

Confused about the rotation-invariant property and network design

Thank you for your excellent work. It is very interesting.

After reading your paper, I have a few questions about your network design, which makes me feel quite confused:

(1) On page 5, the first line of left column, "they are normalized to take the centroid as the origin." So do you mean you just ignore all the absolute coordinates of points? If so, this would transform each point subset into local patch. However, this would lead to inferior performance, as pointed out by the paper "Dynamic Graph CNN for Learning on Point Clouds" (page 5, the third paragraph on the right column): "Note that such a choice encodes only local information, essentially treating the shape as a collection of small patches and losing the global shape structure." In addition, I observe that when using only Euclidean distance as the relation h (so there is no absolute coordinate information utilized in the network), the classification performance of RS-CNN could still achieve 92.5, which is quite impressive. I wonder if this is contradictory to the statement made by "Dynamic Graph CNN for Learning on Point Clouds"?

(2) I observe that RS-CNN has one important property - rotation invariance. But if the absolute coordinates of points are input to the network as initial features, I think RS-CNN cannot achieve intrinsic rotation invariance. Am I missing some important information?

Thanks a lot.

windows下make出错

mingw32-make时出现：
Scanning dependencies of target pointnet2_ext
[ 20%] Building NVCC (Device) object CMakeFiles/cuda_compile.dir/utils/csrc/./cuda_compile_generated_ball_query_gpu.cu.obj
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
nvcc fatal : Host compiler targets unsupported OS.
CMake Error at cuda_compile_generated_ball_query_gpu.cu.obj.cmake:206 (message):
Error generating
F:/RS-CNN/code/Relation-Shape-CNN/build/CMakeFiles/cuda_compile.dir/utils/csrc/./cuda_compile_generated_ball_query_gpu.cu.obj

CMakeFiles\pointnet2_ext.dir\build.make:64: recipe for target 'CMakeFiles/cuda_compile.dir/utils/csrc/./cuda_compile_generated_ball_query_gpu.cu.obj' failed
mingw32-make[2]: *** [CMakeFiles/cuda_compile.dir/utils/csrc/./cuda_compile_generated_ball_query_gpu.cu.obj] Error 1
CMakeFiles\Makefile2:59: recipe for target 'CMakeFiles/pointnet2_ext.dir/all' failed
mingw32-make[1]: *** [CMakeFiles/pointnet2_ext.dir/all] Error 2
Makefile:75: recipe for target 'all' failed
mingw32-make: *** [all] Error 2

vs2013的nmake也会出错

can not compile the extension

Got :
TypeError: dist must be a Distribution instance

Will this also work for Paris Lille Dataset?

Hi,
This is an amazing research on Point Cloud Segmentation. I wish to know if it will also work for Paris Lille Dataset or Semantic3d Dataset too?

When my training environment is Ubuntu16.04+pytorch0.3.1，I am in trouble

[epoch 1: 0/499] train loss: 5.119819 lr: 0.00100
THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1518241554738/work/torch/lib/THC/generic/THCStorage.cu line=58 error=2 : out of memory
Traceback (most recent call last):
File "train_partseg.py", line 214, in
main()
File "train_partseg.py", line 95, in main
train(train_dataloader, test_dataloader, model, criterion, optimizer, lr_scheduler, bnm_scheduler, args, num_batch)
File "train_partseg.py", line 128, in train
loss.backward()
File "/home/liangpan/anaconda3/envs/torch0.3/lib/python3.5/site-packages/torch/autograd/variable.py", line 167, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, retain_variables)
File "/home/liangpan/anaconda3/envs/torch0.3/lib/python3.5/site-packages/torch/autograd/init.py", line 99, in backward
variables, grad_variables, retain_graph)
RuntimeError: cuda runtime error (2) : out of memory at /opt/conda/conda-bld/pytorch_1518241554738/work/torch/lib/THC/generic/THCStorage.cu:58

Requirement satisified but make error occurs

Relation-Shape-CNN/build$ make
[ 20%] Building NVCC (Device) object CMakeFiles/cuda_compile.dir/utils/csrc/cuda_compile_generated_ball_query_gpu.cu.o
/usr/include/string.h: In function ‘void* __mempcpy_inline(void*, const void*, size_t)’:
/usr/include/string.h:652:42: error: ‘memcpy’ was not declared in this scope
return (char *) memcpy (__dest, __src, __n) + __n;
^
CMake Error at cuda_compile_generated_ball_query_gpu.cu.o.cmake:266 (message):
Error generating file
/home/bowen/Work/Git_Project/Relation-Shape-CNN/build/CMakeFiles/cuda_compile.dir/utils/csrc/./cuda_compile_generated_ball_query_gpu.cu.o
CMakeFiles/pointnet2_ext.dir/build.make:196: recipe for target 'CMakeFiles/cuda_compile.dir/utils/csrc/cuda_compile_generated_ball_query_gpu.cu.o' failed
make[2]: *** [CMakeFiles/cuda_compile.dir/utils/csrc/cuda_compile_generated_ball_query_gpu.cu.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/pointnet2_ext.dir/all' failed
make[1]: *** [CMakeFiles/pointnet2_ext.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

and compiles Pointnet2Pytorch project occuring similar problem.

When I was training, I encountered the following problem（ubuntu16.04+pytorch0.4.1）

[epoch 1: 0/499] train loss: 4.879579 lr: 0.00100
[epoch 1: 20/499] train loss: 1.237031 lr: 0.00100
[epoch 1: 40/499] train loss: 0.955243 lr: 0.00100
[epoch 1: 60/499] train loss: 0.735887 lr: 0.00100
[epoch 1: 80/499] train loss: 0.685161 lr: 0.00100
[epoch 1: 100/499] train loss: 0.482436 lr: 0.00100
[epoch 1: 120/499] train loss: 0.854511 lr: 0.00100
[epoch 1: 140/499] train loss: 0.695081 lr: 0.00100
[epoch 1: 160/499] train loss: 0.860770 lr: 0.00100
[epoch 1: 180/499] train loss: 0.354899 lr: 0.00100
[epoch 1: 200/499] train loss: 0.505713 lr: 0.00100
[epoch 1: 220/499] train loss: 0.378871 lr: 0.00100
[epoch 1: 240/499] train loss: 0.411512 lr: 0.00100
[epoch 1: 260/499] train loss: 0.565703 lr: 0.00100
[epoch 1: 280/499] train loss: 0.386525 lr: 0.00100
[epoch 1: 300/499] train loss: 0.400440 lr: 0.00100
[epoch 1: 320/499] train loss: 0.352148 lr: 0.00100
[epoch 1: 340/499] train loss: 0.571778 lr: 0.00100
train_partseg.py:155: UserWarning: volatile was removed and now has no effect. Use with torch.no_grad(): instead.
points, target = Variable(points, volatile=True), Variable(target, volatile=True)
Traceback (most recent call last):
File "train_partseg.py", line 214, in
main()
File "train_partseg.py", line 95, in main
train(train_dataloader, test_dataloader, model, criterion, optimizer, lr_scheduler, bnm_scheduler, args, num_batch)
File "train_partseg.py", line 138, in train
validate(test_dataloader, model, criterion, args, batch_count)
File "train_partseg.py", line 172, in validate
cat = seg_label_to_cat[target[b, 0]]
KeyError: tensor(5)

cuda10 is not compatible with torch below 1.0

Dear author:
I want to ask that if your code run at cuda10?
If can't, should I change your code to torch1.0,or change my cuda to lower version?

Thank your for you work! Looking forward to your reply！

Do you have results on S3DIS?

Do you have results on S3DIS?

A question about the version of pytorch

Can this code support higher versions of pytorch? I was unable to successfully configure the required pytorch environment.

How about the visualization?

Dear author, thank you very much for sharing the classification code.

I am wondering about the visualization about the results of the segmentation experiments, Could you please share the code of the visualization?

Waiting for your reply~

When will you release the code of project?

Thank you for your work on point set learning, I hope you can share your code. Thank you very much!

CUDA/CuDNN related errors occur in Titan-RTX environments

hello.

I changed my environment in many ways,
but I couldn't get a solution for running your code...

First, my GPU is Titan-RTX
and my attempts are follows.

I also tried to run the code on CUDA 8.0 environments before, but the errors occurs as
almost same as on CUDA 9.0 environments

---

1. ---environment---
   ubuntu 18.04
   CUDA 9.0
   CuDNN 7.1
   torch 0.3.1 / 0.4.0
   ==>
   error message :
   Found GPU0 TITAN RTX which requires CUDA_VERSION >= 9000 for
   optimal performance and fast startup time, but your PyTorch was compiled
   with CUDA_VERSION 8000. Please install the correct PyTorch binary
   using instructions from http://pytorch.org

warnings.warn(incorrect_binary_warn % (d, name, 9000, CUDA_VERSION))

and process is "Killed" when data are load to the gpu, specifically operating conv2d() command in
55 line of pointnet2_modules.py, self.mlp[i] - _PointnetSAModuleBase function

2. ---environment---
   ubuntu 18.04
   CUDA 9.0
   CuDNN 7.1
   torch 0.3.1 / 0.4.1
   ==>
   error message :
   RuntimeError: cuda runtime error (11) : invalid argument at /pytorch/aten/src/THC/THCGeneral.cpp:663

3. ---environment---
   ubuntu 18.04
   CUDA 9.0
   CuDNN 7.1
   torch 0.3.1 / 0.4.1

and I additionally revised train_cls.py as

torch.backends.cudnn.benchmark = False

==>
Traceback (most recent call last):
File "train_cls.py", line 217, in
main()
File "train_cls.py", line 125, in main
train(train_dataloader, test_dataloader, model, criterion, optimizer, lr_scheduler, bnm_scheduler, args, num_batch)
File "train_cls.py", line 167, in train
pred = model(points)
File "/home/mvpserverone/.conda/envs/rscnn/lib/python3.5/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/SSD1/dogyoon/Relation-Shape-CNN-master/models/rscnn_ssn_cls.py", line 102, in forward
return self.FC_layer(features.squeeze(-1))
File "/home/mvpserverone/.conda/envs/rscnn/lib/python3.5/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/mvpserverone/.conda/envs/rscnn/lib/python3.5/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/home/mvpserverone/.conda/envs/rscnn/lib/python3.5/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/mvpserverone/.conda/envs/rscnn/lib/python3.5/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/home/mvpserverone/.conda/envs/rscnn/lib/python3.5/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/mvpserverone/.conda/envs/rscnn/lib/python3.5/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/home/mvpserverone/.conda/envs/rscnn/lib/python3.5/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/mvpserverone/.conda/envs/rscnn/lib/python3.5/site-packages/torch/nn/modules/batchnorm.py", line 66, in forward
exponential_average_factor, self.eps)
File "/home/mvpserverone/.conda/envs/rscnn/lib/python3.5/site-packages/torch/nn/functional.py", line 1251, in batch_norm
raise ValueError('Expected more than 1 value per channel when training, got input size {}'.format(size))
ValueError: Expected more than 1 value per channel when training, got input size [1, 512]

---

I really hope to find the solution of this problem as soon as possible
thank you very much

Issue in displaying Accuracy Result in voting_evaluate_cls.py

Hi,
I ran the voting_evaluate_cls.py and surprisingly got following result
Repeat 1 Acc: 0.000000
Repeat 2 Acc: 0.000000
Repeat 3 Acc: 0.000000

and I found the issue in logging line 94 in voting_evaluate_cls.py
please change it to :
acc = (float((preds == labels).sum()) / labels.numel())*100

forward function in PointNetSAModuleMSG

Hi,
Great work!
Your approach to 3d shape classification problem is quite unique.
I was trying to understand your code I am confused as there is no forward function in PointNetSAModuleMSG. So when in rscnn_cls the PointNetSA MSG module is called with data where does it actually point in pointnet2_modules.py and how does it furthers passes data to rsconv function in pointnet2_utils.
Thanks!!

multi-scale classification code

Dear author, Could you release your multi-scale classification code? Thank you。

About the voting accuracy?

I have got the best validation accuracy with 92.1777%. But when I test the model with voting with random scaling as paper, the accuracy drops to 91.1%.

Could you share your voting code? @Yochengliu

Thanks.

'num_points' argument not really work when loading dataset

Hi, thank you for your nice work!

However, I think there is a tiny bug in the file "ModelNet40Loader.py". In the paper you mentioned for the dataset ModelNet40, 1024 points are randomly sampled from each sample. But in the codes, the argument 'num_points' has not really been used to sample points. So actually all the total 2048 points are used in the main loop for training.

Relation-Shape-CNN/data/ModelNet40Loader.py

Lines 49 to 60 in 6464eb8

	def __getitem__(self, idx):
	pt_idxs = np.arange(0, self.points.shape[1]) # 2048
	if self.train:
	np.random.shuffle(pt_idxs)
	current_points = self.points[idx, pt_idxs].copy()
	label = torch.from_numpy(self.labels[idx]).type(torch.LongTensor)
	if self.transforms is not None:
	current_points = self.transforms(current_points)
	return current_points, label

I guess a little fix will work, e.g.

current_points = current_points[:self.num_points]

Thank you :)

Robustness to point permutation and rigid transformation

In 'Robustness to point permutation and rigid transformation' experiment, the article said 'we address this issue by normalizing each sampled point subset to corresponding local coordinate system, which is determined by each sampled point and its normal'. But can't we use only normal vector to determine a local coordinate system? Or would you demonstrate the details of this process?

Bug coming from pointnet2utils

Dear @Yochengliu ,

Do you have any idea where it is coming from ?

THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=663 error=11 : invalid argument
Traceback (most recent call last):
File "train_partseg.py", line 207, in
main()
File "train_partseg.py", line 92, in main
train(train_dataloader, test_dataloader, model, criterion, optimizer, lr_scheduler, bnm_scheduler, args, num_batch)
File "train_partseg.py", line 121, in train
pred = model(points, batch_one_hot_cls)
File "/home/thomas/.cache/pypoetry/virtualenvs/superpoint-graph-job-py3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/thomas/HELIX/research/Relation-Shape-CNN/models/rscnn_msn_seg.py", line 142, in forward
li_xyz, li_features = self.SA_modules[i](l_xyz[i], l_features[i])
File "/home/thomas/.cache/pypoetry/virtualenvs/superpoint-graph-job-py3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/thomas/HELIX/research/Relation-Shape-CNN/models/../utils/pointnet2_modules.py", line 51, in forward
new_features
File "/home/thomas/.cache/pypoetry/virtualenvs/superpoint-graph-job-py3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/thomas/.cache/pypoetry/virtualenvs/superpoint-graph-job-py3.6/lib/python3.6/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/home/thomas/.cache/pypoetry/virtualenvs/superpoint-graph-job-py3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/thomas/.cache/pypoetry/virtualenvs/superpoint-graph-job-py3.6/lib/python3.6/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/home/thomas/.cache/pypoetry/virtualenvs/superpoint-graph-job-py3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/thomas/HELIX/research/Relation-Shape-CNN/models/../utils/pytorch_utils/pytorch_utils.py", line 67, in forward
h_xi_xj = self.mapping_func2(self.activation(self.bn_mapping(self.mapping_func1(h_xi_xj))))
File "/home/thomas/.cache/pypoetry/virtualenvs/superpoint-graph-job-py3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/thomas/.cache/pypoetry/virtualenvs/superpoint-graph-job-py3.6/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 301, in forward
self.padding, self.dilation, self.groups)
RuntimeError: cuda runtime error (11) : invalid argument at /pytorch/aten/src/THC/THCGeneral.cpp:663

我在cmake的时候遇到了问题

_pointnet2.c:22:24: fatal error: pyconfig.h: 没有那个文件或目录
compilation terminated.
Traceback (most recent call last):
File "/usr/lib/python3.6/distutils/unixccompiler.py", line 118, in _compile
extra_postargs)
File "/usr/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
spawn(cmd, dry_run=self.dry_run)
File "/usr/lib/python3.6/distutils/spawn.py", line 36, in spawn
_spawn_posix(cmd, search_path, dry_run=dry_run)
File "/usr/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix
% (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/xu/.local/lib/python3.6/site-packages/cffi/ffiplatform.py", line 51, in _build
dist.run_command('build_ext')
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python3.6/distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/usr/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
self._build_extensions_serial()
File "/usr/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
self.build_extension(ext)
File "/usr/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
depends=ext.depends)
File "/usr/lib/python3.6/distutils/ccompiler.py", line 574, in compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/usr/lib/python3.6/distutils/unixccompiler.py", line 120, in _compile
raise CompileError(msg)
distutils.errors.CompileError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/media/xu/BAF5-9BE9/0_Experiment/0_ex/Relation-Shape-CNN-master/utils/build_ffi.py", line 54, in
build(args)
File "/media/xu/BAF5-9BE9/0_Experiment/0_ex/Relation-Shape-CNN-master/utils/build_ffi.py", line 42, in build
ffi.build()
File "/home/xu/.local/lib/python3.6/site-packages/torch/utils/ffi/init.py", line 184, in build
_build_extension(ffi, cffi_wrapper_name, target_dir, verbose)
File "/home/xu/.local/lib/python3.6/site-packages/torch/utils/ffi/init.py", line 108, in _build_extension
outfile = ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname)
File "/home/xu/.local/lib/python3.6/site-packages/cffi/api.py", line 723, in compile
compiler_verbose=verbose, debug=debug, **kwds)
File "/home/xu/.local/lib/python3.6/site-packages/cffi/recompiler.py", line 1527, in recompile
compiler_verbose, debug)
File "/home/xu/.local/lib/python3.6/site-packages/cffi/ffiplatform.py", line 22, in compile
outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
File "/home/xu/.local/lib/python3.6/site-packages/cffi/ffiplatform.py", line 58, in _build
raise VerificationError('%s: %s' % (e.class.name, e))
cffi.VerificationError: CompileError: command 'x86_64-linux-gnu-gcc' failed with exit status 1
CMakeFiles/pointnet2_ext.dir/build.make:67: recipe for target '../utils/_ext/pointnet2/_pointnet2.so' failed
make[2]: *** [../utils/_ext/pointnet2/_pointnet2.so] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/pointnet2_ext.dir/all' failed
make[1]: *** [CMakeFiles/pointnet2_ext.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

使用您的代码报错缺少文件，请问该如何解决

Visualization of segmentation results for Shapenet Part

I used the code you provided to conduct the experiment on the Shapenet Part dataset and could get the MIOU value, but I could not get the visualization result. So I would like to ask you how to do it.
Looking forward to your reply. Thank you

Confused about the symmetric aggregation function

dear author, RS-CNN is excellent! but I am confused about the aggregation function, which you said it's symmetric, such as summation or max pooling, in Session 4.3, Figure 8, in the 1st layer, there are 512 points(1/2 of the input points), but in the 2nd layer, there remain only 128 points(1/4 of the 1st layer), why? the relation-shape convolution(implemented as shared MLP) and channel-raising mapping(also shared MLP) don't seem to change the point number, because of the aggregation function? how?
Thank you! Looking forward to your reply!

torch-utils not found

I am facing this issue. How to solve it?

from models import RSCNN_SSN_Cls as RSCNN_SSN

File "C:\Users\Nigam Katta\OneDrive - Georgia Institute of Technology\Relation-Shape-CNN-master\models_init_.py", line 1, in
from .rscnn_ssn_cls import RSCNN_SSN as RSCNN_SSN_Cls
File "C:\Users\Nigam Katta\OneDrive - Georgia Institute of Technology\Relation-Shape-CNN-master\models\rscnn_ssn_cls.py", line 9, in
from pointnet2_modules import PointnetSAModule, PointnetSAModuleMSG
File "C:\Users\Nigam Katta\OneDrive - Georgia Institute of Technology\Relation-Shape-CNN-master\models../utils\pointnet2_modules.py", line 5, in
import pointnet2_utils
File "C:\Users\Nigam Katta\OneDrive - Georgia Institute of Technology\Relation-Shape-CNN-master\models../utils\pointnet2_utils.py", line 11, in
from ext import pointnet2
File "C:\Users\Nigam Katta\OneDrive - Georgia Institute of Technology\Relation-Shape-CNN-master\models../utils_ext\pointnet2_init.py", line 2, in
from torch.utils.ffi import _wrap_function
ModuleNotFoundError: No module named 'torch.utils.ffi'

Could you release your trained model? I just run your code, but I just get 84% cls accuracy.

No module named '_ext.pointnet2._pointnet2'

Hi @Yochengliu
thank you very much for sharing this code,I found that the pointnet2 folder under the _ext folder is a _init.py file, not a poinnet2 file. Could you please kindly upload the pointnet2 file?

QueryAndGroup function BUG

Hi,
Thanks for your work on point cloud process.
I find something maybe wrong in your code.
In the QueryAndGroup function, if use raw_grouped_xyz=grouped_xyz, the raw_grouped_xyz and grouped_xyz is actually one vaiable sharing the same pointer.
So when changing the grouped_xyz, the raw_grouped_xyz will also change.

Relation-Shape-CNN/utils/pointnet2_utils.py

Line 388 in cc52c48

raw_grouped_xyz = grouped_xyz

The result new_features just contains related coordinates no absolute coordinates.
I don't sure whether it only occur in my code, please check it.

Thanks for your sharing projcet.

dear author,could you share the code of the project？

How to get-93.6% accuracy?

First Thank You for this contribution.
I run your code without any changes, but not getting 93.6% accuracy which is mention in your paper. I am only getting 92.18% accuracy. Guide me How to get 93.6% accuracy.

Visualization of learned feature

could you let me know hot to visualize the learned feature of the network?
for example, the figure 8 of your paper.

thanks

Question about normal estimation result (table 3)

Hi @Yochengliu,

Many thanks for sharing the code. I have a question about the normal estimation result (table 3) in your paper. The table is attached below:

I am not sure how is the error defined? It looks like the mean cosine distance but I think the cosine distance should be -1 (i.e. the cosine similarity == 1) when the predictions are perfect? Did you apply an abs() to the cosine distance and compute the mean of all cosine distances?

Best,
Zirui

Could you release your multi-scale classification code? Or any detailed config

Could you release the test code for segmentation?

i want to test the segmentation model, could you release the test code for multi scale segmentation?

when will you release the code about semantic part segmetation?

Hello,thank you for our work.I want to know when will you release the code about semantic part segmentation?looking for your reply.

Inference time?

Can you give an idea of what length of time is required for inference? Also, do you plan to share the code on GitHub?

Multi-Scale model for Classification

Dear authors,
Thank you so much for releasing your code. I was wondering if you can also release the code for Multi-Scale classification on ModelNet40 as well which was used to report the best result.
Best,
Ankit

PS: There are other similar issues but I am still reposting so that it might get noticed by the authors.

Why accuracy is remaining 0?

Hi, I follow your README to run the code, but the classification accuracy is remaining 0 after several epoch. Is there something important I neglected? Thank you very much!

[epoch   4: 160/307] 	 train loss: 0.928064 	 lr: 0.00100
[epoch   4: 180/307] 	 train loss: 0.486494 	 lr: 0.00100
[epoch   4: 200/307] 	 train loss: 0.455191 	 lr: 0.00100
[epoch   4: 220/307] 	 train loss: 0.509143 	 lr: 0.00100
[epoch   4: 240/307] 	 train loss: 0.903722 	 lr: 0.00100
[epoch   4: 260/307] 	 train loss: 0.333634 	 lr: 0.00100
[epoch   4: 280/307] 	 train loss: 0.712478 	 lr: 0.00100

val loss: 0.488999 	 acc: 0.000000

[epoch   4: 300/307] 	 train loss: 0.461941 	 lr: 0.00100
[epoch   5:   0/307] 	 train loss: 0.480295 	 lr: 0.00100
[epoch   5:  20/307] 	 train loss: 0.308337 	 lr: 0.00100
[epoch   5:  40/307] 	 train loss: 0.756740 	 lr: 0.00100
[epoch   5:  60/307] 	 train loss: 0.552190 	 lr: 0.00100
[epoch   5:  80/307] 	 train loss: 0.518123 	 lr: 0.00100
[epoch   5: 100/307] 	 train loss: 0.719212 	 lr: 0.00100
[epoch   5: 120/307] 	 train loss: 0.408349 	 lr: 0.00100
[epoch   5: 140/307] 	 train loss: 0.633257 	 lr: 0.00100

val loss: 0.446945 	 acc: 0.000000

[epoch   5: 160/307] 	 train loss: 0.779942 	 lr: 0.00100
[epoch   5: 180/307] 	 train loss: 0.585827 	 lr: 0.00100
[epoch   5: 200/307] 	 train loss: 0.550597 	 lr: 0.00100
[epoch   5: 220/307] 	 train loss: 0.406125 	 lr: 0.00100
[epoch   5: 240/307] 	 train loss: 0.290858 	 lr: 0.00100
[epoch   5: 260/307] 	 train loss: 0.365490 	 lr: 0.00100
[epoch   5: 280/307] 	 train loss: 0.821365 	 lr: 0.00100

val loss: 0.458882 	 acc: 0.000000

have a probliem

d_utils.PointcloudRotate(axis=np.array([1, 0, 0])), 在所有的文件中并没有出现，不知道你怎么跑出来代码的

Confused about relation_prior.

Dear author, thank you for your excellent work about RSCNN. I read your paper and code carefully, but I cannot understand how to predefine the prior relation like Euclidean distance in the code, I just find it get something such as abs_coord, delta_x from input[ ] in pytorch_utils.py line52, but I don't know the input[] like. Can you tell me where I can find the prior relation.
I am looking forward to your reply, thank you.

Segmentation results on ShapeNet part benchmark

****** Airplane: 0.001279
****** Bag: 0.125000
****** Cap: 0.000000
****** Car: 0.015432
****** Chair: 0.066288
****** Earphone: 0.222222
****** Guitar: 0.000000
****** Knife: 0.000000
****** Lamp: 0.204545
****** Laptop: 0.000000
****** Motorbike: 0.102564
****** Mug: 0.000000
****** Pistol: 0.000000
****** Rocket: 0.041667
****** Skateboard: 0.066667
****** Table: 0.159297
************ Test Loss: 3.403158
************ Class_mIoU: 0.062810
************ Instance_mIoU: 0.084091
@Yochengliu
why the model of Cap, Mug and Pistol instance_ious is always 0.000000 ?

about release the code for segmentation task

Hi, @Yochengliu ,

Thanks for releasing your code. However, it currently contains only code for modelnet40 classificaton task. Would you also release the code for segmentation task?

THX!

RuntimeError: no deterministic convolution algorithms available in CuDNN

This one：
h_xi_xj = self.mapping_func2(self.activation(self.bn_mapping(self.mapping_func1(h_xi_xj))))
cause ：

RuntimeError: no deterministic convolution algorithms available in CuDNN

compared with pointConv

Hello authors,
Thanks a lot for releasing the code. The work is awesome!
For now, I am confused by difference pointConv in PointConv: Deep Convolutional Networks on 3D Point Clouds and RSConv in your paper. For me, they look so similar. The main difference it the input of weight function. In RsConv, the input could be euclidean distance, normal, etc. In PointConv, the input could be the transition vector. Are the inputs significant? Could you please help to clarify that? It is appreciated.
Thanks!

Can you share Multi-Scale Grouping Classification config file ?

We have reproduce the reported performance of single scale group for classification task using the MXNet framework, however we tried many times but still can not reproduce the performance under multi-scale config . Can you share your config file ? (such as radius, mlp layers ..etc). Any suggestions are appreciated.