colin97 / msn-point-cloud-completion Goto Github PK
View Code? Open in Web Editor NEWMorphing and Sampling Network for Dense Point Cloud Completion (AAAI2020)
License: Apache License 2.0
Morphing and Sampling Network for Dense Point Cloud Completion (AAAI2020)
License: Apache License 2.0
Hi,
Thanks for your great work.
When I train the model, here are some logs about connection refused, just as the following Traceback shown.
But the model in training ... , do I need care the connection logs?
Best and Thanks
Traceback (most recent call last):
File "/home/yjcai/anaconda3/envs/msn/lib/python3.7/site-packages/visdom/init.py", line 711, in _send
data=json.dumps(msg),
File "/home/yjcai/anaconda3/envs/msn/lib/python3.7/site-packages/visdom/init.py", line 677, in _handle_post
r = self.session.post(url, data=data)
File "/home/yjcai/anaconda3/envs/msn/lib/python3.7/site-packages/requests/sessions.py", line 578, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/home/yjcai/anaconda3/envs/msn/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/home/yjcai/anaconda3/envs/msn/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/home/yjcai/anaconda3/envs/msn/lib/python3.7/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
object at 0x7f1a8f4b1ad0>: Failed to establish a new connection: [Errno 111] Connection refused'))
MSN_TRAIN train [0: 0/9] emd1: 0.341410 emd2: 0.302342 expansion_penalty: 0.003462
MSN_TRAIN train [0: 1/9] emd1: 0.321795 emd2: 0.290033 expansion_penalty: 0.003422
MSN_TRAIN train [0: 2/9] emd1: 0.347615 emd2: 0.314816 expansion_penalty: 0.003539
MSN_TRAIN train [0: 3/9] emd1: 0.331207 emd2: 0.294323 expansion_penalty: 0.003690
MSN_TRAIN train [0: 4/9] emd1: 0.306453 emd2: 0.276825 expansion_penalty: 0.003754
MSN_TRAIN train [0: 5/9] emd1: 0.277850 emd2: 0.246808 expansion_penalty: 0.003884
MSN_TRAIN train [0: 6/9] emd1: 0.303935 emd2: 0.266290 expansion_penalty: 0.003877
MSN_TRAIN train [0: 7/9] emd1: 0.293241 emd2: 0.256959 expansion_penalty: 0.004172
MSN_TRAIN train [0: 8/9] emd1: 0.285149 emd2: 0.239006 expansion_penalty: 0.004101
MSN_TRAIN train [0: 9/9] emd1: 0.309674 emd2: 0.286977 expansion_penalty: 0.004666
Can you provide the preprocessed data for the test set of ShapeNet which contains 1200 models?
Hi,
I have a question about .obj file normalization method.
I want to test your deep-learning model on my 3D modeling dataset.
But, I don't find method that normalize .obj file size x, y, z components like your dataset(ShapeNet).
If you know about it, tell me anything.
Thank you.
_get_cuda_arch_flags
raise ValueError("Unknown CUDA arch ({}) or GPU not supported".format(arch))
Thanks for your repository.
Do you think it is possible to use your code to calcule the distances between two point clouds ?
I have S_A and S_B are two point clouds sampled on the same surface (at two different time, the surface may be deformed or translated/rotated).
And I want to calculate the distance between each point in S_B with the corresponding implicit surface of S_A (an example is a monitoring project to detect changes between two point clouds)
Thanks very much
I successfully compiled EMD using CUDA 10.1 but I encountered some undefined symbol error. Does the code not support CUDA 10.1?
I got the following errors when setting n_pts
to 16384
.
Namespace(batchSize=16, env='MSN_TRAIN', model='', n_primitives=16, nepoch=25, num_points=16384, workers=24)
Random Seed: 3764
Train Set Size: 28974
Traceback (most recent call last):
File "train.py", line 121, in <module>
output1, output2, emd1, emd2, expansion_penalty = network(input, gt.contiguous(), 0.005, 50)
File "/mnt/lustre/xiehaozhe/Applications/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
result = self.forward(*input, **kwargs)
File "/mnt/lustre/xiehaozhe/Applications/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 152, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/mnt/lustre/xiehaozhe/Applications/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 162, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/mnt/lustre/xiehaozhe/Applications/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 83, in parallel_apply
raise output
File "/mnt/lustre/xiehaozhe/Applications/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 59, in _worker
output = module(*input, **kwargs)
File "/mnt/lustre/xiehaozhe/Applications/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
result = self.forward(*input, **kwargs)
File "train.py", line 38, in forward
output1, output2, expansion_penalty = self.model(inputs)
File "/mnt/lustre/xiehaozhe/Applications/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
result = self.forward(*input, **kwargs)
File "/mnt/lustre/xiehaozhe/Development/MSN-Point-Cloud-Completion/model.py", line 159, in forward
dist, _, mean_mst_dis = self.expansion(out1, self.num_points//self.n_primitives, 1.5)
File "/mnt/lustre/xiehaozhe/Applications/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
result = self.forward(*input, **kwargs)
File "./expansion_penalty/expansion_penalty_module.py", line 53, in forward
return expansionPenaltyFunction.apply(input, primitive_size, alpha)
File "./expansion_penalty/expansion_penalty_module.py", line 27, in forward
assert(primitive_size <= 512)
AssertionError
Hi @Colin97,
Thanks for the great work and releasing the code.
I am using the EMD function and I found that it requires the number of input points to be divisible by 1024
class emdFunction(Function):
@staticmethod
def forward(ctx, xyz1, xyz2, eps, iters):
batchsize, n, _ = xyz1.size()
_, m, _ = xyz2.size()
assert(n == m)
assert(xyz1.size()[0] == xyz2.size()[0])
assert(n % 1024 == 0)
assert(batchsize <= 512)
I am trying to compute the EMD distance between two point clouds contain 20 points each so I wonder can I remove this assert safely? or is there an easy way to modify the code for a 20-point point cloud?
Best,
Zirui
Hi Colin,
Nice work!
Figure 6 is so beautiful, how do you draw it? Using matplotlib, cloudcompare or meshlab?
Do you have a function to save the point cloud output?
Hi, Thank you for your good job! How to change the EMD wrapper to deal with flexible number of points, instead of just 1024x?
Thanks for your great work,
G++ error occurred when I entered python setup.py install.
running install running bdist_egg running egg_info writing emd.egg-info/PKG-INFO writing dependency_links to emd.egg-info/dependency_links.txt writing top-level names to emd.egg-info/top_level.txt listing git files failed - pretending there aren't any reading manifest file 'emd.egg-info/SOURCES.txt' writing manifest file 'emd.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_ext building 'emd' extension creating /home/lq/New_p/VRCNet-main/utils/emd/build creating /home/lq/New_p/VRCNet-main/utils/emd/build/temp.linux-x86_64-3.7 Emitting ninja build file /home/lq/New_p/VRCNet-main/utils/emd/build/temp.linux-x86_64-3.7/build.ninja... Compiling objects... Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) 1.10.2.git.kitware.jobserver-1 creating build/lib.linux-x86_64-3.7 g++ -pthread -shared -B /home/lq/anaconda3/envs/mvp/compiler_compat -L/home/lq/anaconda3/envs/mvp/lib -Wl,-rpath=/home/lq/anaconda3/envs/mvp/lib -Wl,--no-as-needed -Wl,--sysroot=/ /home/lq/New_p/VRCNet-main/utils/emd/build/temp.linux-x86_64-3.7/emd.o /home/lq/New_p/VRCNet-main/utils/emd/build/temp.linux-x86_64-3.7/emd_cuda.o -L/home/lq/anaconda3/envs/mvp/lib/python3.7/site-packages/torch/lib -L/usr/local/cuda-10.1/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-3.7/emd.cpython-37m-x86_64-linux-gnu.so g++: error: /home/lq/New_p/VRCNet-main/utils/emd/build/temp.linux-x86_64-3.7/emd.o: No such file or directory g++: error: /home/lq/New_p/VRCNet-main/utils/emd/build/temp.linux-x86_64-3.7/emd_cuda.o: No such file or directory error: command 'g++' failed with exit status 1
Hello, thanks for your wonderful work!
If I have two sets of data whose dimensions are larger than 3. Can I still use your emd to calculate their distance?
x1 = torch.rand(20, 8192, 100).cuda()
x2 = torch.rand(20, 8192, 100).cuda()
emd = emdModule()
dis, assigment = emd(x1, x2, 0.05, 3000)
I have tested it and no error is reported. But I am not sure if it makes sense. Thanks!
It seems that EMD and Verified EMD are different:
Input_size: torch.Size([20, 8192, 100])
Runtime: 0.863681s
EMD: 0.046708
|set(assignment)|: 8192
Verified EMD: 4.075622
How to visualize the results
Nothing's more refreshing than enjoying such a 3d models picture!
You also put some super nice point clouds in:
https://github.com/Colin97/MSN-Point-Cloud-Completion/blob/master/emd/CDEMD.png
I have seen similar point cloud images in some other papers, and I always wanted to know how to draw them. I would be very appreciate it if you could show it to me.
ヾ(≧▽≦*)o
Hello,
i have an issue when trying to train the network. The loss_net.backward() call fails with this pytorch error
RuntimeError: Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false.
But so far I was not able to find the reason for this error. Has someone else experienced this issue yet?
Thank you very much
Hi Colin,
Awesome work !
I had a question regarding the normalisation of the point cloud for the EMD. Could you tell me why is it needed to normalise the coordinates between 0 and 1. What will happen if I don't do that ?
I try the code to generate the partial point clouds but got the failure. Could you please give more details about the environment for the code to generate the partial point clouds?
Hi,
After following the render process in https://github.com/wentaoyuan/pcn/tree/master/render, I can get those .pcd files, which should be the training data? Are there any other steps?
The completed data are provided in the google drive, so there is no need to follow the sample process in https://github.com/wentaoyuan/pcn/tree/master/sample?
Thank you!!
centos7
pytorch1.2
CUDA10.0
python 3.7
ERROR:
Traceback (most recent call last):
File "setup.py", line 2, in
from torch.utils.cpp_extension import BuildExtension, CUDAExtension
ImportError: No module named torch.utils.cpp_extension
But I can run "import torch.utils.cpp_extension" by python in this environment
Hi, thanks for your impressive work, how about your training time over the whole dataset you provided (about 30000 shapes and 50 views per shape)? @Colin97
Hello, how do you display the point cloud image in the paper? Can you publish your code?
I'm running val.py on a windows machine with a Nvidia 1060 6GB VRAM and I get the following:
RuntimeError: CUDA out of memory. Tried to allocate 1.56 GiB (GPU 0; 6.00 GiB total capacity; 2.12 GiB already allocated; 1.06 GiB free; 1.54 GiB cached)
How much VRAM did you have on your system?
When I try to train the network, I got the following error.
Namespace(batchSize=16, env='MSN_TRAIN', model='', n_primitives=16, nepoch=25, num_points=8192, workers=24)
Exception in user code:
------------------------------------------------------------
Random Seed: 1957
Train Set Size: 28974
Traceback (most recent call last):
File "train.py", line 121, in <module>
output1, output2, emd1, emd2, expansion_penalty = network(input, gt.contiguous(), 0.005, 50)
File "/mnt/lustre/xiehaozhe/Applications/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
result = self.forward(*input, **kwargs)
File "/mnt/lustre/xiehaozhe/Applications/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 152, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/mnt/lustre/xiehaozhe/Applications/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 162, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/mnt/lustre/xiehaozhe/Applications/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 83, in parallel_apply
raise output
File "/mnt/lustre/xiehaozhe/Applications/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 59, in _worker
output = module(*input, **kwargs)
File "/mnt/lustre/xiehaozhe/Applications/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
result = self.forward(*input, **kwargs)
File "train.py", line 38, in forward
output1, output2, expansion_penalty = self.model(inputs)
File "/mnt/lustre/xiehaozhe/Applications/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
result = self.forward(*input, **kwargs)
File "/mnt/lustre/xiehaozhe/Development/MSN-Point-Cloud-Completion/model.py", line 168, in forward
resampled_idx = MDS_module.minimum_density_sample(xx[:, 0:3, :].transpose(1, 2).contiguous(), out1.shape[1], mean_mst_dis)
File "./MDS/MDS_module.py", line 29, in forward
MDS.minimum_density_sampling(xyz, npoint, mean_mst_length, idx)
AttributeError: module 'MDS' has no attribute 'minimum_density_sampling'
Hi,
I tried to run emd_module/test_emd(). However, after "python3 setup.py install" and run emd_module.py, it shows "AttributeError: module 'emd' has no attribute 'forward'", at line 58.
Any possible reason about it?
Here's the configuration of my machin:
Python 3.7
Pytorch 1.3.0
CUDA 9.2
Thanks!
Hi has anyone been able to train MSN-PCC on their own custom dataset? If so, which categories and how well did it perform? Were there any dataset generation/loading problems?
Hey all,
I see that you have a custom dataloader for the solution, but I was curious how to use this network on my own data. For example, say I have a PCD which is already incomplete, I do not have the ground truth for this. I just want to run inference on this data. How would be the best way to progress using this network? Any help on this would be very much appreciated. Thank you for the contribution !!
Hello @Colin97 , I was wondering if there's a way to perform transfer learning on the model you provided? Also, could you elaborate on the data prep part? I'm using ply files of my own, not from ShapeNet.
Hello~ Thank you for sharing your nice work!
I have two questions about the MDS_cuda.cu
file.
In line 114, (float t = 5.0 * mean_mst_length[batch_index] * mean_mst_length[batch_index];
)
I read your paper(arXiv ver.), and It seems that you used the mean_mst_length
as parameter sigma(in the paper). But I can't understand why the coefficient is 5.0, not 2.0. Could you explain me about this?
In line 130, (temp[k] += k < 8192 ? exp(-d / t) : exp(-d / t) * 2.0;
)
Could you explain me why this line is needed?
Just out of curiosity, have you tried training jointly on multiple categories and/or tested on unknown categories?
Of course results would degrade, but would it still give meaningful results?
Thanks!
Hi, on your paper, you tested your model with Chamfer Distance as a loss function. Can you share the resulting model?
Also, Is there a reason that you created a new dataset rather than using the existing "PCN dataset"?
Hi,
Thanks for your great work.
When I tried to evaluate MSN on the completion 3d benchmark, I found some difficulties, such as the version of cuda and pytorch. Completion 3d benchmark uses cuda9.0 and pytorch0.4.0.
Have you evaluated MSN on Completion 3d Benchmark?
Thanks for your nice implementation.
I was very confused by the running time for computing EMD. When eps = 0.05, k = 1000, I got the following results. The size of input is (32, 16834, 3).
The runtime is much longer with real data and could you please explain the difference?
Dear,
Thanks for your outstanding work!
I have followed the render process in PCN, and obtained the partial point cloud from ShapeNet for training. However, I find that some of my results are a little noisy as follows:
And I notice the val data you provide is title as 'model_id/scan_id_denoised.pcd', so I am wondering whether there is a further denosing step to be taken to refine the generated partial data?
Have a good day!
Great work and thanks for sharing the code!
I am wondering how to modify your code to work with points in a 4 dim space, i.e. points with shape [#batch, #points, 4].
It seems that I only need to modify some lines in <emd_cuda.cu>, but since I do not fimilar with cuda programming at all,
it will be appreciated if you can give me some detail advises!
It seems like that the partial point cloud dataset generated from ShapeNet is very large (up to 49 GB), so could you please give the information of the GPU you used in the experiment ShapeNet completion? Can we train the model with a general GPU?
Is it able to run VAL on a CPU computer? Look like I need an expansionPenaltyFunction class that supports CPU tensor
Is The PCN model you compared with trained by yourself? Or you just use the pretrained model released by the PCN official? And I want to know the PCN you used is PCN-CD or PCN-EMD.
Hi Colin,
I have an other question regarding the eps and k parameters for the EMD loss. Could you provide a more detail explanation of their purpose ?
Thanks,
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.