angeladai / cnncomplete

[CVPR'17] Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis

cnncomplete

This repo contains code to train a volumetric deep neural network to complete partially scanned 3D shapes. More information can be found in our paper.

Data

Train/test data is available for download on our project website.

Code

Installation:

Training tasks use Torch7, with torch packages cudnn, cunn, torch-hdf5, xlua.

Matlab visualization of the isosurface in testing uses the matio package.

The shape synthesis code was developed under VS2013, and uses flann (included in external).

Training:

• th train_class.lua -model epn-unet-class -save logs-epn-unet-class -train_data data/h5_shapenet_dim32_sdf/train_shape_voxel_data_list.txt -test_data data/h5_shapenet_dim32_sdf/test_shape_voxel_data_list.txt -gpu_index 0
• For more options, see help: th train_class.lua -h or th train.lua -h
• Trained models: trained_models.zip (700mb)

Testing:

• th test.lua --model_path [path to model] --test_file sampledata/scan.h5 --output_path [path to output] --classifier_path [path to classifier model, only specify if using epn-class or epn-unet-class models]
• For more options, see help: th test.lua -h

Citation:

@inproceedings{dai2017complete,
  title={Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis},
  author={Dai, Angela and Qi, Charles Ruizhongtai and Nie{\ss}ner, Matthias},
  booktitle = {Proc. Computer Vision and Pattern Recognition (CVPR), IEEE},
  year = {2017}
}

License

This code is released under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (please refer to LICENSE.txt for details).

Contact:

If you have any questions, please email Angela Dai at [email protected].

cnncomplete's Issues

The usage of shapesynth

Hi, I saw there are two stages of this method, where the first one is the network training and the second one is the multi-resolution shape synthesis. I assume the current training and testing scripts are just for the first stage. Could you please provide how to run the second stage?

Error when executing test.lua

Hi,

I have faced an error when running the test script. I am using cuda 9.0, so I installed cudnn by this command git clone https://github.com/soumith/cudnn.torch.git -b R7 && cd cudnn.torch && luarocks make cudnn-scm-1.rockspec based on this: https://github.com/soumith/cudnn.torch/issues/383, instead of luarocks install. And then I ran luarocks install nngraph. I downloaded the trained_models that were released in the project website. Please advice on how to properly set-up and run this scripts. Thanks a lot!

{
truncation : 3
classifier_path : "models/trained_models/classifier-partial.net"
output_path : "sample_output/"
test_file : "sampledata/scan.h5"
model_path : "models/trained_models/epn-unet-class.net"
use_log_transform : 1
gpu_index : 0
visualize_matlab : 1
help : false
}
loading trained network from file: models/trained_models/epn-unet-class.net
Warning: cudnn.convert does not work with nngraph yet. Ignoring nn.gModule/orion/u/mikacuy/torch/distro/install/bin/luajit: test.lua:83: attempt to call global 'isempty' (a nil value)
stack traceback:
test.lua:83: in main chunk
[C]: in function 'dofile'
...rch/distro/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00405d50

failure with model.forward()

hey guys,

got this code
vector class_names;
ifstream ifs(string("F:\DNN\alphabet_36.txt").c_str());
string line;
while (getline(ifs, line))
{
class_names.push_back(line);
}

if (Rotated_Image.empty())
{
	cout << "CHECK IMAGE FILE PATH!" << endl;

}
else
{
	cout << "IMAGE SUCCESSFULLY READ!" << endl;
}

auto model = readNetFromONNX("F:\\DNN\\CRNN_VGG_BiLSTM_CTC.onnx");
if (model.empty())
{
	cout << " CAN'T LOAD NEURAL NETWORK MODEL" << endl;
	return 0;
}
else
{
	cout << " NEURAL NETWORK MODEL LOADED" << endl;
}
Mat blob = blobFromImage(Rotated_Image,1.0, Size(300, 300), Scalar(127.5, 127.5, 127.5),true,false);
model.setInput(blob);
Mat outputs = model.forward();
Point classIdPoint;
double final_prob;
minMaxLoc(outputs.reshape(1, 1), 0, &final_prob, 0, &classIdPoint);
int label_id = classIdPoint.x;
string out_text = format("%s, %.3f", (class_names[label_id].c_str()), final_prob);
// put the class name text on top of the image
putText(Rotated_Image, out_text, Point(25, 50), FONT_HERSHEY_SIMPLEX, 1, Scalar(0, 255, 0), 2);
imshow("[]***", Rotated_Image);

return 0;

but i get these errors
OpenCV(4.4.0) Error: Assertion failed (ngroups > 0 && inpCn % ngroups == 0 && outCn % ngroups == 0) in cv::dnn::ConvolutionLayerImpl::getMemoryShapes, file C:\build\master_winpack-build-win64-vc15\opencv\modules\dnn\src\layers\convolution_layer.cpp, line 348
[ERROR:0] global C:\build\master_winpack-build-win64-vc15\opencv\modules\dnn\src\dnn.cpp (3444) cv::dnn::dnn4_v20200609::Net::Impl::getLayerShapesRecursively OPENCV/DNN: [Convolution]:(45): getMemoryShapes() throws exception. inputs=1 outputs=0/1 blobs=2
[ERROR:0] global C:\build\master_winpack-build-win64-vc15\opencv\modules\dnn\src\dnn.cpp (3447) cv::dnn::dnn4_v20200609::Net::Impl::getLayerShapesRecursively input[0] = [ 1 3 480 640 ]
[ERROR:0] global C:\build\master_winpack-build-win64-vc15\opencv\modules\dnn\src\dnn.cpp (3455) cv::dnn::dnn4_v20200609::Net::Impl::getLayerShapesRecursively blobs[0] = CV_32FC1 [ 64 1 3 3 ]
[ERROR:0] global C:\build\master_winpack-build-win64-vc15\opencv\modules\dnn\src\dnn.cpp (3455) cv::dnn::dnn4_v20200609::Net::Impl::getLayerShapesRecursively blobs[1] = CV_32FC1 [ 64 1 ]
[ERROR:0] global C:\build\master_winpack-build-win64-vc15\opencv\modules\dnn\src\dnn.cpp (3457) cv::dnn::dnn4_v20200609::Net::Impl::getLayerShapesRecursively Exception message: OpenCV(4.4.0) C:\build\master_winpack-build-win64-vc15\opencv\modules\dnn\src\layers\convolution_layer.cpp:348: error: (-215:Assertion failed) ngroups > 0 && inpCn % ngroups == 0 && outCn % ngroups == 0 in function 'cv::dnn::ConvolutionLayerImpl::getMemoryShapes'

what did i miss

mat2obj command for Matlab

Matlab returns error when executes the command
"mat2obj('%s', '%s', %d);", matfilename, objfilename, isoval) generated in test.lua.

How can I find the implementation of the mat2obj function, please? Thanks.

Error when running model:forward()

Hi,
Thanks a lot for this great code!

I meet a problem when I tried to run the test code.

First, I am not sure what is the difference between 'train.lua' and 'train_class.lua' and the difference between the trained_models between 'epn-unet.net' and 'epn-unet-class.net'. Since I found there is no file "classifier-partial.net" which is required in train_class.lua, I run the -th train.lua instead.

However, I got an error after loading the first data as follow:
/home/dan/torch/install/bin/luajit: /home/dan/torch/install/share/lua/5.1/nngraph/gmodule.lua:311: expecting table of 2 inputs
stack traceback:
[C]: in function 'error'
/home/dan/torch/install/share/lua/5.1/nngraph/gmodule.lua:311: in function 'forward'
train.lua:202: in function 'test_batch'
train.lua:171: in function 'test'
train.lua:218: in main chunk
[C]: in function 'dofile'
.../dan/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00405d50

I think this error come out from here:
output = model:forward(inputs)

I am a newbie and not sure whether anything wrong about the model structure. Could you please help me to check this problem. Thanks in advance!

Unable to run train_class.lua

Hi,
When I run the training command as you said on Readme. It said that I don't have "getLinesFromFile"
/home/bonnie/torch/install/bin/luajit: train_class.lua:31: attempt to call global 'getLinesFromFile' (a nil value) stack traceback: train_class.lua:31: in main chunk [C]: in function 'dofile' ...nnie/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk [C]: at 0x00405d50
I also add require 'util' at top of train_class.lua but it still revealed the same error.
Could you please help me?
Thanks!

Convert to sdf/df for custom objects

Is there any open source converter to convert obj/ply/pcd files to tsdf/sdf?

the problem of th test.lua

Hi, I am interested in your paper. I have a problem when running the code, as follows:
hp@hp-HP-Z840-Workstation:~/cnncomplete-master/torch$ th

______ __ | Torch7 / Debian
/_ / ________/ / | Scientific computing for Lua.
/ / / _ / __/ / _ \ | Type ? for help
/_/ _// _//// | https://github.com/torch
| http://torch.ch

th> os.exit()
hp@hp-HP-Z840-Workstation:~/cnncomplete-master/torch$ th test.lua
/usr/bin/luajit: /usr/share/lua/5.1/trepl/init.lua:389: module 'nn' not found:
no field package.preload['nn']
no file './nn.lua'
no file '/usr/share/luajit-2.1.0-beta3/nn.lua'
no file '/usr/local/share/lua/5.1/nn.lua'
no file '/usr/local/share/lua/5.1/nn/init.lua'
no file '/usr/share/lua/5.1/nn.lua'
no file '/usr/share/lua/5.1/nn/init.lua'
no file './nn.so'
no file '/usr/local/lib/lua/5.1/nn.so'
no file '/usr/lib/x86_64-linux-gnu/lua/5.1/nn.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
stack traceback:
[C]: in function 'error'
/usr/share/lua/5.1/trepl/init.lua:389: in function 'require'
test.lua:3: in main chunk
[C]: in function 'dofile'
/usr/lib/torch-trepl/th:149: in main chunk
[C]: at 0x562346d621d0

Unable to execute the test script

Hi

I have successfully compiled the code, but while executing the sample script I get the following error

/home/siddharth/.luarocks/share/lua/5.1/hdf5/init.lua:15 Unable to find the HDF5 lib we were built against - trying to find it elsewhere
/home/siddharth/torch/install/bin/luajit: /home/siddharth/torch/install/share/lua/5.1/trepl/init.lua:389: /home/siddharth/torch/install/share/lua/5.1/trepl/init.lua:389: /home/siddharth/.luarocks/share/lua/5.1/hdf5/ffi.lua:29: libhdf5.so: cannot open shared object file: No such file or directory
stack traceback:
[C]: in function 'error'
/home/siddharth/torch/install/share/lua/5.1/trepl/init.lua:389: in function 'require'
train_class.lua:5: in main chunk
[C]: in function 'dofile'
...arth/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00405d50

I have checked that lua.config points to the correct version of HDF5. Can anyone help ?