fqnchina / imagesmoothing Goto Github PK

Directly addding TH_API void THNN_CudaXXX to ./torch/extra/cunn/lib/THCUNN/generic/THCUNN.h does work.

I establish torch environment with Ubuntu 16.04, gcc-5/g++-5, GTX 1660 and CUDA 10.0. My Lua version is 5.3.

你好，我想问下，这篇论文中训练使用到Edge selection 和 Edge filtering边缘计算算法，使用pytorch训练该如何处理呢，能不能提供好的建议呢，谢谢！！

Great work! I have few questions, can you explain a bit for me please?

1. Why batch size set to 1?
2. I try implement it in pytorch and found that the value of Flattening Criterion is pretty big, is that normal?

Explain the torch version，Plwase. We can't release items of the compilation file. Thx

@fqnchina thanks for share the great paper and code.
Machine environment is ubuntu16.04; torch 7; cuda 8.
When I set up the environment to run this command
th test_smooth.lua
I got the error message:

torch/install/share/lua/5.2/nn/EdgeComputation.lua:17: attempt to call field' EdgeComputation_updateOutput' (a nil value)
stack traceback:
~/torch/install/share/lua/5.2/nn/EdgeComputation.lua:17: in function <~/torch/install/share/lua/5.2/nn/EdgeComputation.lua:9>
...

Edited file ~/torch/extra/nn/init.lua

require('nn.Jacobian')
require('nn.SparseJacobian')
require('nn.hessian')
require('nn.test')

require('nn.EdgeComputation')
require('nn.EdgeDetector')
require('nn.SmoothAndEdgeTerm')

return nn

Edited file ~/torch/extra/cunn/lib/THCUNN/generic/THCUNN.h

TH_API void THNN_(VolumetricUpSamplingTrilinear_updateGradInput)(
                  THCState *state,
                  THCTensor *gradOutput,
                  THCTensor *gradInput,
                  int nbatch,
                  int nchannels,
                  int inputDepth,
                  int inputHeight,
                  int inputWidth,
                  int outputDepth,
                  int outputHeight,
                  int outputWidth);

void THNN_CudaEdgeComputation_updateOutput(
		THCState *state, 
		THCudaTensor *input, 
		THCudaTensor *output);
void THNN_CudaEdgeComputation_updateGradInput(
		THCState *state, 
		THCudaTensor *input, 
		THCudaTensor *gradOutput, 
		THCudaTensor *gradInput);

void THNN_CudaEdgeDetector_updateOutput(
		THCState *state, 
		THCudaTensor *input_image, 
		THCudaTensor *input_edge, 
		THCudaTensor *label_preserve, 
		THCudaTensor *label_eliminate, 
		int isSmoothing);

void THNN_CudaSmoothAndEdgeTerm_updateOutput(
		THCState *state, 
		THCudaTensor *input_cnn, 
		THCudaTensor *input_edge, 
		THCudaTensor *target_yuv, 
		THCudaTensor *target_edge, 
		THCudaTensor *target_edge_mask, 
		THCudaTensor *smooth_mask_pre, 
		THCudaTensor *smooth_mask, 
		THCudaTensor *weight, 
		THCudaTensor *output, 
		float sigma_color, 
		float sigma_space, 
		int window_size, 
		float lp, 
		int isDetailEnhancement, 
		int isStylization, int w_L2);

void THNN_CudaSmoothAndEdgeTerm_updateGradInput(
		THCState *state, 
		THCudaTensor *input_cnn, 
		THCudaTensor *smooth_mask, 
		THCudaTensor *target_edge_mask, 
		THCudaTensor *weight, 
		THCudaTensor *gradInput, 
		float sigma_color, 
		int window_size, 
		float lp, 
		int w_L2);

If I add "TH_API" before "void" , then will receive another error.

/home/hl/torch/install/bin/lua: /home/hl/torch/install/share/lua/5.2/trepl/init.lua:389:   
/home/hl/torch/install/share/lua/5.2/trepl/init.lua:389:   
/home/hl/torch/install/share/lua/5.2/cunn/THCUNN.lua:77:   
unknown type TH_API on line 1428
stack traceback:
	[C]: in function 'error'

I can't find the solution for this, could you help me? Thanks a lot.

If anyone of you were able to complete the setup and run the code, let me know. I need help.

When run "th ./test_smooth.lua", I got the following error "...age_Smoothing/torch/install/share/lua/5.1/torch/File.lua:343: unknown Torch class <cudnn.SpatialDilatedConvolution>", which occurred in the line

In my installation, the 'SpatialDilatedConvolution' can be found in "./torch/extra/nn/init.lua" but not in "./torch/extra/cudnn/init.lua", which makes me think this cudnn version of SpatialDilatedConvolution is your implementation. Is that the case?

My setting is Ubuntu16.04, torch7, cuda-8.0 and cudnn-6.0, and I have succeeded in compiling all the files under the "/compilation" folder according to your instructions.

Any help would be appreciated.

HI,

I was able to install lua successfully and was also able to compile self defined layers mentioned in the project. (No compilation error for all 3 modules)

But when I run the test_smooth.lua
I get following error

mkothule@mkothule:~/image_smoothing/ImageSmoothing$ th test_smooth.lua
not found: THNN_CudaDoubleEdgeComputation_updateOutput/home/mkothule/torch/install/share/lua/5.1/nn/THNN.lua:108: /home/mkothule/torch/install/lib/lua/5.1/libTHCUNN.so: undefined symbol: THNN_CudaDoubleEdgeComputation_updateOutput
not found: THNN_CudaDoubleEdgeComputation_updateGradInput/home/mkothule/torch/install/share/lua/5.1/nn/THNN.lua:108: /home/mkothule/torch/install/lib/lua/5.1/libTHCUNN.so: undefined symbol: THNN_CudaDoubleEdgeComputation_updateGradInput
not found: THNN_CudaDoubleEdgeDetector_updateOutput/home/mkothule/torch/install/share/lua/5.1/nn/THNN.lua:108: /home/mkothule/torch/install/lib/lua/5.1/libTHCUNN.so: undefined symbol: THNN_CudaDoubleEdgeDetector_updateOutput
not found: THNN_CudaDoubleSmoothAndEdgeTerm_updateOutput/home/mkothule/torch/install/share/lua/5.1/nn/THNN.lua:108: /home/mkothule/torch/install/lib/lua/5.1/libTHCUNN.so: undefined symbol: THNN_CudaDoubleSmoothAndEdgeTerm_updateOutput
not found: THNN_CudaDoubleSmoothAndEdgeTerm_updateGradInput/home/mkothule/torch/install/share/lua/5.1/nn/THNN.lua:108: /home/mkothule/torch/install/lib/lua/5.1/libTHCUNN.so: undefined symbol: THNN_CudaDoubleSmoothAndEdgeTerm_updateGradInput
not found: THNN_CudaHalfEdgeComputation_updateOutput/home/mkothule/torch/install/share/lua/5.1/nn/THNN.lua:108: /home/mkothule/torch/install/lib/lua/5.1/libTHCUNN.so: undefined symbol: THNN_CudaHalfEdgeComputation_updateOutput
not found: THNN_CudaHalfEdgeComputation_updateGradInput/home/mkothule/torch/install/share/lua/5.1/nn/THNN.lua:108: /home/mkothule/torch/install/lib/lua/5.1/libTHCUNN.so: undefined symbol: THNN_CudaHalfEdgeComputation_updateGradInput
not found: THNN_CudaHalfEdgeDetector_updateOutput/home/mkothule/torch/install/share/lua/5.1/nn/THNN.lua:108: /home/mkothule/torch/install/lib/lua/5.1/libTHCUNN.so: undefined symbol: THNN_CudaHalfEdgeDetector_updateOutput
not found: THNN_CudaHalfSmoothAndEdgeTerm_updateOutput/home/mkothule/torch/install/share/lua/5.1/nn/THNN.lua:108: /home/mkothule/torch/install/lib/lua/5.1/libTHCUNN.so: undefined symbol: THNN_CudaHalfSmoothAndEdgeTerm_updateOutput
not found: THNN_CudaHalfSmoothAndEdgeTerm_updateGradInput/home/mkothule/torch/install/share/lua/5.1/nn/THNN.lua:108: /home/mkothule/torch/install/lib/lua/5.1/libTHCUNN.so: undefined symbol: THNN_CudaHalfSmoothAndEdgeTerm_updateGradInput
Found Environment variable CUDNN_PATH = /opt/cuda/cuda/lib64/libcudnn.so.5
/home/mkothule/torch/install/bin/luajit: /home/mkothule/torch/install/share/lua/5.1/torch/File.lua:343: unknown Torch class <cudnn.SpatialDilatedConvolution>
stack traceback:
[C]: in function 'error'
/home/mkothule/torch/install/share/lua/5.1/torch/File.lua:343: in function 'readObject'
/home/mkothule/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject'
/home/mkothule/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject'
/home/mkothule/torch/install/share/lua/5.1/nn/Module.lua:192: in function 'read'
/home/mkothule/torch/install/share/lua/5.1/torch/File.lua:351: in function 'readObject'
/home/mkothule/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject'
/home/mkothule/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject'
/home/mkothule/torch/install/share/lua/5.1/nn/Module.lua:192: in function 'read'
/home/mkothule/torch/install/share/lua/5.1/torch/File.lua:351: in function 'readObject'
/home/mkothule/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject'
...
/home/mkothule/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject'
/home/mkothule/torch/install/share/lua/5.1/torch/File.lua:353: in function 'readObject'
/home/mkothule/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject'
...mkothule/torch/install/share/lua/5.1/nngraph/gmodule.lua:495: in function 'read'
/home/mkothule/torch/install/share/lua/5.1/torch/File.lua:351: in function 'readObject'
/home/mkothule/torch/install/share/lua/5.1/torch/File.lua:409: in function 'load'
test_smooth.lua:27: in main chunk
[C]: in function 'dofile'
...hule/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x560d702bd570

I am not sure what I am missing here.
If you have seen this error pleae let me know

Please add the link to your Arxiv paper, i.e. https://arxiv.org/abs/1811.02804 to the repo's readme. Thanks.

I have some trouble about running run_edgedetector.lua and test_smooth.lua

/home/xupp/torch/install/bin/luajit: .../xupp/torch/install/share/lua/5.1/nn/EdgeComputation.lua:17: attempt to call field 'EdgeComputation_updateOutput' (a nil value)
stack traceback:
	.../xupp/torch/install/share/lua/5.1/nn/EdgeComputation.lua:17: in function 'func'
	/home/xupp/torch/install/share/lua/5.1/nngraph/gmodule.lua:345: in function 'neteval'
	/home/xupp/torch/install/share/lua/5.1/nngraph/gmodule.lua:380: in function 'forward'
	run_edgedetector.lua:53: in main chunk
	[C]: in function 'dofile'
	...xupp/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
	[C]: at 0x00405d50

Can you provide a docker environment?
Thanks.

Hi, @fqnchina
Let's say the input is "./images/detail_enhancement/detail_2_5.png", so the intermediate smooth image "detail_2_5-predict.png" can be inferred by the pre-trained model "./netfiles/model_smooth_detail-enhance_30.net", then how can I further get the final "detail_2_5-predict-detail.png"?

According to the paper, it can be obtained by the following process:

img_input = imread("./images/detail_enhancement/detail_2_5.png")
img_smooth = model(img_input, "./netfiles/model_smooth_detail-enhance_30.net")  # "detail_2_5-predict.png"

img_detail = img_input - img_smooth
img_detail_enhanced = enhance(img_detail)  # What's this exactly?

img_output = img_input + img_detail_enhanced  # "detail_2_5-predict-detail.png"

if I don't misunderstand, not so sure of myself.
Thank you in advance.