sparse input about convolution-with-sparse-kernel-in-tf

[Connor323]

Hi David, it only works for sparse kernel, but not input, since the purpose for this program is for deep learning pruning. Sorry about that..

[davidenitti]

I understand. However, since the convolution operator is commutative (up to a mirroring), I could swap kernel and input.
Thus the kernel becomes the input becomes the kernel.
do you think that having a kernel that changes at every image kills the speed improvements?
in other words, do you have a kernel processing step that is computationally heavy? (and thus not convenient if done at every image)

thank you very much!

[Connor323]

Gotcha! As I mentioned in the TODO list, for now, the conversion of dense to sparse tensor is inside the operation, which means that for each time of calling the operation, it needs to process the conversion every time. Therefore, it is the same as the case you mentioned that processing kernel at every image and it wouldn't kill the current performance. However, the speed also depends on the size of input, since I only implemented multi-threading for convolution along the width, height, channel of the input tensor, but not kernel channels. So if the input size is small comparing with kernel, the operation will be slow (even slower than the build-in operation, since their convolution program is more robust to this situation :P).

[Connor323]

That's why in the last plot figure the improvement is mediocre when the input size is less than 128.

[davidenitti]

if I have an input of 300x300 and a kernel of 3x3 and I invert, I have an input of 3x3 and a kernel of 300x300. However, the input needs to be heavily padded, to more than 600x600, so you will still have a big input. do you think that the performance would still be good?
I would have tried by myself, but I'm not able to compile your code with Tensorflow 1.4

[Connor323]

That's a really good question! I just tested the case you mentioned, the result of sparse convolution is 5272ms while the build-in one is 379ms. However, the time for the actual convolution computing from the sparse one is only 89ms and the rest of time (5183ms) is for dense to sparse conversion. Therefore, in this case, the sparse operation is still faster, but the overall performance is not good.

Regarding the issue of TF1.4, according to the issue, it seems TF1.4 has some problem for op customization with GPU. I'll compile 1.4 later to see if there is a solution for this. By the way, could you please tell me the error when you compile the code?

[davidenitti]

my idea is to work directly on sparse input/kernel, so I don't actually need a dense to sparse conversion.

there are several issues with TF1.4, first it doesn't find cuda.h, but I fixed that and I have other issues.

[Connor323]

Yea, this implementation assumes the input sparse tensor is in dense format; I'll separate the conversion and convolution parts later, but for now, it seems this program is not suitable in your case. Sorry about that.

[davidenitti]

no problem, thank you very much for the help!