[Question]: The number of convolutional multiplication decreases but the communication cost increases in SPU about spu

[fionser]

You can see, the number of truncations increases a lot. That should be normal.
When doing matmul, the number of truncations should be quadratic to the matrix size, but for Winograd, I think you need to write SPU in C++ to reduce the number of truncations. For example, to add some multiplication results then perform one truncation, instead of calling many truncations on them.

[llCurious]

hi, @warpoons . Interesting idea. As pointed out by @fionser , the problem is due to the increasing amont of truncations.

According to the Winograd algorithm, the matmul is separated into several parts (currently, each part shall incur additional truncations), which I believe is not friendly in SPU.

In my opinion, to maximize the performance of Winograd, you may need to add a backend op for Winograd, and implement the algorithm in C++.

[warpoons]

Hi @llCurious @fionser ! Thanks for your response!

As pointed out by @fionser , when doing matmul, the number of truncations should be quadratic to the matrix size. In Winograd, the input feature map is separated into several overlapped parts (or called tiles) and do element-wise matmul separately. Reasonably, there will be additional EWMMs among all the tiles than the standard conv. Is this understanding correct?

I have another question, is there a method to estimate the theoretical communication cost of standard conv and Winograd conv (considering only the EWMMs in Winograd and do the transformation of weights offline) in SPU?

Thanks!