iOS support about nnpack

[Maratyszcza]

iOS is supported since we landed CMake configs.

[Maratyszcza]

What do you mean by supporting iOS?
The PSIMD implementation can be built for any architecture supported by clang. Performance, of course, wouldn't be as good as native assembly.

[joker512]

Actually, I don't understand what is PSIMD implementation...
That's what I see in your README.md:

NNPACK can be build on OS X and Linux
...

and

Cross-compilation for Native Client
...

Both are not about compilation for iOS.

[Maratyszcza]

Its not yet documented, but you can configure with --enable-psimd option, and NNPACK will use kernels from src/psimd

[joker512]

Ok, I see, thank you.

We've tried this option on iPhone 6S and got only 10% performance boost. Is it correct or we did something wrong? Or maybe, not all capabilities of ARM architecture are used in the current NNPACK and you could support ARM better in future.

In short, do you think it is possible to get more boost on iOS?

[Maratyszcza]

The PSIMD branch is generic 128-bit SIMD, it misses many ARM-specific optimization opportunities. It is possible to do much better on ARM, but currently I have no time to work on it.

However, you can get some perf boost on ARM with little effort: in the src/psimd/blas replace multiplication-accumulation with FMA (a * b + c -> vmlaq_f32).

[carlodelmundo-zz]

@joker512 Did you roll out your own version of ARM-based intrinsics in lieu of the SSE/AVX2 versions of @Maratyszcza ?

I'm interested in a fast iOS port as well.

[bhack]

@carlodelmundo There is some initial NEON support in master (I suppose mainly blas)

[wangxuewen99]

@Maratyszcza for ios, i have tried to compile nnpack with --enable-psimd option,but some error occurred. ld: warning: ignoring file *****/libnnpack.a, file was built for archive which is not the architecture being linked (arm64): *****/libnnpack.a. how can i solve this problem?3x

[Maratyszcza]

For iOS you'd have to create XCode project and do some porting. Its not supported out of the box.

[wangxuewen99]

@Maratyszcza I see ,thanks a lot for your reply

[vpatil131]

The PSIMD branch is generic 128-bit SIMD, it misses many ARM-specific optimization opportunities. It is possible to do much better on ARM, but currently I have no time to work on it.

@Maratyszcza I see that you have added optimized NEON intrinsic BLAS code under src/neon since your above comment. However, input & output Fourier transform is still using generic 128-bit SIMD code from src/psimd.

Do you think it's possible to get performance boost by writing ARM NEON intrinsics for input & output transform as well?

Thanks

[Maratyszcza]

@vpatilvasu Yes, I believe so.

[Yangqing]

You can potentially build it manually with the set of source files - here is how you can do it for example:

https://github.com/caffe2/caffe2/blob/58113f945426b0bacc33020ec499384417b86952/cmake/External/nnpack.cmake#L57-L122