Two questions about the library about xnnpack

[Maratyszcza]

For FullyConnected operator, it is (output_dim, input_dim). Generally operators use NHWC layout.

[Maratyszcza]

You can set all height dimensions (input height, kernel height, height subsampling) to 1, this would be equivalent to 1D convolution.

[Maratyszcza]

1. F32 functions don't quantize computations. However, many F32 operators accept output_min and output_max arguments which enable clamping output to arbitrary range (helpful e.g. for fusing simple activation functions). If you don't want to clamp output, just set them to +-std::numeric_limits<float>::infinity()
2. XNNPACK is a fork of QNNPACK. Q8 operators in XNNPACK are remnants of QNNPACK code, but many internal optimizations were removed, and performance of these operators would be worse than in QNNPACK.

[RuABraun]

Okay thank you!

[RuABraun]

@Maratyszcza one more question, is the kernel shaped (input_dim, output_dim) or (output_dim, input_dim) ?

edit: seems to be (output_dim, input_dim)

[RuABraun]

Works great!

Two more questions, any plans on adding an elementwise product operator? And is it somehow possible to make the FullyConnected add to the the output instead of setting it?

[Maratyszcza]

Elementwise product (including broadcasting support) landed just yesterday, see xnn_create_multiply_nd_f32 and xnn_setup_multiply_nd_f32. Currently Fully Connected operator doesn't support fused addition, but you can use separate Add operator (see xnn_create_add_nc_f32 and xnn_setup_add_nc_f32)

[RuABraun]

Haha perfect timing! Thanks

[RuABraun]

@Maratyszcza seems broadcasting support is not available for the add operation (for when one wants to implement batch normalization for example)? is my impression correct?

could disabling the check and setting the bias mean stride to 0 work as quick hack?

edit: hah looks like it does!

[Maratyszcza]

For batch normalization, I'd recommend to convert it into 1x1 depthwise convolution. XNNPACK has special optimizations for 1x1 DW convolution.

[RuABraun]

I'm not working with images, so my input/output matrixes are only 2D, I assume xnn_create_convolution2d... would crash then (don't see a 1D version)? Don't see an option to set stride to 0 in one direction.

[RuABraun]

Ah thanks, forgot to check the setup function.