What is role of scaling_per_output_channel in QuantReLU? about brevitas

[Giuseppe5]

Similar to what happens for weight scaling, you can have one scale factor for the entire tensor to quantize, or one per each channel of said tensor. Other slicing of the tensor to compute scale factors are also possible, although arguably less common (e.g., per-row, per-group, etc.).

The use of per tensor vs per channel depends on the network topology, hardware constraints of the device where you plan to execute your network, and other factors.

As a rule of thumb, the more fine grained the granularity of your scale factors, the better the final accuracy of the quantized network. Similarly, the computational cost and memory usage of your network will increase since scaling factors are stored in high precision.