What's the default quantization type? about tinyneuralnetwork

[peterjc123]

What's the default quantization type?

Asymmetric quantization, which uses uint8 as the data type.

where, even users set the 'asymmetric' to False, the codes still force to use uint8.

No, your understanding is not correct. Since we use the QNNPACK backend, the dtype of the weights are required to be of uint8 type at the PyTorch side. What actually matters here is the argument qscheme=torch.per_tensor_symmetric, apart from the default value torch.per_tensor_affine, which means the offset is always pinned to 128, while the scale remains learnable. Later, during the conversion phase, we will minus the unsigned quantized values (ranges from 0 to 256) by 128 so that they will become int8.
BTW, to use symmetric quantization, you'll have to pass additional flags to both the quantizer and the converter. Related comments:

• https://github.com/alibaba/TinyNeuralNetwork/blob/main/examples/qat/qat.py#L36
• https://github.com/alibaba/TinyNeuralNetwork/blob/main/examples/qat/qat.py#L74

[liamsun2019]

I get it. Thanks a lot.

[liamsun2019]

I set asymmetric=False to TFLiteConverter and the generated tflite is int8 quantized then. Big thanks.