Broadcasting about tflite2onnx

[zhenhuaw-me]

Facts and general design

ONNX has two concepts of broadcasting:

• Multidirectional Broadcasting for trival Binary operators. This is very similar to NumPy's broadcasting rules.
• Unidirectional Broadcasting for some others that are layout sensitiave.

By intuition, we can do something like a fake broadcasting for shape such that the resulted shape is always broadcastable but layout propagation compatible, while element size remains unchanged. That is prepending or appending 1 to both input shapes of the inputs, therefore the resulted shapes have same length which can be moved around by layout propagation without impact the semantic.

That sounds faire enough but there is a significant challenge: fake broadcasting may break semantic of other operators as the rank of the tensors has changed. For this, we can insert Reshape operator, the only open question is to transform the shape attribution of Reshape.

[zhenhuaw-me]

How to handle layout issue for Reshape

For Reshape, regardless of whether it is newly added or raw in TFLite model, we need to try to fix the shape attribution for it after layout propagation. Since the shape attribution describes the shape of output tensor, if the output as propagated to transform layout, so as the shape attribution.

The layouts for shape could be obtained during parsing, but not always. If it is avaiable during parsing, then the shape attribution (tensor actually) can be transformed directly. Otherwise, shape has no layout information even after layout propagation (unless shape is output of some other operators that have layout information, which is unusual case). For such issue, we perform sepcial handling in Reshape.transform() to update the shape tensor. Then, we are all set.

We may markup the layout of shape and re-propagate the layouts, but that is unnecessary and complicate commen scenarios.