Unable to execute GPT-2 onnx model about armnn

[Colm-in-Arm]

Hi,

The fatal error message indicates that there are INT64 types in the model. Our Onnx parser does not support this data type. Our ONNX parser is very outdated and has been marked for future deprecation. So unless you're willing to contribute the work yourself, I'm afraid this model won't work.

Colm.

[somasundaram1702]

@Colm-in-Arm : Does tf-lite support INT64 types or do you recommend any other parsers? My objective is to run inferencing on any of the LLM model on Mali G710 GPU utilizing ExecuteNetwork. Any successful use case available ? If so, kindly can you direct me to the working model ?

[Colm-in-Arm]

Hi,

TfLite runtime does support INT64 in some limited cases. I don't know of other ONNX runtimes you could use.

In Arm NN we have not done any work on LLM's. The work I have seen tends to target the CPU rather than GPU. LLM's tend to be memory bound rather than CPU bound so there's not as much potential for performance increase using GPU's.

Colm.