Loading ONNX neuron-compiled models about aws-neuron-sdk

[fishuke]

@aws-taylor We need onnx neuron support in order to run one of the most common nude detection ai application NudeNet please consider supporting it.

[aws-renfu]

Hi Robert,

Thanks for the question. We currently do not support onnxruntime nor loading of model.neff natively. Could you provide some background information of your use model (framework, execution environment, ...) so that we can provide better guidance for you?

--Randy

[RobertLucian]

@aws-renfu it would exactly be the model compiled in the link I've given you: #59 (comment). To us, the Resnet50 model is a nice model to present to our users. It is to you what it is to us as well: a demo model.

Does this make sense? Thanks!

[aws-renfu]

@RobertLucian,

I understand you want to demo the ResNet-50 model. Is your motivation to use ONNX because you do not want to use a framework like TensorFlow (to save system memory)?

Thanks,

--Randy

[RobertLucian]

@aws-renfu the motivation is that ONNX may just be yet another framework that our users might want to use. Just like Tensorflow and PyTorch. I'm not the end-user here.

Personally, if you ask me, I'm in favor of Keras, which is basically based on Tensorflow. That's because it's simple to use.

[aws-taylor]

Hello @RobertLucian,

We agree and will continue to monitor ONNX requests from our customers as we prioritize customer production workloads. I will go ahead and close this issue for now, feel free to re-open if needed.

Regards,
Taylor

[stancld]

Hi @aws-taylor, I'd be curious if there's any consideration about adding support for onnx even though there's no public plans in the roadmap.

I'd consider onnxruntime not only as another framework like torch/tensorflow, but a great way for inference unification of models trained in tensorflow and torch. A direct consequence is one can have docker images containing only very light onnxruntime, instead of both heavy tensorflow/torch, which brings several advantages.

Thanks a lot for your reply and for the work of AWS team on another great open-source tool. <3

[kurosch]

@aws-taylor I am with @stancld onnxruntime would be a great way of inference unification. Also how do you prefer customer requests to pop in, shall I contact our aws solution architect, account manager or just create a support request ticket?