Input size limitations and undocumented compiler args about aws-neuron-sdk

[mrnikwaws]

Thanks for the question. The docs here document a max size of 480x480 fp/bf16, with batch size=4. So the use case you are describing should work and be within the capacity of the hardware

Can you share more of your specific use case, or some example code? Information like which instance type you were using and whether you had any special configuration for the runtime will also help.

On compiler flags: I am following up with other members of the team. In some cases (particularly at launch) we wanted to show some features of the hardware not ready for general consumption. We want you to be able to fully leverage Inferentia, without the experience being confusing or messy! We also don't want to tease you with functionality you can't use, or won't be able to use soon ... This is a great question and I'll find out more.

In this particular case there should be no magic needed, so let's see if we can figure out what is going on.

[davidas1]

Thanks for the quick reply.

I'm using the inf1.6xlarge instance for compilation with dlami 26.0, and I didn't make any changes to the runtime (that I know of..).
I did update the packages like described in DLAMI with Neuron Release Notes

I'd love to share code for reproducing the issue, but the network architecture I used is confidential.
If possible, I can open a ticket with AWS support and attach the code there.

[aws-taylor]

Hello David,

If you could open an AWS support issue with the attached network we would be happy to investigate further.

Regards,
Taylor

[davidas1]

Thanks, I opened an issue with case ID 6733794191

…

On Wed, Jan 8, 2020 at 9:00 PM aws-taylor ***@***.***> wrote: Hello David, If you could open an AWS support issue with the attached network we would be happy to investigate further. Regards, Taylor

[aws-taylor]

Thanks David,

We will follow up on the support issue.

Regards,
Taylor