NEURON_PROFILE does not appear to work when running Reinvent lab3 about aws-neuron-sdk

[mrnikwaws]

Thanks for your question. You are correct that profiling does not work on this specific tutorial example. The example uses a feature which varies batch sizes dynamically, and is not compatible with profiling at the moment

[mrnikwaws]

On further investigation if you update the file infer_resnet50_keras_loadtest.py and modify the line:

USER_BATCH_SIZE = 50

to

USER_BATCH_SIZE = 5

then profiling should work for you. Currently the compilation batch size and the inference batch size must match for profiling to run correctly.

There is a lot of data in the load test, and tensorboard_neuron will take a long time to process all of it on startup. I recommend that you use a much smaller data set which will give you useful profiling information.

Consider changing:

NUM_LOOPS_PER_THREAD = 100

to

NUM_LOOPS_PER_THREAD = 5

[cprice404-aws]

Excellent, thanks for the information!

Do you expect for the profiling to be available for dynamic batch sizes in the future?

[mrnikwaws]

Thanks for the input. I have made the team aware of the request, we've documented it and we will consider adding it in a future release. For now I am going to close this issue (since you can run the profiler), but please reopen it if you feel that this specific feature is a priority to track in github.