Failed to fuse subgraph about aws-neuron-sdk

[micwade-aws]

Hi, thias42.

We’re going to take a look into this soon. Please double check that you’ve updated all of Neuron SDK and are running the latest ingredients. Instructions for updating Neuron are available here: https://github.com/aws/aws-neuron-sdk/blob/master/docs/neuron-install-guide.md

[thias42]

We’re going to take a look into this soon. Please double check that you’ve updated all of Neuron SDK and are running the latest ingredients. Instructions for updating Neuron are available here: https://github.com/aws/aws-neuron-sdk/blob/master/docs/neuron-install-guide.md

Yes, the Neuron SDK and tools are up-to-date.

[jeffhataws]

Thanks thias42. We have reproduced your issue and are investigating it.

[jeffhataws]

Currently compiler is unable to compile certain configurations due to tensor size limitations. We are aware of the limitations of the compiler and working to remove those limitations. For more information, please see release notes at https://github.com/aws/aws-neuron-sdk/tree/master/release-notes. For now, if you change the input representation (input_repr) to "mel128", and use compile option "no_fuse_ops=['melspectrogram/mul']" you will be able to get 99 operations out of 151 (inference optimized) operations to be placed on Neuron runtime.

model = openl3.models.load_audio_embedding_model(input_repr="mel128", content_type="music", embedding_size=512)
tf.saved_model.simple_save(
    session = keras.backend.get_session(),
    export_dir = model_dir,
    inputs = {'input': model.inputs[0]},
    outputs = {'output': model.outputs[0]})
tfn.saved_model.compile(model_dir, compiled_model_dir, no_fuse_ops=["melspectrogram/mul"])

You will see that the framework partitions the graph into 3 subgraph, but only one subgraph is compiled successfully (the portion after melspectrogram/mul):

INFO:tensorflow:fusing subgraph neuron_op_d4374713206b859f with neuron-cc
INFO:tensorflow:fusing subgraph neuron_op_2fdb7316ce04901d with neuron-cc
INFO:tensorflow:fusing subgraph neuron_op_bc8fe16a32d9e22a with neuron-cc
WARNING:tensorflow:Failed to fuse subgraph neuron_op_2fdb7316ce04901d with '/home/ubuntu/test_venv/bin/neuron-cc compile /tmp/tmp63uejh3m/neuron_op_2fdb7316ce04901d/graph_def.pb --framework TENSORFLOW --pipeline compile SaveTemps --output /tmp/tmp63uejh3m/neuron_op_2fdb7316ce04901d/graph_def.neff --io-config "{\"inputs\": {\"melspectrogram/transpose_20/_1:0\": [[1, 1, 199, 1025], \"float32\"], \"melspectrogram/unstack0/_2:0\": [[], \"int32\"]}, \"outputs\": [\"melspectrogram/Log:0\"]}"'
WARNING:tensorflow:Failed to fuse subgraph neuron_op_bc8fe16a32d9e22a with '/home/ubuntu/test_venv/bin/neuron-cc compile /tmp/tmp63uejh3m/neuron_op_bc8fe16a32d9e22a/graph_def.pb --framework TENSORFLOW --pipeline compile SaveTemps --output /tmp/tmp63uejh3m/neuron_op_bc8fe16a32d9e22a/graph_def.neff --io-config "{\"inputs\": {\"input_10/_3:0\": [[1, 1, 48000], \"float32\"]}, \"outputs\": [\"melspectrogram/transpose_2:0\", \"melspectrogram/Shape:0\"]}"'
INFO:tensorflow:Number of operations in TensorFlow session: 790
INFO:tensorflow:Number of operations after tf.neuron optimizations: 151
INFO:tensorflow:Number of operations placed on Neuron runtime: 99

Please let us know if further assistance is needed. You can also file AWS support ticket or contact us directly at [email protected].

[thias42]

Thank you! I'll stick to mel128 for now. Looking forward to a version without the tensor size limitations.