onnx 1.16.1
onnx-graphsurgeon 0.3.12
onnxruntime-gpu 1.18.1
onnxsim 0.4.36
Python 3.10.9
cuda 11.7
================ Diagnostic Run torch.onnx.export version 2.0.0 ================
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
WARNING: failed to run "SequenceEmpty" op (name is "/transformer/SequenceEmpty"), skip...
WARNING: failed to run "SequenceEmpty" op (name is "/transformer/SequenceEmpty"), skip...
Traceback (most recent call last):
File "/workspace/Salience-DETR/tools/pytorch2onnx.py", line 133, in <module>
pytorch2onnx()
File "/workspace/Salience-DETR/tools/pytorch2onnx.py", line 108, in pytorch2onnx
model_ops, check_ok = onnxsim.simplify(args.save_file)
File "/opt/conda/lib/python3.10/site-packages/onnxsim/onnx_simplifier.py", line 199, in simplify
model_opt_bytes = C.simplify(
onnx.onnx_cpp2py_export.checker.ValidationError: Nodes in a graph must be topologically sorted, however input '/transformer/SequenceEmpty_output_0' of node:
name: /transformer/Loop OpType: Loop
is not output of any previous nodes.
================ Diagnostic Run torch.onnx.export version 2.0.0 ================
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
Successfully exported ONNX model: torchconverted.onnx
2024-07-22 18:06:28.479503543 [W:onnxruntime:, transformer_memcpy.cc:74 ApplyImpl] 328 Memcpy nodes are added to the graph torch_jit for CUDAExecutionProvider. It might have negative impact on performance (including unable to run CUDA graph). Set session_options.log_severity_level=1 to see the detail logs before this message.
2024-07-22 18:06:28.481116649 [W:onnxruntime:, transformer_memcpy.cc:74 ApplyImpl] 1 Memcpy nodes are added to the graph torch_jit13 for CUDAExecutionProvider. It might have negative impact on performance (including unable to run CUDA graph). Set session_options.log_severity_level=1 to see the detail logs before this message.
2024-07-22 18:06:31.742070878 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-07-22 18:06:31.746102699 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2024-07-22 18:06:31.746114767 [W:onnxruntime:Default, scatter_nd.h:51 ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
Traceback (most recent call last):
File "/workspace/Salience-DETR/tools/pytorch2onnx.py", line 133, in <module>
pytorch2onnx()
File "/workspace/Salience-DETR/tools/pytorch2onnx.py", line 128, in pytorch2onnx
np.testing.assert_allclose(onnx_res, pytorch_res, rtol=1e-3, atol=1e-5, err_msg=err_msg)
File "/opt/conda/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 1592, in assert_allclose
assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
File "/opt/conda/lib/python3.10/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/opt/conda/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 862, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Not equal to tolerance rtol=0.001, atol=1e-05
The numerical values are different between Pytorch and ONNXBut it does not necessarily mean the exported ONNX is problematic.
Mismatched elements: 19 / 300 (6.33%)
Max absolute difference: 3.3614924e-05
Max relative difference: 0.01449186
x: array([0.002411, 0.002407, 0.0024 , 0.002384, 0.002363, 0.002363,
0.002358, 0.002353, 0.002345, 0.002333, 0.002331, 0.002331,
0.002323, 0.00232 , 0.002318, 0.002309, 0.002307, 0.0023 ,...
y: array([0.002411, 0.00241 , 0.002398, 0.00239 , 0.002369, 0.002367,
0.002363, 0.00235 , 0.002349, 0.002349, 0.002346, 0.002336,
0.002325, 0.002325, 0.002323, 0.002322, 0.002321, 0.00232 ,...
[07/22/2024-18:07:37] [E] Error[4]: /eval_transform/eval_transform.0/If_OutputLayer: IIfConditionalOutputLayer inputs must have the same shape. Shapes are [3,-1,-1] and [1,3,-1,-1].
[07/22/2024-18:07:37] [E] [TRT] ModelImporter.cpp:771: While parsing node number 113 [If -> "/eval_transform/eval_transform.0/If_output_0"]:
[07/22/2024-18:07:37] [E] [TRT] ModelImporter.cpp:772: --- Begin node ---
[07/22/2024-18:07:37] [E] [TRT] ModelImporter.cpp:773: input: "/eval_transform/eval_transform.0/Equal_output_0"
output: "/eval_transform/eval_transform.0/If_output_0"
name: "/eval_transform/eval_transform.0/If"
op_type: "If"
attribute {
name: "then_branch"
g {
node {
output: "/eval_transform/eval_transform.0/Constant_15_output_0"
name: "/eval_transform/eval_transform.0/Constant_15"
op_type: "Constant"
attribute {
name: "value"
t {
dims: 1
data_type: 7
raw_data: "\000\000\000\000\000\000\000\000"
}
type: TENSOR
}
}
node {
input: "/eval_transform/eval_transform.0/Resize_output_0"
input: "/eval_transform/eval_transform.0/Constant_15_output_0"
output: "/eval_transform/eval_transform.0/Squeeze_2_output_0"
name: "/eval_transform/eval_transform.0/Squeeze_2"
op_type: "Squeeze"
}
name: "torch_jit1"
output {
name: "/eval_transform/eval_transform.0/Squeeze_2_output_0"
type {
tensor_type {
elem_type: 1
shape {
dim {
dim_param: "Squeeze/eval_transform/eval_transform.0/Squeeze_2_output_0_dim_0"
}
dim {
dim_param: "Squeeze/eval_transform/eval_transform.0/Squeeze_2_output_0_dim_1"
}
dim {
dim_param: "Squeeze/eval_transform/eval_transform.0/Squeeze_2_output_0_dim_2"
}
}
}
}
}
}
type: GRAPH
}
attribute {
name: "else_branch"
g {
node {
input: "/eval_transform/eval_transform.0/Resize_output_0"
output: "/eval_transform/eval_transform.0/Identity_output_0"
name: "/eval_transform/eval_transform.0/Identity"
op_type: "Identity"
}
name: "torch_jit2"
output {
name: "/eval_transform/eval_transform.0/Identity_output_0"
type {
tensor_type {
elem_type: 1
shape {
dim {
dim_param: "Identity/eval_transform/eval_transform.0/Identity_output_0_dim_0"
}
dim {
dim_param: "Squeeze/eval_transform/eval_transform.0/Squeeze_2_output_0_dim_0"
}
dim {
dim_param: "Squeeze/eval_transform/eval_transform.0/Squeeze_2_output_0_dim_1"
}
dim {
dim_param: "Squeeze/eval_transform/eval_transform.0/Squeeze_2_output_0_dim_2"
}
}
}
}
}
}
type: GRAPH
}
[07/22/2024-18:07:37] [E] [TRT] ModelImporter.cpp:774: --- End node ---
[07/22/2024-18:07:37] [E] [TRT] ModelImporter.cpp:777: ERROR: ModelImporter.cpp:195 In function parseGraph:
[6] Invalid Node - /eval_transform/eval_transform.0/If
/eval_transform/eval_transform.0/If_OutputLayer: IIfConditionalOutputLayer inputs must have the same shape. Shapes are [3,-1,-1] and [1,3,-1,-1].
[07/22/2024-18:07:37] [E] Failed to parse onnx file
[07/22/2024-18:07:37] [I] Finished parsing network model. Parse time: 1.04033
[07/22/2024-18:07:37] [E] Parsing model failed
[07/22/2024-18:07:37] [E] Failed to create engine from model or file.
[07/22/2024-18:07:37] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8601] # trtexec --onnx=torchconverted.onnx
[07/22/2024-18:24:34] [W] [TRT] onnx2trt_utils.cpp:400: One or more weights outside the range of INT32 was clamped
[07/22/2024-18:24:35] [E] Error[4]: [graph.cpp::symbolicExecute::539] Error Code 4: Internal Error ((Unnamed Layer* 10573) [LoopOutput]: an ILoopOutputLayer cannot be used to compute a shape tensor)
[07/22/2024-18:24:35] [E] [TRT] ModelImporter.cpp:771: While parsing node number 4882 [Slice -> "/transformer/Slice_21_output_0"]:
[07/22/2024-18:24:35] [E] [TRT] ModelImporter.cpp:772: --- Begin node ---
[07/22/2024-18:24:35] [E] [TRT] ModelImporter.cpp:773: input: "/transformer/enc_output_norm/LayerNormalization_output_0"
input: "/transformer/Unsqueeze_68_output_0"
input: "/transformer/Constant_289_output_0"
input: "/transformer/Constant_290_output_0"
input: "/transformer/Constant_291_output_0"
output: "/transformer/Slice_21_output_0"
name: "/transformer/Slice_21"
op_type: "Slice"
[07/22/2024-18:24:35] [E] [TRT] ModelImporter.cpp:774: --- End node ---
[07/22/2024-18:24:35] [E] [TRT] ModelImporter.cpp:777: ERROR: ModelImporter.cpp:195 In function parseGraph:
[6] Invalid Node - /transformer/Slice_21
[graph.cpp::symbolicExecute::539] Error Code 4: Internal Error ((Unnamed Layer* 10573) [LoopOutput]: an ILoopOutputLayer cannot be used to compute a shape tensor)
[07/22/2024-18:24:35] [E] Failed to parse onnx file
[07/22/2024-18:24:35] [I] Finished parsing network model. Parse time: 1.77338
[07/22/2024-18:24:35] [E] Parsing model failed
[07/22/2024-18:24:35] [E] Failed to create engine from model or file.
[07/22/2024-18:24:35] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8601] # trtexec --onnx=torchconverted.onnx