ssdlite_mobilenet_v2_coco about tensorrt_demos

[jkjung-avt]

I want to convert ssdlite_mobilenet_v2_coco with ./build_engines.sh, I get this error:

I spent some time to check this, and found out the problem lies in the "frozen_inference_graph.pb" in ssdlite_mobilenet_v2_coco_2018_05_09.tar.gz (missing "MultipleGridAnchorGenerator" nodes).

So I downloaded the model checkpoint and exported the pb file by myself. With this ssdlite_mobilenet_v2_coco.pb exported by myself, I was able to build and run a TensorRT engine with the code in this repo.

I've attached the pb file for reference, and I think this issue could finally be clsoed.

ssdlite_mobilenet_v2_coco.zip

[PythonImageDeveloper]

I solve the problem.
In the build_engines.py, I change the output_nodes=['NMS'], to output_nodes=['BoxPredictor_0/ClassPredictor_depthwise/Relu6'], the convert correctly is done.

[PythonImageDeveloper]

The model is correctly converted to .uff and .bin file, but when I run trt_ssy.py, I get this error:

  File "trt_ssd.py", line 106, in main
    loop_and_detect(cam, trt_ssd, conf_th=0.3, vis=vis)
  File "trt_ssd.py", line 67, in loop_and_detect
    boxes, confs, clss = trt_ssd.detect(img, conf_th)
  File "/home/jnano/tensorrt_demos/utils/ssd.py", line 109, in detect
    self.host_outputs[1], self.cuda_outputs[1], self.stream)
IndexError: list index out of range

[jkjung-avt]

Output of the TensorRT optimized SSD engine should be "NMS".

I think the error message means "dimension of the output tensor" is not as expected.

[PythonImageDeveloper]

The output of TensorRT ssd_mobilenet_v2_coco should be "NMS", but for ssdlite_mobilenet_v2_coco should be "BoxPredictor_0/ClassPredictor_depthwise/Relu6"
https://devtalk.nvidia.com/default/topic/1065353/python3-nmsplugin-cpp-54-virtual-nvinfer1-dims-nvinfer1-plugin-detectionoutput-getoutputdimensions-int-const-nvinfer1-dims-int-assertion-nbinputdims-3-failed-aborted-core-dumped-quot-this-error-is-coming-/?offset=4#5416566

for ssdlite_mobilenet_v2_coco:
when I comment the line 110 cuda.memcpy_dtoh_async(self.host_outputs[1], self.cuda_outputs[1], self.stream) and change line 30 to for prefix in range(0, len(output)-1, output_layout), in ssd.py
it works for ssdlite_mobilenet_v2_coco, but I get 0.6 FPS and incorrect The BBoxes. Why?

[jkjung-avt]

In that case, you'll have to modify postprocessing code. I don't think "NMS" and "BoxPredictor_0/ClassPredictor_depthwise/Relu6" would produce the same kind of outputs.

[zychen2016]

Have one solution?@PythonImageDeveloper

[PythonImageDeveloper]

No, Currently I work on ssd_resnet50_v1_fpn_640*640

[PythonImageDeveloper]

Hi @jkjung-avt
1-I can convert the ssdlite_mobilenet_v3 to .onnx model, In your opinion, How do I use .onnx model for parser to TensorRT in the ssd_trt.py?
2-In the parsing .onnx model to TensorRT, maybe happens error like unsupported layers , ...?
3-Can I use onnx_to_tensorrt.py in yolov3, for convert the ssdlite_mobilenet_v3.onnx to .bin file (tensorrt)?
why you convert the yolov3.onnx to .trt file? why you don't convert to .bin file likes in ssd?

[jkjung-avt]

1. You need to use ONNX parser and optimize/convert the model to a TensorRT engine. You could reference the code in yolov3_onnx/onnx_to_tensorrt.py. Once the TensorRT engine is created (assuming it produces the same NMS output), then you could use "trt_ssd.py" directly to do inferencing.

2. Replace unsupported layers with plugins, if possible.

3. The ".trt" is basically the same as ".bin". I followed the original NVIDIA sample code. The optimized TensorRT engines are saved to different file names in the 2 examples.

[PythonImageDeveloper]

How I can replace unsupported layers with plugins, have you reference for that?

[jkjung-avt]

I haven't done that with ONNX. I'd be interested to know if you manage to get it working.

[sangjoonhong]

Hello, @jkjung-avt
Please understand that I am not good at English.
I know you are TRT and TF experter than me.
Your github is helping me a lot. Thank you very much.
Sorry to ask you for this, If you can afford, plz can you help me?
If you don't want it, ignore.

I am TensorRT beginner. my goal is running my network in TRT.
But, I faced one issue like this article.
TensorRT_MobileNetV2: nmsPlugin.cpp:54: virtual nvinfer1::Dims nvinfer1::plugin::DetectionOutput::getOutputDimensions(int, const nvinfer1::Dims*, int): Assertion 'nbInputDims == 3' failed.

My network is customized ssd_mobilenet_v2.
Actually, I don't know exactly because it is not the network I made.

But, I know this network is for detecting head, hand box and some feature points (eyes and finger tip)
Something customizing is added for detecting feature points.
If you could take a look and give some hints, help me plz.
This is network file link

If you need more information, plz tell me.
Thank you.

[PythonImageDeveloper]

Hi @sangjoonhong ,
You used TFOD API for training?

[sangjoonhong]

@PythonImageDeveloper
That network is trained using tensorflow python.

[jkjung-avt]

@sangjoonhong I looked at your "ssd_mobilenet_v2_custom.pb". Its architecture is different from the original coco model. Here's a screenshot when I checked your custom model using TensorBoard.

There are at least 2 issues. This custom model has only 4 BoxPredictors (the original ssd design has 6). In addition, the input image_tensor is not set to a fixed shape (say, 300x300).

As a comparison, here's how the FeatureExtractor and BoxPredictors look like in the original ssd_mobilenet_v2_coco.

[sangjoonhong]

@jkjung-avt
Thanks for your reply.
I knew it already about that.
That is to make it more efficient for inference speed in our cases.
So, When I translate .pb to .uff I using this parameter in config.py numLayers=6 -> 4

PriorBox = gs.create_plugin_node( name="MultipleGridAnchorGenerator", numLayers=4, ...

My problem is why occure this error
Assertion nbInputDims == 3' failed.` in nmsPlugin.cpp, getOutputDimensions()

I'm still trying to solve the problem, so I'll let you know if any progress is made.
Thank you very much.

[MarsJacobs]

@sangjoonhong has anyone fixed nbInputDims == 3' failed?