Cuda failure after loading TRT Engine about yolo-tensorrt

[enazoe]

how many images of inference process?

[marcelomizuki]

Yes, that was it! The batch size on the "yolov3.cfg" file was 1 image and the code was reading in 2 images by default.

I commented out one of the pushes for 1 of the images in the "sample_detector.cpp" file and it works well. Looks to be much
faster than the CuDNN implementation I tried with OpenCV.

batch_img.push_back(temp0);
// batch_img.push_back(temp1);

On a laptop with an RTX2070 Max Q for YoloV3
~22ms using FP32
~14ms using FP16

On a desktop with an RTX 2080Ti for YoloV3
~16.5ms using FP32
~10ms using FP16

Thanks for the hint and for the repository!

[enazoe]

ok, you should set the batch size by this,and the default value is 4 , you could set it by your own display memory.

[marcelomizuki]

Yes very good point, I will check what can be gained in the application by batching... Thanks!