UUF Convertor about tensorrt_demos

[jkjung-avt]

Please read TensorRT documentation: https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#api

Only Caffe, UFF and ONNX parsers are available in current releases of TensorRT.

[jkjung-avt]

Regarding your question 3 above,

• "bindings" refer to the inputs and outputs of a TensorRT engine
• For each binding, we allocate 1 "host" memory block and 1 "cuda" memory block. The 2 memory blocks are of the same size.
• The "host" memory blocks should be blocks of contiguous memory location belonging to the CPU.
• Lilewise, the "cuda" memory blocks belong to the GPU.
• To do inference, we first copy image data to the input "host" memory. This makes sure the data is contiguous and in the correct order. Code: https://github.com/jkjung-avt/tensorrt_demos/blob/master/utils/ssd.py#L99
• Then we ask GPU to copy the data from "host" to "cuda" memory. Code: https://github.com/jkjung-avt/tensorrt_demos/blob/master/utils/ssd.py#L101
• Next, we "execute" the Tensor engine.
• Finally, we copy the outputs from "cuda" back to "host" memory.
• Note that CUDA driver would make sure the memcpy's and engine execution are done sequentially (no race condition).