chufei1995 / tiny-tensorrt Goto Github PK

This project forked from zerollzeng/tiny-tensorrt

a simple, efficient, easy-to-use nvidia TensorRT wrapper for cnn,sopport c++ and python

CMake 1.16% C++ 92.62% Cuda 4.84% C 1.26% Python 0.12%

tiny-tensorrt's Introduction

tiny-tensorrt

a simple, efficient, easy-to-use nvidia TensorRT wrapper for cnn with c++ and python api,support caffe, uff and onnx format models. you will be able use tiny-tensorrt deploy your model with few lines of code!

// create engine
trt.CreateEngine(onnxModelpath, engineFile, customOutput, maxBatchSize, mode, calibratorData);
// transfer you input data to tensorrt engine
trt.DataTransfer(input,0,True);
// inference!!!
trt.Forward();
//  retrieve network output
trt.DataTransfer(output, outputIndex, False) // you can get outputIndex in CreateEngine phase

Features

System Requirements

cuda 10.0+

TensorRT 6 or 7

for python api, python 2.x/3.x and numpy in needed

Installation

Make sure you had install dependencies list above, if you are familiar with docker, you can use official docker

# clone project and submodule
git clone --recurse-submodules -j8 https://github.com/zerollzeng/tiny-tensorrt.git

cd tiny-tensorrt

mkdir build && cd build && cmake .. && make

then you can intergrate it into your own project with libtinytrt.so and Trt.h, for python module, you get pytrt.so

Docs

User Guide

Custom Plugin Tutorial (En-Ch)

if you want some examples with tiny-tensorrt, you can refer to tensorrt-zoo

Extra Support layer

upsample with custom scale, under test with yolov3.
yolo-det, last layer of yolov3 which sum three scales output and generate final result for nms. under test with yolov3.
PRELU, under test with openpose and mtcnn.