Comments (7)
The image read uses about 10ms, so the inference still uses 30ms for INT8 engine.
from tensorrt-for-yolo-series.
The image read uses about 10ms, so the inference still uses 30ms for INT8 engine.
which code ?
I think the infernce speed not related with engine which build by python script or trtexec tool.
from tensorrt-for-yolo-series.
The image read uses about 10ms, so the inference still uses 30ms for INT8 engine.
which code ?
I think the infernce speed not related with engine which build python script or trtexec tool.
Sorry, I didn't express clearly. I didn't mean the difference between python script and trtexec tool. I mean I use trtexec to get the INT8 inference perfomance like this which is 3x faster than your code.
I use the code in cpp/end2end.
from tensorrt-for-yolo-series.
The image read uses about 10ms, so the inference still uses 30ms for INT8 engine.
which code ?
I think the infernce speed not related with engine which build python script or trtexec tool.Sorry, I didn't express clearly. I don't mean the difference between python script and trtexec tool. I mean I use trtexec to get the inference perfomance like this.
I use the code in cpp/end2end.
int main(int argc, char** argv) {
if (argc == 5 && std::string(argv[1]) == "-model_path" && std::string(argv[3]) == "-image_path") {
char* model_path = argv[2];
char* image_path = argv[4];
float* Boxes = new float[4000];
int* BboxNum = new int[1];
int* ClassIndexs = new int[1000];
Yolo yolo(model_path);
clock_t startTime, endTime;
int num = 0;
double total_time = 0;
cv::Mat img;
// warmup
while (num != 1000) {
startTime = clock();
img = cv::imread(image_path);
yolo.Infer(img.cols, img.rows, img.channels(), img.data, Boxes, ClassIndexs, BboxNum);
endTime = clock();
double cur_timae = (double)(endTime - startTime) / CLOCKS_PER_SEC;
total_time += cur_timae;
num += 1;
}
// run inference
auto start = std::chrono::system_clock::now();
yolo.Infer(img.cols, img.rows, img.channels(), img.data, Boxes, ClassIndexs, BboxNum);
auto end = std::chrono::system_clock::now();
std::cout << std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count() << "ms" << std::endl;
// cout << "The run time is:" << total_time / 1000 << "s" << endl;
yolo.draw_objects(img, Boxes, ClassIndexs, BboxNum);
} else {
std::cerr << "--> arguments not right!" << std::endl;
std::cerr << "--> yolo -model_path ./output.trt -image_path ./demo.jpg" << std::endl;
return -1;
}
}
you can use the code
I'm guessing it's just the warmup that affects the speed
from tensorrt-for-yolo-series.
Oh, thank you! I forgot using warmup. I will test it tomorrow.
from tensorrt-for-yolo-series.
Oh, thank you! I forgot using warmup. I will test it tomorrow.
This might look more simply, my test is correct
int main(int argc, char** argv) {
if (argc == 5 && std::string(argv[1]) == "-model_path" && std::string(argv[3]) == "-image_path") {
char* model_path = argv[2];
char* image_path = argv[4];
float* Boxes = new float[4000];
int* BboxNum = new int[1];
int* ClassIndexs = new int[1000];
Yolo yolo(model_path);
cv::Mat img;
img = cv::imread(image_path);
// warmup
for (int num =0; num < 10; num++) {
yolo.Infer(img.cols, img.rows, img.channels(), img.data, Boxes, ClassIndexs, BboxNum);
}
// run inference
auto start = std::chrono::system_clock::now();
yolo.Infer(img.cols, img.rows, img.channels(), img.data, Boxes, ClassIndexs, BboxNum);
auto end = std::chrono::system_clock::now();
std::cout << std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count() << "ms" << std::endl;
// cout << "The run time is:" << total_time / 1000 << "s" << endl;
yolo.draw_objects(img, Boxes, ClassIndexs, BboxNum);
} else {
std::cerr << "--> arguments not right!" << std::endl;
std::cerr << "--> yolo -model_path ./output.trt -image_path ./demo.jpg" << std::endl;
return -1;
}
}
from tensorrt-for-yolo-series.
int main(int argc, char** argv) {
if (argc == 5 && std::string(argv[1]) == "-model_path" && std::string(argv[3]) == "-image_path") {
char* model_path = argv[2];
char* image_path = argv[4];
float* Boxes = new float[4000];
int* BboxNum = new int[1];
int* ClassIndexs = new int[1000];
Yolo yolo(model_path);
cv::Mat img;
img = cv::imread(image_path);
// warmup
for (int num =0; num < 10; num++) {
yolo.Infer(img.cols, img.rows, img.channels(), img.data, Boxes, ClassIndexs, BboxNum);
}
// run inference
auto start = std::chrono::system_clock::now();
yolo.Infer(img.cols, img.rows, img.channels(), img.data, Boxes, ClassIndexs, BboxNum);
auto end = std::chrono::system_clock::now();
std::cout << std::chrono::duration_caststd::chrono::milliseconds(end - start).count() << "ms" << std::endl;// cout << "The run time is:" << total_time / 1000 << "s" << endl; yolo.draw_objects(img, Boxes, ClassIndexs, BboxNum);
} else {
std::cerr << "--> arguments not right!" << std::endl;
std::cerr << "--> yolo -model_path ./output.trt -image_path ./demo.jpg" << std::endl;
return -1;
}
}
It works well! Thank you!
from tensorrt-for-yolo-series.
Related Issues (20)
- auto in_dims = engine->getBindingDimensions(engine->getBindingIndex("image_arrays")); HOT 1
- En715 Jetson xaiver Nx Yolov7.trt Not detect HOT 2
- yolov7,official,int8,onnx-> trt报错 HOT 3
- c++ endtoend 关于预测的置信度绘制 HOT 4
- memory leak: Destroy function does not work
- Detection duplicates with fp16 on Jetson Nano (TensorRT v8.2.1.8) HOT 2
- Support for windows?
- License? HOT 4
- 关于V8 tensorrt 出现乱框的情况 HOT 33
- TensorRT Conversion Issue "TypeError: pybind11::init(): factory function returned nullptr" HOT 2
- yolox 自己训练的模型 trt推理 位置不对 HOT 1
- int8量化的时候,输入是多个,怎么修改呢? calib_shape = [calib_batch_size] + list(inputs[0].shape[1:])不对吧 HOT 4
- 怎么将这个项目与RealSense深度相机结合起来? HOT 3
- yolov9能支持么? HOT 1
- Jetson nano yolov8部署 onnx ->> trt HOT 2
- Is the TRT conversion I used in YOLOV7 not available in YOLOV9?
- Can I use this code for exporting the RTDETR Model
- 有v8obb的部分嘛
- calibration数据集制作 HOT 1
- yolox和yolov7的官方代码似乎已经做了nms onnx推理 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tensorrt-for-yolo-series.