Comments (5)
Hi @jinfagang ,
Thanks for your attention here!
At that time, my first plan is to let each side could compile and infer successfully, and because of time limited, I have not carefully compared the time of each side between libtorch
, onnxruntime
and tvm
. (I have the plan to support openvino
, also for the time limited, I haven't done this now).
Now, I'm writing and refactoring the trainer
, after this is done, I will add the documents of time consuming and comparison.
BTW, we are welcome for contributions in any ways.
from yolort.
And I've uploaded the notebooks of tvm
compiling and inference here, within my current experience, the procedure of compiling will consume much more time than libtorch
or onnxruntime
. the procedure of inference seems normal (Edit: more experiments needed here)
This repo is my first glance on tvm
, and I will do more experiments on it.
from yolort.
@zhiqwang That's weried, tvm should be faster if compares on same CPU device? at least should faster than onnxruntime if chosen CPU as provider.
from yolort.
tvm should be faster if compares on same CPU device? at least should faster than onnxruntime if chosen CPU as provider.
Hi @jinfagang , I agree with you on this point, and this is my goal. The current realization on the tvm
backend is only the initial attempt, let us do more efforts to achieve this goal!
from yolort.
Hi @jinfagang
I've added a rough comparison of inference time consumed on Jupyter notebook (iPython).
-
On the ONNXRuntime backend,
CPU times: user 2.04 s, sys: 0 ns, total: 2.04 s Wall time: 55.8 ms
-
On the TorchScript backend,
CPU times: user 2.03 s, sys: 32 ms, total: 2.06 s Wall time: 60.5 ms
-
On the PyTorch backend,
CPU times: user 3.87 s, sys: 60 ms, total: 3.93 s Wall time: 116 ms
-
On the TVM backend,
CPU times: user 528 ms, sys: 364 ms, total: 892 ms Wall time: 22.3 ms
You could check the latest updated notebook for more details.
BTW, the displayed time of the onnxruntime notebook is on GPU, I just test it locally.
Although this comparison is a bit rough, we could come to this conclusion that tvm
will increase inference speed on CPU device.
So I'll close this issue. If you have more concerns please let me know.
from yolort.
Related Issues (20)
- `numpy` does not support newline delimiter from version 1.23
- zlibwapi.dll (solved)
- No module named 'yolort.utils.update_module_state' while saving the Yolo Model HOT 6
- Dynamic batch dimension not working with ONNX export HOT 1
- module 'yolort' has no attribute 'utils' HOT 4
- Can't load custom trained model HOT 2
- Unexpected side effect on matplotlib's backend HOT 2
- Remove `NestedTensor` from pre-processing
- Loading pre-trained model is not supported for num_classes != 80 HOT 1
- Can bbox coordinates be negative in yolo output? HOT 6
- Remove `ComputeLoss` from TorchScript graph
- SpeedUp with microsoft/nni HOT 3
- Can not export to ONNX model. AttributeError: 'NoneType' object has no attribute 'shape' HOT 7
- CLI tool for exporting models.: error: the following arguments are required: --checkpoint_path HOT 16
- Is it correct to subtract x_offset twice when performing bbox scale as post-processing? HOT 1
- If put yolov5 onnx exported from ultralytics into export_engine api, the postprocess speed slows down in cpp deploy. HOT 8
- SetCriterion's forward() incompatible with P6 models. Can't train P6 models.
- [defaultAllocator.cpp::deallocate::42] Error Code 1: Cuda Runtime (invalid argument) Segmentation fault (core dumped)
- Exporting ONNX Model with Fixed Batch Size of 1 Using export_tensorrt_engine
- cant batch infer?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yolort.