Comments (10)
from yolov8.
I'm planning to try using SkiaSharp (which is said to have better image processing performance) or a dll exported from C++/Rust for image processing in the future. If there's a significant performance improvement, can I submit a pull request?
from yolov8.
from yolov8.
I tried SkiaSharp, but there was no significant improvement in image processing performance. So, based on my previous experience with System.Drawing.Bitmap
, I attempted to change the method ImageSharp uses to traverse images.
The current optimizations include:
- Using a single layer of
Parallel.For
.
InDetectionOutputParser.Parse()
from line 34 to line 64:
Parallel.For(0, output.Dimensions[2], i =>
{
for (int j = 0; j < metadata.Classes.Count; j++)
{
var confidence = output[0, j + 4, i];
if (confidence <= parameters.Confidence)
continue;
// same as the original code ...
}
});
- Using
image.DangerousTryGetSinglePixelMemory(out Memory<TPixel> memory)
instead ofimage[x,y]
.
InImageSharpExtensions.ForEachPixel()
:
var width = image.Width;
var height = image.Height;
var totalPixels = width * height;
var flag = image.DangerousTryGetSinglePixelMemory(out Memory<TPixel> memory);
Parallel.For(0, totalPixels, index =>
{
int x = index % width;
int y = index / width;
var point = new Point(x, y);
var pixel = memory.Span[index]; // This line of code triggers an OutOfIndex error when processing certain images. It seems to be related to image size conversion, but it works normally for most images.
action(point, pixel);
});
- Detect - GPU
Image origin size: 2448x2048
Image count: 19
imgsz: 640x640
Preprocess: 0.0879 s ===> 0.03151 s
Inference: 0.06722 s ===> 0.05268 s // I don't know why it's getting faster
Postprocess: 0.34554 s ===> 0.00428 s
The processing time is taken as the average.
These optimizations perform well in the detect task, but I'm still not satisfied with the processing speed in segment. Primarily the SegmentationOutputParser.ProcessMask
is still too slow. But Preprocess time and Postprocess time have indeed become faster (similar changes have already been made to DetectionOutputParser.Parse()
).
Therefore, I'm planning to move all the image processing code to C++ in my project.
from yolov8.
The documentation for the DangerousTryGetSinglePixelMemory
function says that there can be a memory corruption while accessing the Span
, this may be related to the error you are getting, maybe ProcessPixelRows
will give a better result.
The bottleneck in deciphering the segmentation results is increasing the masks from 120x120 to the original size, to do that you need an interpolation algorithm to increase the pixels, I'm currently doing it using ImageSharp and that's what slows down the process, I can give up this increase but then it creates pixelated masks, so I chose to get good masks in exchange for a little longer time in postprocess.
Would you be willing to submit a PR with the fixes you've made so far?
Thanks for your work!
from yolov8.
I've submitted the PR.
In Image.ForEachPixel()
, I finally used the following approach to avoid exceptions. It works for most images. If the flag is false, the mask will not be drawn on the result image.
var flag = image.DangerousTryGetSinglePixelMemory(out Memory<TPixel> memory);
if (flag)
{
Parallel.For(0, totalPixels, index =>
{
int x = index % width;
int y = index / width;
var point = new Point(x, y);
var pixel = memory.Span[index];
action(point, pixel);
});
}
I'm still looking for the reason why flag is set to false.
from yolov8.
@FunJoo Your PR is merged, can you confirm that it works without errors?
from yolov8.
I've verified it. Both the demo weights and dataset tests, as well as my custom weights and dataset tests, can pass successfully.
from yolov8.
@FunJoo I fixed some things in segmentation postprocess, can you check the performance of it now?
from yolov8.
Amazing! The postprocess is much faster now than before. Thank you, I think it's ready to be implemented in my project.
- Detect
Image origin size: 2448x2048
imgsz: 640x640
Average Postprocess Time: 0.00428 s ==> 0.00102 s
- Segment
Image origin size: 3840x2748
imgsz: 800x800
Average Postprocess Time: 1.18653 s ==> 0.25893 s
from yolov8.
Related Issues (20)
- Understanding Yolov8 Task Input and Output Formats in OnnxRuntime HOT 1
- Difference between Python and DotNet HOT 1
- Is there a best example of multithreading or WebApi? HOT 1
- How to enable GPU HOT 3
- Inconsistent model results HOT 2
- Suggest adding OBB support HOT 2
- [discussion] optimize code logic to improve operational efficiency HOT 2
- Index was outside the bounds of the array. HOT 5
- Logic for resizing images to fit the size of the model input data. HOT 4
- Inference with Camera HOT 2
- It's not really using the GPU
- Getting lower confidence values HOT 2
- Load Image HOT 2
- Possible to select between fp16 and fp32? HOT 1
- Error Sequence contains no elements on image plotting HOT 3
- some pictures does not show transparent boxes HOT 1
- Произошла ошибка в обработке клиента onnxruntime_providers_cuda.dll мой проект пробовал на GPU с библиотекой YoloV8.Gpu v4.0.0 HOT 15
- DetectObb is not showing any bounding boxes for object (car in my case) HOT 6
- Processing video files HOT 1
- can't set configuration for create a predictor HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yolov8.