Yolo8-OnnxRuntime-CPP-Inference awful output about ultralytics

[EjenY-Poltavchiny]

Thank you for help !!!

[github-actions]

👋 Hello @EjenY-Poltavchiny, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

Hi @EjenY-Poltavchiny,

Thank you for reaching out and providing detailed information about your setup and the issue you're encountering. Let's work through this together to get your YOLOv8 ONNXRuntime CPP inference running smoothly.

Steps to Troubleshoot

1. Minimum Reproducible Example:
   To better understand and reproduce the issue, could you please provide a minimal reproducible example of your code? This will help us investigate the problem more effectively. You can refer to our Minimum Reproducible Example Guide for more details on how to create one.

2. Verify Versions:
   Ensure you are using the latest versions of torch and ultralytics. You can update them using the following commands:

   pip install --upgrade torch ultralytics

3. Check ONNX Model:
   Verify that the ONNX model you are using is correctly exported and compatible with the ONNXRuntime. You can do this by running a simple inference in Python to ensure the model outputs are as expected.

4. ONNXRuntime and OpenCV Installation:
   Your installation steps for OpenCV and ONNXRuntime look comprehensive. However, ensure that all dependencies are correctly installed and there are no version mismatches. Sometimes, using a virtual environment can help isolate and manage dependencies better.

Example Code for Verification

Here’s a basic example to verify the ONNX model inference in Python before moving to C++:

import onnxruntime as ort
import numpy as np
import cv2

# Load the ONNX model
session = ort.InferenceSession("path/to/your/model.onnx")

# Prepare an input image
image = cv2.imread("path/to/your/image.jpg")
input_image = cv2.resize(image, (640, 640))  # Resize to model input size
input_image = input_image.transpose(2, 0, 1)  # Convert to CHW format
input_image = input_image[np.newaxis, :, :, :].astype(np.float32)  # Add batch dimension

# Run inference
outputs = session.run(None, {"images": input_image})

# Process outputs
print(outputs)

If the outputs are as expected in Python, the issue might be specific to the C++ implementation or environment setup.

Additional Resources

For further troubleshooting, you can refer to our Common Issues Guide, which covers a wide range of potential problems and their solutions.

Please provide the minimal reproducible example and let us know if updating the packages resolves the issue. We're here to help!

[EjenY-Poltavchiny]

Hi there!
Thanks for quick respond. With your script I've compared python and C++ outputs and understood the essence of the bug. The problem was connected with postprocessing. I should've use transpose to result matrix (in your code there was commended line):

rawData = rawData.t(); 

Also i had to change output cv::Mat structure (its shape). After all of that your yolov8n shows beautiful output. Like this:

Could you help me with another question please. Or at least point to some sort of exmaple/documentation.
Question: If i have model with several classes (like yours) but this classes have different confidenceThresholds. How should I do postprocessing?

[glenn-jocher]

@EjenY-Poltavchiny hi there!

I'm glad to hear that you resolved the issue with the postprocessing and that your yolov8n model is now producing beautiful outputs! 🎉

Regarding your new question about handling different confidence thresholds for various classes during postprocessing, you can achieve this by applying class-specific thresholds after obtaining the raw outputs from the model. Here's a concise example to illustrate this in C++:

Example Code for Class-Specific Confidence Thresholds

#include <opencv2/opencv.hpp>
#include <vector>

// Assuming `detections` is a vector of detected objects with their class IDs and confidence scores
std::vector<cv::Rect> applyClassSpecificThresholds(const std::vector<cv::Rect>& detections, const std::vector<int>& classIds, const std::vector<float>& confidences, const std::vector<float>& classThresholds) {
    std::vector<cv::Rect> filteredDetections;

    for (size_t i = 0; i < detections.size(); ++i) {
        int classId = classIds[i];
        float confidence = confidences[i];

        // Apply class-specific confidence threshold
        if (confidence >= classThresholds[classId]) {
            filteredDetections.push_back(detections[i]);
        }
    }

    return filteredDetections;
}

int main() {
    // Example usage
    std::vector<cv::Rect> detections;  // Your detections
    std::vector<int> classIds;         // Class IDs for each detection
    std::vector<float> confidences;    // Confidence scores for each detection
    std::vector<float> classThresholds = {0.5, 0.6, 0.7};  // Example thresholds for 3 classes

    std::vector<cv::Rect> filteredDetections = applyClassSpecificThresholds(detections, classIds, confidences, classThresholds);

    // Process filteredDetections as needed
    return 0;
}

In this example, classThresholds is a vector where each index corresponds to a class ID and its associated confidence threshold. The applyClassSpecificThresholds function filters the detections based on these thresholds.

For more detailed guidance, you can refer to the Ultralytics documentation and explore the sections on postprocessing and custom inference logic.

Feel free to reach out if you have any more questions. Happy coding! 😊

[glenn-jocher]

Hi @EjenY-Poltavchiny,

You're very welcome! 😊 I'm glad to hear that the adjustments to the postprocessing resolved your issue and that you're now seeing great results with your yolov8n model.

Handling Class-Specific Confidence Thresholds

To address your question about applying different confidence thresholds for various classes during postprocessing, here's a concise approach you can implement in C++:

#include <opencv2/opencv.hpp>
#include <vector>

// Function to apply class-specific confidence thresholds
std::vector<cv::Rect> applyClassSpecificThresholds(const std::vector<cv::Rect>& detections, const std::vector<int>& classIds, const std::vector<float>& confidences, const std::vector<float>& classThresholds) {
    std::vector<cv::Rect> filteredDetections;

    for (size_t i = 0; i < detections.size(); ++i) {
        int classId = classIds[i];
        float confidence = confidences[i];

        // Apply class-specific confidence threshold
        if (confidence >= classThresholds[classId]) {
            filteredDetections.push_back(detections[i]);
        }
    }

    return filteredDetections;
}

int main() {
    // Example usage
    std::vector<cv::Rect> detections;  // Your detections
    std::vector<int> classIds;         // Class IDs for each detection
    std::vector<float> confidences;    // Confidence scores for each detection
    std::vector<float> classThresholds = {0.5, 0.6, 0.7};  // Example thresholds for 3 classes

    std::vector<cv::Rect> filteredDetections = applyClassSpecificThresholds(detections, classIds, confidences, classThresholds);

    // Process filteredDetections as needed
    return 0;
}

This code snippet demonstrates how to filter detections based on class-specific confidence thresholds. Adjust the classThresholds vector to match the thresholds for your specific classes.

If you encounter any further issues or have additional questions, please don't hesitate to ask. We're here to help!