Demo for YoloV8 about norfair

[DiegoFernandezC]

Hello @kenifxyz, a YOLOv8 demo would indeed be beneficial. For now, I'm providing you with code to make Norfair works for your model.

As you pointed out, it's necessary to modify the function that transforms YOLO detections into Norfair detections. The code below adjusts this function to accommodate the new model.

import logging
from typing import List

import numpy as np
import torch
from ultralytics import YOLO

from norfair import Detection, Tracker, Video, draw_boxes

# Disable logging from the ultralytics package
logger_name = "ultralytics"
logging.getLogger(logger_name).setLevel(logging.CRITICAL)


def yolov8_detections_to_norfair_detections(
    yolo_detections: torch.tensor, track_points: str = "bbox"  # bbox or centroid
) -> List[Detection]:
    """convert yolo_detections to Norfair's detections"""
    norfair_detections: List[Detection] = []

    if track_points == "centroid":
        for object in yolo_detections:
            detections_as_xywh = object.boxes.xywh
            scores = object.boxes.conf
            classes = object.boxes.cls
            for detection_as_xywh, score, class_id in zip(detections_as_xywh, scores, classes):
                centroid = np.array([detection_as_xywh[0].item(), detection_as_xywh[1].item()])
                scores = np.array([score.item()])
                norfair_detections.append(
                    Detection(points=centroid, scores=scores, label=int(class_id))
                )
    elif track_points == "bbox":
        for object in yolo_detections:
            detections_as_xyxy = object.boxes.xyxy
            scores = object.boxes.conf
            classes = object.boxes.cls
            for detection_as_xyxy, score, class_id in zip(detections_as_xyxy, scores, classes):
                bbox = np.array(
                    [
                        [detection_as_xyxy[0].item(), detection_as_xyxy[1].item()],
                        [detection_as_xyxy[2].item(), detection_as_xyxy[3].item()],
                    ]
                )
                scores = np.array([score, score])
                norfair_detections.append(
                    Detection(points=bbox, scores=scores, label=int(class_id))
                )

    return norfair_detections


video = Video(input_path=<video-path>)

tracker = Tracker(distance_function="iou", distance_threshold=0.7)

model = YOLO(<model-path>)


for frame in video:
    yolo_detections = model(frame)
    detections = yolov8_detections_to_norfair_detections(yolo_detections)
    tracked_objects = tracker.update(detections=detections)
    frame = draw_boxes(frame, tracked_objects)
    video.write(frame)

If you have any further questions, please don't hesitate to ask.

[kenifxyz]

Hey @DiegoFernandezC, that works perfectly! Thanks so much!

[AbinashSankaran]

This works great. Able to do with yolov8. But if in case you come up with this error, "Can’t convert CUDA tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first", you can solve this by adding this before line 44,
score = score.cpu().numpy()

[delomeo]

If I may add something useful to this discussion, I found this snippet of code super handy but kind of sluggish. Therefore, I optimized a bit the function yolov8_detections_to_norfair_detections with a few (tiny) changes. Just for reference, it went from around 2fps rate to more than 5fps. Again thanks for your support @DiegoFernandezC and hope someone else will find this code helpful.

def yolov8_detections_to_norfair_detections(
    yolo_detections: torch.tensor, track_points: str = "bbox"  # bbox or centroid
) -> List[Detection]:
    """convert yolo_detections to Norfair's detections"""
    norfair_detections: List[Detection] = []

    if track_points == "centroid":
        for object in yolo_detections:
            detections_as_xywh = object.boxes.xywh
            scores = object.boxes.conf
            classes = object.boxes.cls
            norfair_detections += [
                Detection(
                    points=np.array([detection_as_xywh[0].item(), detection_as_xywh[1].item()]),
                    scores=np.array([score.item()]),
                    label=int(class_id)
                )
                for detection_as_xywh, score, class_id in zip(detections_as_xywh, scores, classes)
            ]

    elif track_points == "bbox":
        for object in yolo_detections:
            detections_as_xyxy = object.boxes.xyxy
            scores = object.boxes.conf
            classes = object.boxes.cls
            norfair_detections += [
                Detection(
                    points=np.array([
                        [detection_as_xyxy[0].item(), detection_as_xyxy[1].item()],
                        [detection_as_xyxy[2].item(), detection_as_xyxy[3].item()],
                    ]),
                    scores=np.array([score, score]),
                    label=int(class_id)
                )
                for detection_as_xyxy, score, class_id in zip(detections_as_xyxy, scores, classes)
            ]

    return norfair_detections