Darknet integration about norfair

[joaqo]

Hi @wjassim ! Currently the solution is to make one tracker for each class, it works fine but it could be better from a usability standpoint, and improving this is in our roadmap. Look here for more discussion on this: #47

[haviduck]

Hi, haviduck! Glad you got it sorted out, good luck!

thank you, its a stellar tracker. replaced my custom SORT now and its alot better with the additional params i can control. easiest heart transplant ive done - good job.

if i have one feature request and its purely cosmetics - trajectory when u hit draw_points.

[haviduck]

yeah, like dot history instead of a single centroid showing the 8 last points by circles and lines connecting them.

this is my approaches atm with SORT

def get_patterns(self, center, track_id, class_name):
        track_id_str = str(track_id)
        # This function stores all tracked objand their moving patterns
        if class_name not in ['obj1', '_obj2h']:
            return self.dict_tracks[class_name][track_id_str]
        # makin' tracks (if necessary)
        if track_id_str not in self.dict_tracks[class_name]:
            self.dict_tracks[class_name][track_id_str] = []
        # append
        self.dict_tracks[class_name][track_id_str].append(center)
        if len(self.dict_tracks[class_name][track_id_str]) > 60:
            del self.dict_tracks[class_name][track_id_str][0]

        return self.dict_tracks[class_name][track_id_str]

or this im working on now:

class Objname:
    def __init__(self):
        self.weight = 0
        self.health = 'Ok'
        self.track = list()

    def add_coord(self, coord):
        self.track.append(coord)
        # trim to size
        if len(self.track) > 600:
            del self.track[0]


class ObjTrajectoryTracker:
    def __init__(self):
        self.dict_tracks = {"classname1": {}, "classname2": {}}

    def set_trajectory(self, center, track_id, class_name):
        track_id_str = str(track_id)
        # This function stores all tracked obj and their moving patterns
        if class_name not in ['classname1', '_othernames']:
            return self.dict_tracks[class_name][track_id_str]
        # makin' tracks (if necessary)
        if track_id_str not in self.dict_tracks[class_name]:
            self.dict_tracks[class_name][track_id_str] = Objname()
        # append
        self.dict_tracks[class_name][track_id_str].add_coord(center)

        return self.dict_tracks[class_name][track_id_str].track

    def get_trajectory(self, track_id, class_name):
        track_id_str = str(track_id)
        return self.dict_tracks[class_name][track_id_str].track

pattern = self.my_tracker.set_trajectory(centroid, int(self.tracked_id), "Classname")

[joaqo]

Thats a great suggestion, and it meshes really well with the addition of a history of past detections and embeddings to each track, which is what we are working on now. I'll try to get it into our next release!

[haviduck]

i understand. im facing that with my custom SORT. Ill check if theres any performance dips using 3 trackers and if not ill switch over :) using it for counting atm so pretty essential.
looking forward to next release

[haviduck]

memleak was not related.

[joaqo]

Hi, haviduck! Glad you got it sorted out, good luck!

[joaqo]

Thanks a lot for the nice comments and the feature request! Could you expand a bit on the "trajectory when u hit draw_points" though? You mean like drawing like a trace of the history of where each point was in the last X frames?

[wjassim]

Hello, thanks for sharing this awesome tracker.
Im trying to integrate it to darknet, but its not behaving as expected.

heres what i do:

detections = darknet.detect_image(network, class_names, darknet_image, thresh=0.2)
        detections2 = [
            Detection(get_centroid(detection[2], width, height), data=detection[2])
            for detection in detections
        ]
tracked_objects = tracker.update(detections=detections2)
norfair.draw_tracked_objects(frame_resized, tracked_objects)

the tracker is somewhat populated if i do a print(tracker.tracker_objects) but it doesnt look right and the draw_tracked_objects is dead. since darknet outputs x and y by default ive tried a variety of variations with converting to bbox etc. nothing has worked and im hoping for a little push :)

is the data parameter necessary in my case?

edit
i forgot to output darknets centroid as numpy array, its working now :)
for those who maybe end up in the same issue, change out get_centroid with something like this (just clean up my messy demodef):

def get_detlist(detections):
    arr = np.empty((0, 2), int)
    arr = np.append(arr, np.array([[detections[0],detections[1]]]), axis=0)
    return arr
    
Detection(get_detlist(detection[2]), data=detection[2])

Thank you @haviduck. I've tried you modified code for darknet and it is working perfectly. Great stuff.

How about if we want to track objects from two classes (ex. one class for player and second for ball) in the same frame from darkent? Thanks a minion.

[haviduck]

shouldnt this be just a matter of passing class identifier to detection and return it in repr?

[joaqo]

No, because a big issue with object tracking in general is occlusion, when an object is hidden behind another for a short period of time, the tracker may get confused and flip their ids incorrectly. If the tracker knows they are of different classes, it wont make this mistake.

In this case, by using a different tracker for each class, each tracker only sees objects belonging to its own class and doesn't have the opportunity to get confused with objects of different classes.

This is obviously not an ideal solution from a user experience point of view, and we are working to improve this, but it works.

[joaqo]

Awesome, thanks!

[haviduck]

Awesome, thanks!

no performance dip, but i am noticing a memleak if i draw. gonna investigate a bit more before i make a claim

[joaqo]

Ok thanks for taking the time! Please let us know so we may fix any potential bugs.

[joaqo]

Great, thanks!

[bwithai]

Hi @haviduck thanks for your nice work.

Please help me in #124, give your time i'm in beginner level in ML. #124 focus on (person, vehicle)

yeah, like dot history instead of a single centroid showing the 8 last points by circles and lines connecting them.

can you share that how you calculate the dx and dy