tryolabs / norfair Goto Github PK

Hi thanks for the great work.
I want to get the index of input detections based on an object id, how can I achieve that ?
For example, I tried:

detections = [Detection(p) for p in predictions.pred_boxes.get_centers().numpy()]
tracked_objects = tracker.update(detections=detections)

## I want to get the detections of objects whose id are 1 or 2
targeted_ids = [1, 2]
targeted_idx = [idx for idx, tObj in enumerate(tracked_objects) if tObj.id in targeted_ids]

## however, this is not I want to track
targeted_detections = detections[targeted_idx]

However, it turns out that the targeted_detections are not the objects I am interested in (i.e., with tracked id 1 or 2)

Thanks!

Hi,

I was running your YOLOv4 demo on a standard Google Colab CPU environment earlier. In YOLO.init(*) line 21, the map_location did not check the availability of CUDA. This threw me a NoneType Error which took a few minutes to locate and solve. Perhaps you can do a quick revision of this line to also include a check on cuda.is_available()?

Thanks =)

Hi,

I'm trying to implement a tracker which uses embedding.
I'm storing each embedding in the detection.data, however I also need a place where to store the TrackedObject embedding, in order to compute a suitable distance.
Is there something similar to detection.data for TrackedObject?

Hey, how about an extension for the Detection class and drawing.py to add bboxes and masks to directly visualize them?

Do you think this would fit norfair?

Hello,

Thanks for the repository at first. I am working on a NVIDIA Jetson TX2 developer kit and I can detect humans with yolov5 on GPU. How to make norfair work on GPU? If we can not is it lightweight for Jetson TX2 too?

How do i reduce the match rate to initialize a TrackedObject faster?

Are you accepting any PRs? I can create a PR for yolov5 demo using the yolov5 pip package.

Hello ,

I am trying to implement norfair along with yolov4 with my own dataset. I want to know how well norfair is tracking objects in my dataset ? can you please help me in evaluating norfair ?

@gerhc @draix @dekked hi thanks for sharing this source code when i try to install pip install norfair[video] # Adds several video helper features running on OpenCV
pip install norfair[metrics] # Supports running MOT metrics evaluation
pip install norfair[metrics,video] # Everything included
I i get the following error "ImportError: cannot import name 'metrics'"

Please let me knw what is the issue

Hi,
Would it be possible to implement a simple vehicle speed tracking based on https://github.com/tryolabs/norfair/blob/master/demos/yolov5/yolov5demo.py code?

I'm new to the area and I'm trying to understand what would be the steps to implement it?

Thank you.

Hi, I'm successfully able to convert my x,y,w,h coordinates to cx,cy, create Detection objects with those center points, and draw them with draw_tracked_objects.

However I saw that there's a drawing.draw_tracked_boxes function, and thought it'd look nicer if I drew boxes instead, but I can't seem to figure out how this works. I attempted to create Detection objects with 4 points (xyxy), but this class doesn't seem to accept 4 points. Is drawing tracked boxes currently supported?

Hello, What if I have multiple cameras, for example, 2 cameras and need to track the object from these 2 cameras, is this applicable?

Hi Joaqo,

Using colab, when I run the Norfair detectron2 demo code by pasting the code into a cell, I receive the following error related to this line: video = Video(input_path="./video.mp4")
ValueError: not enough values to unpack (expected 2, got 0)
(Full error pasted below)

When the same code is called as a script (!python detectron2_cars.py) it runs with no issues.
Perhaps this is not unexpected behaviour, but I can't work out why it works in a script but not in the cell.

ValueError Traceback (most recent call last)
in ()
19
20 # Norfair
---> 21 video = Video(input_path="./video.mp4")
22 tracker = Tracker(distance_function=centroid_distance, distance_threshold=20)
23

1 frames
/usr/local/lib/python3.6/dist-packages/norfair/video.py in init(self, camera, input_path, output_path, output_fps, label, codec_fourcc)
83 )
84 self.task = self.progress_bar.add_task(
---> 85 self.abbreviate_description(description),
86 total=total_frames,
87 start=self.input_path is not None,

/usr/local/lib/python3.6/dist-packages/norfair/video.py in abbreviate_description(self, description)
185 def abbreviate_description(self, description):
186 """Conditionally abbreviate description so that progress bar fits in small terminals"""
--> 187 _, terminal_columns = os.popen("stty size", "r").read().split()
188 space_for_description = (
189 int(terminal_columns) - 25

ValueError: not enough values to unpack (expected 2, got 0)

What happens when the hit inertia counter hits hit_inertia_max?
As per docs, hit_inertia_max defines how large inertia can grow, and therefore defines how long an object can live without getting matched to any detections. But does hit_inertia_min dictate how long an object lives before being destroyed for not getting matched?

In our experience, tracking runtime can be greatly reduced by threading each of the individual tracked objects (in this case TrackedObject) as there's less blocking etc. I don't have exact numbers, but it could easily be 10x with lots of objects and not-to-expensive detector. It's pretty easy to do as well.

Would there be interest in this? My proposal would be to have a single-threaded or multi-threaded mode, as the former can be easier to debug etc.

Hi all,

When I call Paths.draw() I get the following error even though draw_tracked_objects(), which gets the same input arguments, works fine.

Traceback (most recent call last):
  File "C:\Users\path/1445870238.py", line 76, in <module>
    Paths.draw(original_frame, tracked_objects)
TypeError: draw() missing 1 required positional argument: 'tracked_objects'

First of all, thanks for the package. I am currently working on tracking cars using the detections from a yolov5 network. The problem is that it works fine when cars go through different lanes. My problem is that I am recording an entrance and exit (same lane) and when a car leaves and another one enters they get matched as the same car. I tried reducing the point_transience but that did not really help. Also, is there a common practice when setting up the hyperparameters of the tracker (like recording the average distance between detections?

I'm using Norfair and another library. However, I'm getting a version conflict for rich.

Is it possible to change the dependency of norfair to be able to use higher version of rich? I'm using poetry to install norfair and I'm getting a "SolveProblemError Because norfair (0.4.0) depends on rich (>=9.10.0,<10.0.0)". I need to use at least version 11.2.0.

I can force install higher version of rich using pip but I think it would be cleaner if the dependencies are clear. Norfair seems to work fine even with higher version of rich.,

Thank you for the useful project! I have some misunderstanding of tracker parameters from the API. From the description of initialization_delay:

Each tracked object waits till its internal hit inertia counter goes over hit_inertia_min to be considered as a potential object to be returned to the user by the Tracker. The argument initialization_delay determines by how much the object's hit inertia counter must exceed hit_inertia_min to be considered as initialized and get returned to the user as a real object.

From there, my understanding was the following. Let's say we have a static object which we get during the next 10 shots, and then it disappears. If we set hit_inertia_min = 5, hit_inertia_max = 10, initialization_delay = 3, then the tracker will wait till it gets 5 matches (hits), then it considers the object as a potential object (which is not clear what it is). Then we wait till the counter exceed hit_inertia_min + initialization_delay = 8, and on the 9th hit, we will get a real object.

At the same time, I did a test run with a fixed detection that I feed to the tracker during 10 shots. After 20 updates, the outputs of tracker.update(detections=detections) looks like this:

Update number: 0
Tracked objects: []
____________
Update number: 1
Tracked objects: []
____________
Update number: 2
Tracked objects: []
____________
Update number: 3
Tracked objects: [Object_1(age: 3, hit_counter: 9, last_distance: 0.00, init_id: 33)]
____________
Update number: 4
Tracked objects: [Object_1(age: 4, hit_counter: 10, last_distance: 0.00, init_id: 33)]
____________
Update number: 5
Tracked objects: [Object_1(age: 5, hit_counter: 11, last_distance: 0.00, init_id: 33)]
____________
Update number: 6
Tracked objects: [Object_1(age: 6, hit_counter: 10, last_distance: 0.00, init_id: 33)]
____________
Update number: 7
Tracked objects: [Object_1(age: 7, hit_counter: 11, last_distance: 0.00, init_id: 33)]
____________
Update number: 8
Tracked objects: [Object_1(age: 8, hit_counter: 10, last_distance: 0.00, init_id: 33)]
____________
Update number: 9
Tracked objects: [Object_1(age: 9, hit_counter: 11, last_distance: 0.00, init_id: 33)]
____________
Update number: 10
Tracked objects: [Object_1(age: 10, hit_counter: 10, last_distance: 0.00, init_id: 33)]
____________
Update number: 11
Tracked objects: [Object_1(age: 11, hit_counter: 9, last_distance: 0.00, init_id: 33)]
____________
Update number: 12
Tracked objects: [Object_1(age: 12, hit_counter: 8, last_distance: 0.00, init_id: 33)]
____________
Update number: 13
Tracked objects: [Object_1(age: 13, hit_counter: 7, last_distance: 0.00, init_id: 33)]
____________
Update number: 14
Tracked objects: [Object_1(age: 14, hit_counter: 6, last_distance: 0.00, init_id: 33)]
____________
Update number: 15
Tracked objects: [Object_1(age: 15, hit_counter: 5, last_distance: 0.00, init_id: 33)]
____________
Update number: 16
Tracked objects: [Object_1(age: 16, hit_counter: 4, last_distance: 0.00, init_id: 33)]
____________
Update number: 17
Tracked objects: []
____________
Update number: 18
Tracked objects: []
____________
Update number: 19
Tracked objects: []
____________

Which means that when hits (age) exceed initialization_delay, we already have Object_1 in the list of object. At age value 9, nothing happens (change from potential to real object). Also, we see that when age exceeds initialization_delay, hit_counter start from the value 9 (what means this number?). Then the disappearance of an object at hit_counter value 3 is expected as well as max hit_counter value = 10 (but it's fluctuating from 10 to 11).

I would be glad for some clarifications on params meaning.

Hello, we are using some of the features that were added afte rthe 0.3.1 release, when do you think to publish a new release?

hi, I just tried to real time- detect people and want to get for example named A person enter and camera saw it then first x,y coordinate and last one according to tracking but I couldnt. I also can not show the ids on the frame.. I examined on README tutorial but I can not still fix it. Could you help me ?

Hello,

I've a file that contains keypoints from my objects from all the frames of my video. I convert this values into numpy arrays to pass them to the norfair Detection object, which works correct. However, the don't always have the same shape in all the frames sometimes they have shape (11,2) frame n and in frame n+1 the shape is (10,2) which led me to the following error.

distances = np.linalg.norm(detected_pose.points - tracked_pose.estimate, axis=1)
ValueError: operands could not be broadcast together with shapes (11,2) (10,2) 

I'm trying to use this script as reference to develop my own one.

Thank you for the help.

I was trying out to increase the length of a tracked object's past detections, but got an 'Unexpected keyword argument'. I am running norfair==0.3.1 (pip installed). I think the version of the pip package doesn't correspond with the one published in github? I attach a simple script to reproduce the issue:

from norfair import Tracker

max_distance_between_points=50

def euclidean_distance(detection, tracked_object):
    return np.linalg.norm(detection.points - tracked_object.estimate)

tracker = Tracker(
        distance_function=euclidean_distance,
        distance_threshold=max_distance_between_points,
        past_detections_length=50

    )

Hi,

Thank you for the useful tracker. I just wanted to ask if it is possible to run a detector (such as yolo) to detect objects every N frames only, while norfair tracks objects and fills in the gaps efficiently to speed up the overall process. Running yolo or any other detector for each frame all the time might slow down the FPS.

If the idea is workable, which function from norfair can do the tracking when the detector is disabled?

Thanks a million..

Hi @joaqo, thank you for the amazing work!

Im currently using deepsort with yolov5 for tracking the objects that Im interested in. I would like to change the deep sort and use your library which is faster. However I a question regard to results of the tracking. In Deepsort, Im able to return Tracker_id, Class name, confidence score and bbox. I wanted to ask you is there any way to access to these information via norfair tracking? I tried to access these information but no success. Would appropriate any help..

Thanks!

Hi, I have a question related to issue # 19:
I'm working through the detectron2 demo (detectron2_cars.py) to understand the code and wanted to change the detected class from cars to person. I found that changing row 32 to "if c == 0" does this. How do I export/print a list of the detected classes?

Also, I would like to export to a csv file the centroid/center coordinates for each tracked object for each frame (similar to print_objects_as_table, but ideally with the data from each frame in one row). Any suggestions?

Hi,

I am trying to track pose estimates using the "Tracking pedestrians with AlphaPose" demo as a reference. However I am using Nvidia trt-pose (https://github.com/NVIDIA-AI-IOT/trt_pose) instead of alpha pose as given in the demo.

The pose estimation alone runs well at around 25fps (having about 50% CPU usage), however when I include the pose tracking, my fps drops to about 10-12 fps and its definitely a CPU bottleneck as my CPU usage is around 98% when running tracking.
I would like to know if this is considered "normal" with the pose estimation tracking or I am doing something wrong in my end.

PC specs
GTX 1060 6GB
intel i7 8500 H
6GB ram

Thanks for the great work.

Hi. Thank you for the great tool.

I am wondering if it is possible to wrap darknet-based yolov4 (https://github.com/AlexeyAB/darknet/blob/master/darknet_video.py) in Norfair's detection objects. The darknet-based yolov4 (darknet_video.py) provides detections for each detected object in the frame in the following format:

class (str),
confidence (one value in str), and
bbox (a tuple with 4 values).

This is shown in the following image:

Thanks a million.

Hi community, I am trying to use the demo of yolov5 in a custom video.
The problem is that norfair does not draw anything at all.
I have done some debugging and I can see that yolo_detections look fine. It contains the bounding boxes, the score and the category. Besides, detections also looks fine, it contains the centroid of the object and the score.
The main problem lies on: tracked_objects = tracker.update(detections=detections).
tracked_objects remains an empty list...
Any ideas on what is going on?

Thanks in advance! 😄

Hi,
before I begin I would like to thank you for the amazing work on this library and for the clear repo!

As the title suggests, I can't see why you are not giving end-users the possibility to keep track of class labels when working with traditional object detector. I read that you added the data attribute to the Detection class (which can store them eventually), and I also read on issue #47 that you don't use Norfair too much for different object classes, but I still think that an (optional) attribute regarding object classes would be great.

I was working on tracking different object classes, and I think it would be not optimal of me instantiating multiple trackers in order to keep track of multiple object classes. I guess the codebase right now is not ready to handle multiple classes in a single tracker, since different classes could be mixed up together by mistake, even though it would be a really nice addition.

If I had a class-label attribute for the Detection and/or TrackedObjects classes, I would love that.

Thanks for the hard work!

Thanks for this work!

Where is the object class to be detected defined (out of 80 classes) in yolov4demo.py? The class 'car' is defined somewhere I believe so where to change that?

I made a demo that works with Googlecolabratory using norfiar - yolov4 demo as a reference.
GoogleColabratory - Yolov4 CarDetection Demo

I just couldn't find a video to use for the demo, so the results of the demo are hard to understand.
Can you tell me if you have any other good demo videos?

Hi, thanks for your explanations first :) I want to ask instead of using yolo, I want to use opencv haarcascade for real time body detection, Could you guide me for this purpose?

This tool is amazing, and deserves to be cited in my work. Can you please provide the name of the authors so I can cite this tool properly?

Are there any more examples of distance functions? I'm doing tracking with bounding boxes, and a feature extractor on the box. Is there an example that uses this type of setup for norfair?

Hi, is there any way this library can handle multiple classes?

Hello,

Thank you for the nice work.
I'm still experimenting, but can we add also multiclass tracking?

Hello, thanks for the great work. My question is: I have trained weights for my custom objects. Could I just applied the weights and track the custom objects? Thanks a lot.

Meanwhile, in testing, I found there is a problem about drawbox:

module 'norfair' has no attribute 'draw_boxes'

Hello. I'm using Norfair to track objects in MP4 videos (which also contain audio). I'm using code similar to the OpenPose example (updated to support the new OpenPose API). Here's an excerpt of the code:

import norfair
from norfair import Detection, Tracker, Video
import pyopenpose as op

frame_skip_period = 3
detection_threshold = 0.01
distance_threshold = 0.4

for input_path in args.files:
    video = Video(input_path=input_path)
    tracker = Tracker(
        distance_function=keypoints_distance,
        distance_threshold=distance_threshold,
        detection_threshold=detection_threshold,
        point_transience=2,
    )
    keypoint_dist_threshold = video.input_height / 25

    for i, frame in enumerate(video):
        detected_poses = pose_detector(frame)
        if detected_poses is not None:
            detections = (
                []
                if not detected_poses.any()
                else [
                    Detection(p, scores=s)
                    for (p, s) in zip(detected_poses[:, :, :2], detected_poses[:, :, 2])
                ]
            )
            tracked_objects = tracker.update(
                detections=detections, period=frame_skip_period
            )
            norfair.draw_points(frame, detections)
            norfair.draw_tracked_objects(frame, tracked_objects)
        video.write(frame)

Here's the output MP4:

It has the poses overlaid appropriately. However, the original video had audio. Is there a way this can be added to the output video with Norfair?

Hi,

I am trying to track static objects on the road with a camera mounted to a moving vehicle. I set hit_inertia_min to 1, hit_inertia_max to 10 and point_transcience to 50. I noticed that at certain vehicle speeds, the tracking works fine. However, at slower speeds in a parking lot for example, the same object in view is interpreted as new object every few seconds. Is this because of hit_inertia_max?
What parameters are recommended to change and can we change these dynamically based on vehicle speed while tracking is in progress?

Hello, thanks for sharing this awesome tracker.
Im trying to integrate it to darknet, but its not behaving as expected.

heres what i do:

detections = darknet.detect_image(network, class_names, darknet_image, thresh=0.2)
        detections2 = [
            Detection(get_centroid(detection[2], width, height), data=detection[2])
            for detection in detections
        ]
tracked_objects = tracker.update(detections=detections2)
norfair.draw_tracked_objects(frame_resized, tracked_objects)

the tracker is somewhat populated if i do a print(tracker.tracker_objects) but it doesnt look right and the draw_tracked_objects is dead. since darknet outputs x and y by default ive tried a variety of variations with converting to bbox etc. nothing has worked and im hoping for a little push :)

is the data parameter necessary in my case?

edit
i forgot to output darknets centroid as numpy array, its working now :)
for those who maybe end up in the same issue, change out get_centroid with something like this (just clean up my messy demodef):

def get_detlist(detections):
    arr = np.empty((0, 2), int)
    arr = np.append(arr, np.array([[detections[0],detections[1]]]), axis=0)
    return arr
    
Detection(get_detlist(detection[2]), data=detection[2])

hi,professor:

this project is very good! but in practical use , we prefer to C++， so, can realize this project with C++? thanks!

Hi, there are a lot of applications these days on embedded devices that need a customizable object tracking library like norfair. Is there any plan for supporting micropython?

heya, im trying to figure out what param i can tinker with to track fast moving objects. in my case the object is present in 4,5 frames. its being detected by yolo, but tracker wont pick up on it. i set distance to a higher amount and thinking transience might be it but i dont 100% understand it.

Is there any way to pass class indexes to force same color for distinct classes?

I'm coming across with this error on the title, and couldn't fix that. Norfair is installed with pip on Colab, but can't import Paths from it. Thanks in advance!

can we use this tracker with yolov5? if yes then any documentation available for that?

Thanks for this great library 👍

I'm wondering why is here the prediction score of the detection is exported as -1?

Hi, firstly thanks for the work.

I'm implementing a face mask detection system over street cameras and I wonder that is there any way to integrate face mask detection with norfair tracking system. I detected faces and I use it with norfair no problem. But when i want to integrate the face mask detection system i couldn't figure out how to track and determine whether a traced face have mask or not depending on the previous results.

Is there any way to do that? Thanks in advance