Comments (4)
Thanks for your full guidance.
from norfair.
Hi @aguscas
Thank you very much
I fixed the above issue by revising the mean_eculidean function as the following.
Most of cases, it works well.
That is, when associating a detection and an already existing tracked object, I used the tracked object's last detection position rather than the estimated next position.
from norfair.
Hello! This is a common problem in tracking, but I have a few suggestions you can try. The main reason this problem happens is because we estimate the velocities of the objects when tracking them, to predict where they should be in the following frame. The sudden change in velocity during these collisions makes the tracking more challenging.
These are a couple things you might want to try:
- Don't use the Kalman filter: When initializing your tracker, you can set the argument
filter_factory
toNoFilterFactory()
to not use the velocity.
from norfair.filter import NoFilterFactory
tracker = Tracker(
distance_function=distance_function,
detection_threshold=DETECTION_THRESHOLD,
distance_threshold=DISTANCE_THRESHOLD,
hit_counter_max=HIT_COUNTER_MAX,
initialization_delay=INITIALIZATION_DELAY,
pointwise_hit_counter_max=POINTWISE_HIT_COUNTER_MAX,
filter_factory=NoFilterFactory(), # with this line, you wouldn't be using the velocity
)
Remember that doing this might cause other problems when the balls are moving normally, since we just use the last position of the ball (and not its velocity) to estimate where it should be on the following frame. You might want to change your distance_function
and DISTANCE_THRESHOLD
for that. I don't know which distance function you are currently using, but for example, I would suggest using the euclidean distance between the centers of the balls instead of IoU, since the boxes of the predicted position and the detected position might not even overlap if a ball moves too fast. If you need any help with this, please let me know.
- You can try using a distance function that considers an embedding of how the object looks like. So in a similar fashion (but not exactly the same) to what it was done in the Reid demo, you can compare how similar two objects look instead of just their position on the screen.
So, for example, for each detection you could crop the bounding box and compute its color histogram with opencv and normalize it.
def get_hist(image):
hist = cv2.calcHist(
[cv2.cvtColor(image, cv2.COLOR_BGR2Lab)],
[0, 1],
None,
[128, 128],
[0, 256, 0, 256],
)
return cv2.normalize(hist, hist).flatten()
and put that in the embedding
attribute of the detection
for detection in detections:
cut = get_cutout(detection.points, frame) # get_cutout would be a function that crops the bounding box
if cut.shape[0] > 0 and cut.shape[1] > 0:
detection.embedding = get_hist(cut)
else:
detection.embedding = None
Then you can define a custom distance_function
that compares the embedding of the detection in the current frame, with the embedding of the last_detection
associated to a tracked object (or with a subset of past_detections
). You can even combine both the spatial distance and the similarity of the embeddings, either by simply multiplying or adding both values, or for example doing conditionals.
For example, compare the embeddings, and if the objects look similar, then you compute the spatial distance and use that as the final distance, otherwise if the objects don't look similar then set that final distance to a really large number so that they won't ever match. Or you can do it the other way around, if the objects are close (spatially) to each other, then compute the similarity of the embeddings and set that as the final distance, otherwise if they are really far from each other then set the final distance to a super large value without even looking at the embeddings.
The following is an example just using the embedding similarity with the most recent past_detection
:
def your_custom_distance_function(detection: "Detection", tracked_object: "TrackedObject") -> float:
for past_detection in reversed(tracked_object.past_detections):
if past_detection.embedding is not None:
last_embedding = past_detection.embedding
break
else:
return 1
# this should give a distance between 0 and 1, since we are just doing 1-correlation
distance = 1 - cv2.compareHist(
last_embedding, detection.embedding, cv2.HISTCMP_CORREL
)
return distance
Please note that I haven't tested any code snippet here, so there might be some problems to address when trying to run them. If you need any further assistance, please let me know.
from norfair.
Nice! That was a great idea also!
from norfair.
Related Issues (20)
- error: (-28:Unknown error code -28) The input arrays should have at least 4 corresponding point sets to calculate Homography in function 'findHomography' HOT 6
- How to get object/track id and their bbox coordinates for a video? HOT 2
- Question about skip_period HOT 2
- Skipped frames on avi videos
- Suggestions to make the processing faster HOT 5
- Typing issues (example: `Detection.embedding` is `Unknown | None`)
- object ids in tracked_objects skipped a number? HOT 9
- Detection confidence when drawing tracked_objects' boxes? HOT 5
- Any tips on how to use Norfair for re-identification accross cameras HOT 1
- Ask a question about object tracking HOT 3
- Configure logging behavior? HOT 2
- The class of object being tracked changes over time HOT 3
- FixedCamera crashes with an inappropriate point array shape
- Detection age not always set
- Detection age not updated after merging tracked objects
- What are the tracked objects? HOT 6
- Object Placement using Homography HOT 2
- Tracking Algorithm
- 2D Tracking in World Space
- YOLOv7 Demo flips the Red and Blue channels on inference
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from norfair.