olgkad / graphtrack

A feature tracking algorithm implemented in C++.

GraphTrack

A feature tracking algorithm implemented in C++.

This is my implementation of the following paper: B. Amberg and T. Vetter. 2011. "GraphTrack: Fast and globally optimal tracking in videos". In CVPR 2011. 1209–1216.

graphtrack's Issues

bug: select-candidates

In frames where interest points were marked, only one node should be included in the corresponding graph layer.

Web-service

Think about building a web-application out of this

Too many interest points selected.

How it's done now: A point of interest is marked in 5 different frames. Then candidate nodes are selected. In select_candidates_(), which is slow, approximate_distance_to_positive_patches() is computed for 5 positive patches. Then the path is traced.

How it's done in GraphTrack: A point of interest is only marked in the 1st frame. Then candidate nodes are selected. select_candidates_() will run faster now (5 times faster?) as it only checks one positive patch in each approximate_distance_to_positive_patches() call. Then the path is traced. If in a certain frame the point was marked incorrectly, we manually mark it (that changes the graph structure), and then we run Djikstra() again (which is inexpensive). No need to call select_candidates() again.

can't use large patches

When the patch-size is increased to 20x20, the algorithm performs very poorly. However, small patches often do not capture the entire point of interest. When the part captured is too small, only local (and not global) features are captured, sometimes resulting in errors. (e.g. when run with "rabbit_fast.avi". Rabbits limb is too big to be accurately represented by a 10x10 patch.

parameters might need to be tweaked

The default values of variables below work well for tracking faces. Different values might need to be assigned for a particular video.

lambda_f
lambda_b
lambda_s
lambda_d

candidate_selection() #2

The candidates are chosen to be the N_nodes patches which are closest to the positive examples and are locally minimal. That last part is not accounted for right now.

Track multiple points

The system is already reasonably good at tracking a single point of interest. Now it needs to be extended to track multiple points of interest. I propose the following design:

1. User experience. In the first frame, the user marks several points. Once the video is processed, the user goes through all the frames to see if any of the points was mistracked in any of the frames. If there's a rectangle (or a circle - whatever shape I use to mark points) is mislocated in some frame, how to know which of the points it was supposed to mark?

• Maybe they should be leaving a trace (trajectory) behind them.

• Maybe just build the system good enough so that no two points are mislocated at the same time. That's not a robust solution though.

• They could have different colours.

Besides, when marking the mislocated point, the system should know what rectangle (circle) we're trying to move to a different position. The best solution seems to be to make all the rectangles draggable. Implementing this, and assigning different colours to each mark should be enough.

2. Implementation.
   ... to be figured out

candidate_selection()

Currently, only 250 candidate nodes are being selected in each frame, and the rest is thrown away.

What actually should be done is 250 candidate nodes should be placed up in the priority queue, and only nodes at a distance >= 72 from positive patches should be discarded.

Djikstra: negative node weights

Need to add a constant to each node in a graph since as of now path lengths could be negative, and Djikstra may not always be returning the shortest path.