shaneson0 / ballclustering_iccv2019

Video Face Clustering with Unknown Number of Clusters - Ball Cluster Learning

Video Face Clustering (ICCV 2019)

Video Face Clustering with Unknown Number of Clusters
M. Tapaswi, M. T. Law, and S. Fidler
International Conference on Computer Vision (ICCV), October 2019.
arXiv

• Realistic setting for clustering face tracks in videos
• Number of clusters is not known
• Background character face tracks are not removed and need to be resolved
• Ball Cluster Learning: a new loss function that carves feature space into balls of a learned radius that can be used as a stopping criterion of agglomerative clustering

---

Dataset

We use 6 episodes of season 1 of The Big Bang Theory and 6 episodes of season 5 of Buffy - The Vampire Slayer. Face track labels are resolved between background characters.

Original tracks were provided by:
M. Bäuml, et al. Semi-supervised Learning with Constraints for Person Identification in Multimedia Data. CVPR 2013.

Please use the download.sh script inside data/ for convenience.
Face tracks (5.3 MB)
VGG Face SE-ResNet50-256 features (519 MB)

---

Evaluation

(02.09.2019) Our final checkpoint with evaluation code has been released. Numbers can be reproduced by downloading the features and labels and running with video_name as one of the following videos: bbt_s01e01..06 or buffy_s05e01..06

python evaluate.py <video_name>

---

Code

coming soon