This repository contains the video dataset, implementation and baselines from Condensed Movies: Story Based Retrieval with Contextual Embeddings.
Project page | arXiv preprint | Read the paper | Preview the data
The dataset consists of 3K+ movies, 30K+ professionally captioned clips, 1K+ video hours, 400K+ facetracks & precomputed features from 6 different modalities.
Requirements:
- Storage
- 20GB for features
- 250GB for source videos (optional)
- Libraries
- ffmpeg (video download)
- youtube-dl (video download)
- pandas, numpy
- python 3.6+
- Navigate to directory
cd CondensedMovies/prep/
- Edit configuration file
config.json
to suit your needs. - If downloading the source videos (
src: true
), you can edityoutube-dl.conf
for desired resolution, subtitles etc. Please see youtube-dl for more info - Run
python download.py
Coming soon
- youtube download script
- missing videos check
- precomputed features download script
- dataloader
- video-text retrieval baselines
- facetrack visualisation
Why did some of the source videos fail to download?
This is most likely due to geographical restrictions on the videos, email me at [email protected] and I can help.
The precomputed features are averaged over the temporal dimension, will you release the original features?
This is to save space, original features in total are ~1TB, contact me to arrange download of this.
I think clip X is incorrectly identified as being from movie Y, what do do?
Please let me know any movie identification mistakes and I'll correct it ASAP.