breezeo / acrm-for-moment-retrieval

ACRM-for-moment-retrieval

This is the repository of our paper https://arxiv.org/abs/2009.10434.

For now, only the testing model and the corresponding code is available. We will release the whole project once the paper is accepted.

The pretrained models are provided for Charades-STA which shall be stored in 'Home_path/checkpoints/charades_sta_train' and TaCoS which shall be stored in 'Home_path/checkpoints/tacos_train'.

The extracted I3D features for TaCoS and for Charades-STA are provided for both of them, which should be stored in 'Home_path/proposal_free/preprocessing/tacos' and 'Home_path/preprocessing/charades-sta', respectively.

The above models and features are stored in Baiduyun disk, where the extraction key is th08 for all of them.

The pre-trained glove embedding that we use is glove.840B.300d.zip trained with the Common Crawl (840B tokens, 2.2M vocab, cased, 300d vectors, 2.03 GB download), which shall be unzipped and stored in 'Home_path/data/TMLGA'.

The code is based on https://github.com/crodriguezo/TMLGA. And thanks to their features.

run the program with python main.py --config-file experiments/tacos_train.yaml

Citing

If you find our paper useful in your research, please consider citing:

@Article{tang2021frame, author = {Tang, Haoyu and Zhu, Jihua and Liu, Meng and Gao, Zan and Cheng, Zhiyong}, title = {Frame-wise Cross-modal Matching for Video Moment Retrieval}, journal = {IEEE Transactions on Multimedia}, year = {2021}, publisher = {IEEE}, }