alexandrosstergiou / squeeze-and-recursion-temporal-gates

Code for : [Pattern Recognit. Lett. 2021] "Learn to cycle: Time-consistent feature discovery for action recognition" and [IJCNN 2021] "Multi-Temporal Convolutions for Human Action Recognition in Videos".

Hi there

I am an Assistant Professor at University of Twente's Data Management & Biometrics (DMB) group. Previously, I was a Postdoc at VUB and a Research Associate at the University of Bristol working with Dima Damen on video understanding. I obtained my PhD from Utrecht University where I was lucky to be supervised by Ronald Poppe and Remco C. Veltkamp. My thesis was on human action and interaction recognition in everyday social settings. During my time in Utrecht, I additionally worked on improving the efficiency and interpretability of spatiotemporal deep learning video classification models.

squeeze-and-recursion-temporal-gates's Issues

Training on HMDB51

Hello, thank you for providing the code. I am using your code to train on HMDB-51. However, I found your HMDB-51 models are trained from fine-tuning the HACS pre-trained model.(I got very bad result training on HMDB-51 data only) Can you provide the pre-trained HACS model? Another question: The HACS model is training on the HACS Segments data right? Thank you!

inference code on single video？

Dear all:
thanks for sharing this great work! I read the readme.md carefully and could not found any tips on inference on single video? anyone share the inference code?

All the batches are not cyclic consistent when using my own dataset

Hello, thanks for the paper and code.
I'm doing some research on gesture recognition using FMCW radar. It's similar to video action recognition because the radar data I use is a video-like(3 channels pics, 32 frames) style.
When I tring to use the SRTG on my own radar data set, all the batches are not cyclic consistent, that is to say, the function soft_nnc returns to an empty array. So when I set the 'gate' open, the program cannot continue to run.
Is there some advice ?
Looking forward to your reply!

The SRTG part may won't be update

As the code and paper show, if TG gate keep close,oprations on the input of pool and lstm part won't move forward. And it means when we do back propagation, parameters of lstm part and TG gate won't be updated, which can easily happen. So how did you solve the problem? or did I missing something?

There may be a bug in srtg_resnet.py

Hello, I find the line 177 in srtg_resnet.py is
' nearest_n = embeddings2.scatter_(2,indices,1.)'.
This code will change parameter embeddings2.
So the next line:
'b_consistent = embeddings2 - nearest_n'
will always get a zero result.
Is it a bug? Or did I neglect something?

Pretrained model release

Hello! First, thank you for providing the great model. I'm desperately looking for the pretrained model for the HACS dataset, but I can hardly find one. Due to my computational resource, I cannot train it from scratch. When can I get the pretrained weight of your network?

Pretrained model released

Hi, Thanks for your fantastic work! I am using your pretrained model on HACS dataset to compute video activation, where I found several confused things.

1. The link to pre-trained weights on HACS has some strange point. The option "srtg_r2plus1d_101“ 's checkpoint link corresponds to "srtg_r2plus1d_50_best.pth". I am wondering if there is some naming errors or they just have the same name.
2. I load the model of srtg_r2plus1d_50 using pytorch. However, its full connected layer has out_features of 400. The HACS dataset uses a taxonomy of 200 action classes, which is identical to that of the ActivityNet-v1.3 dataset. So I can't understand why the out features are 400 here. Does it mean the model is pretraining on HACS and then tranfering on kinetics of 400 classes?
   Looking forward your reply!

frames.db file has no record

I has tried to use the convert function twice，but still frames.db file has no record，I don't know why?

Few questions about pretrained model.

I am currently trying to reproduce some of your results.

Regarding your mtnet pretrained models, in Table 1 of your paper, I see a softpooling 1 x 2 x 2. However in your code, it seems that you only use a maxpool (MTnet.py l 499). In the pretrained weights are you using a maxpool or a softpool for your results ?

Hew to get softpool_cuda

Thank for this very nice work.

I would to test your work. How to get softpool_cuda library ?