sangminwoo / actionmae Goto Github PK
View Code? Open in Web Editor NEW[AAAI 2023] Official pytorch implementation of "Towards Good Practices for Missing Modality Robust Action Recognition"
License: MIT License
[AAAI 2023] Official pytorch implementation of "Towards Good Practices for Missing Modality Robust Action Recognition"
License: MIT License
ActionMAE/lib/modeling/actionmae.py
Line 365 in 4ae5a81
Thank you for the interesting work! I just had a few questions from the paper...
In Section 3.2 "Note that only memory token and dummy tokens are affected by the reconstruction loss (i.e., reconstruction loss is not computed for the remaining tokens) during training." Does this mean the reconstruction loss is not backpropagated to the encoders of the "healthy" (retained) modalities? If so, why might this be done/were you able to do experiments ablating this?
Also, I was wondering if you had run experiments on simply dropping out a modality during training for the baseline to help the model get used to missing modalities.
Thank you!
I am working on the reproduction of this paper, and I found that using the baseline model to train the NTU dataset with RGB+Depth two modalities, the highest accuracy I achieved was acc==77, I don’t know why.
In addition, my data set reading speed is very slow.
I wonder if the author or other friends can give me some training experience and suggestions.
I used the dataset downloaded from the official website and used the script to extract frames from the RGB modality videos, 16 frames per sample.
I made very few changes to the source code, and the parameters are basically the default parameters of the source code.
Thank you very much.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.