The SRTG part may won't be update about squeeze-and-recursion-temporal-gates

[RongchangLi]

Yes, the main mindset is to only use information that is cyclic-consistent. Therefore, the LSTM parts of the network are only updated if and only if they are consistent with the spatially pooled version of the activations. (You can think of this as worse-case scenario with only using the backbone CNN). For most experiments, the gates were inactive for the first 30~50 epochs, thus the weights of the LSTM would not be completely random when the gates are indeed active for the rest of the training.

Best,
Alex

Ha, the result seems very reasonable. Thanks for your reply and your work(both the code and paper) did inspire me much.

[alexandrosstergiou]

Yes, the main mindset is to only use information that is cyclic-consistent. Therefore, the LSTM parts of the network are only updated if and only if they are consistent with the spatially pooled version of the activations. (You can think of this as worse-case scenario with only using the backbone CNN). For most experiments, the gates were inactive for the first 30~50 epochs, thus the weights of the LSTM would not be completely random when the gates are indeed active for the rest of the training.

Best,
Alex