Weights for frame prediction used in the paper about unsupervised-videos

[mansimov]

Hey

Interesting. Last time I tried running it I had no problem converging and managed to train within a day or two on Titan/Titan Black GPUs.

The UCF-101 patches I uploaded specifically here (the one used in the paper was much larger) is a very small dataset due to memory constraints so as far as I remember I managed to heavily overfit at training set.

I will try finding weights but I can't promise anything :)

[b3nk4n]

Thank you for your fast answer!

I just realized I was not that precise.
Up to now I only tried to train it on MovingMNIST. For training data (which is using the ground truth as input to the future-predictor), it is actually converging quite good. I think it was down to a training-loss to about 170, but the validation-loss (which is using the previously predicted frames as input to the future-predictor) stays at a value of about 2600. In consequece, it can predict the first frame very well, but it does not seem to learn the overall motion that good.

[mansimov]

Hm, I don't remember having this issue of validation loss being way worse compared to training loss.

[jbboin]

Hi,

I’m a bit late to the party but it seems that I found the reason why this problem occurred. It was actually caused by the bug fixing that occurred in commit fae11d6. During testing, for each time step the sigmoid was applied to the inputs of the LSTM (which are the outputs of the previous stages). But in the end, the sigmoid was applied again to all outputs. This caused it to be incorrectly applied twice for the outputs at t = 0, …, T-2. My pull request (#26) fixes this problem.

Running the default training mode (1 layer LSTM) on MovingMNIST now gives me a validation cross entropy of around 335 after 120k iterations, which is much more consistent with the values reported in the paper.

[mansimov]

Thanks for noticing and fixing this bug @jbboin !