overfitting - BatchNormalization and Dropout about shainet

[bararchy]

@drujensen

1. I would suggest adam instead of sgdm.
2. you can try mini batch training so that the net will train on different chunks which makes the over fitting less likely
3. Make sure you shuffle the test train array so if you have similar examples they are not close together .

@ArtLinkov any pointers ?

[drujensen]

@bararchy not sure why but adam always returns NaN as the results. Maybe I need to use adam with train_batch?

I am using shuffle and that seems to help. I am going to try the parallel eraser layer and see if that helps.

@ArtLinkov Is there an equivalent BatchNormalization?

[bararchy]

adam is best for batch training , we are working on the NaN issue

[ArtLinkov]

@drujensen do you still get the same problem?
Sorry for not answering sooner, it's been a while since we worked on SHAInet (busy with the startup)

[drujensen]

We can close this. When I get some time, I will jump back on this and perform more testing.