Disagreements with the async paper about atari

[Kaixhin]

Thanks for finding these - I'll just update on this comment if it's simple.

1. Oh that's subtle - they are indeed using the smaller architecture from the pre-Google 2013 NIPS paper. Just pushed a fix in 20181f3. This should make the async experiments faster as well!
2. Yes, that's part of the ALE wrapper they wrote - but it can be controlled using the -actRep option with this code.
3. Leaving to @lake4790k.
4. That looks like an error to me - fixed with dc25dad.
5. From what I remember, that has something to do with recurrent networks, but I'll let @lake4790k have a look since he wrote this bit.

[lake4790k]

@YurongYou re 3: the actor and critique need to have different learning rates, so that means that learns half as fast.

re 5: I think lines 59 and 67 are indeed correct! If alwaysComputeGreedyQ is false then forward is NOT done when the action is random (notice the return statement), only when the action selection is greedy.

I think this can be closed, all has been addressed.