Documentation improvement about gym

[tlbtlbtlb]

The reward should be a function of only the previous state and action. To get good rewards, you have to plan ahead to get into states where there are good actions to take.

[alexnikulkov]

Can we come back to this question? I think the reward is actually a function of the current state and action, not previous. A few examples from the simpler environments where it's really obvious:

• Pendulum
• CMC

Also, I'd argue that returning the reward due to previous action is problematic conceptually because for the first step() call there is no previous action; and the last action's reward would end up being discarded since there's no next step