PPO + gSDE / A2C + gSDE / SAC + gSDE about reacher-deep-reinforcement-learning

[araffin]

achieve faster convergence

using less samples? i would definitely try SAC with tuned hyperparameters for that ;)

Also kudos for stable baselines 3; it has been a real pleasure using this framework up to now!

Your welcome =) (we tried to keep the user-friendly api while cleaning up the internals)

but would be great to achieve faster convergence and also reduce the shaking behavior of the joints.

Yes, gSDE may help on that part. A continuity penalty to have smooth actions will improve things too (but you need to pass an history of obs/actions in that case).

[koulakis]

Hello Antonin,

Thanks for the hints. Also kudos for stable baselines 3; it has been a real pleasure using this framework up to now!

No, I haven't tried either the zoo parameters or gSDE yet, but I am curious to try both. The agents reach an almost optimal solution in terms of reward, but would be great to achieve faster convergence and also reduce the shaking behavior of the joints.

My plan is to:

• re-run each algorithm (A2C, PPO, TD3 and maybe try out SAC) with the half cheetah hyper-parameters you proposed and fine-tune them a bit for reacher
• repeat the above experiments with the addition of gSDE and check its effect on convergence and the final agents' shaking behavior

I will keep you posted when I make progress. If I am lucky I might get some time to work on this one this weekend :)

Cheers,
Marios