toshikwa / rljax Goto Github PK

I've also recently been implementing RL algorithms using JAX, especially the off-policy family.

Compared to Pytorch, can you tell me if the computational efficiency of SAC or SAC+AE in the Gym tasks is significantly improved after using JAX? About how much? Such as in 1M steps?

Hi!

Thanks for the implementation for a really exhaustive list of RL algorithms. I have been referring to this repo when trying to implement some of the RL algorithms in JAX.

I noticed that there might be an issue with the loss function for the temperature parameter in SAC and SAC-AE, but this may be present in potentially other variants of SAC.

https://github.com/ku2482/rljax/blob/master/rljax/algorithm/sac.py#L267

I think it should be alpha instead log_alpha in the beginning, I cross-referenced with the soft_learning and this seems to be what they did. https://github.com/rail-berkeley/softlearning/blob/master/softlearning/algorithms/sac.py#L256

Also there may be some typos in SAC_AE. https://github.com/ku2482/rljax/blob/master/rljax/algorithm/sac_ae.py#L213, this is not the update_interval_ae in the constructor.

Best,
YL

Hi, Mr. Watanabe

Sorry for posting a question here. I understand the SAC discrete can work very well on discrete action space. I wonder if this can work on the MultiDiscrete action space. Suppose, you have a special cart pole game where you have to handle 10 cart poles simultaneously( I know originally there is only one pole in the game). How I handle this issue? I need 10 discrete action spaces where each space has an action to left and action to right.

In this case, does SAC discrete work?