manantomar / mirror-descent-policy-optimization Goto Github PK

According to the paper, from table 4, I think the Bregman stepsize is the klcoeff in the code, and from table5 we can set tsallis_coeff for each task. As discussed in the paper, the entropy-coeff set 0.2 in all tasks. However, since I run python run_mujoco.py --env HalfCheetah-v3 --num_timesteps 3e6 --sgd_steps 5 --klcoeff 0.3 --lam 0.2 --tsallis_coeff 0.5 --run 0, I got negative mean reward at last timesteps. And some other tasks are the same. Do I miss something?