chingyaoc / pytorch-reinforce

PyTorch Implementation of REINFORCE for both discrete & continuous control

PyTorch REINFORCE

PyTorch implementation of REINFORCE.
This repo supports both continuous and discrete environments in OpenAI gym.

Requirement

• python 2.7
• PyTorch
• OpenAI gym
• Mujoco (optional)

Run

Use the default hyperparameters. (Program will detect whether the environment is continuous or discrete)

python main.py --env_name [name of environment]

Experiment results

continuous: InvertedPendulum-v1

discrete: CartPole-v0

Reference

• pytorch example

pytorch-reinforce's Issues

continuous-control doesn't work for MountainCarContinuous-v0

Hi. First af all - thanks for good clear code!
My problem. I am trying to run this continuous-control algorithm "as is" for simplest gym's enviroments such as MountainCarContinuous-v0, Pendulum-v0 and nothing work. I tryed to change lr, number of episodes, quantity of neural network layers, and so on - nothing. There are random fluctuations on the end of learning near low-reward position. Can you provide good parametres for this enviroments or give recommendations what to do?

A little confusion from a newbie.How to get the loss function of REINFORCE?

Hi, thanks for your code, they are well written and very helpful to newbie like me!!

But there is one line that I don't understand, I'd appreciate if you could explain that to me.

loss = loss - (log_probs[i](Variable(R).expand_as(log_probs[i])).cuda()).sum() - (0.0001entropies[i].cuda()).sum()
loss = loss / len(rewards)

I wonder how can I derive the loss function of REINFOCE algorithm. And why does the entropy term appear? This doesn't seem to be straight forward to me.

can you give me the paper about the continuous situation?

thank you

Please add some explanation

Hi,

Thank you for the sample code.
I could not understand what exactly is happening here:
https://github.com/JamesChuanggg/pytorch-REINFORCE/blob/master/reinforce_discrete.py#L52

If possible can you please give a little explanation.

Thanks

normalized_actions.py line16 actions -> action

bug

Inconsistent policy gradient

The policy gradient here seems to be different from the policy gradient in most places, e.g., Berkeley CS285. Can the author provide where did you cite the original algorithm?