fangihsiao / gail-tensorflow Goto Github PK

Hi,

Thank you so much for sharing this code. I'm new to GAIL and I'm trying to understand how to implement the algorithm into code.

There are two things I'm confused about.

The first one is, I didn't see an interaction between the PPO update and the discriminator update. It seems that PPO update is still using its own loss, and discriminator loss is not involved. Then how is discriminator going to influence the PPO policy? Could you point out which segmentation of codes is doing this?

The second question is: in the loss calculation of discriminator,

    self.loss = (tf.losses.sigmoid_cross_entropy(tf.ones_like(policy_out), policy_out)
                 + tf.losses.sigmoid_cross_entropy(tf.zeros_like(expert_out), expert_out))

Is expert_out given a label as 0? From my understanding, expert should always be classified as label 1. Am I missing anything here?

I'm looking forward to your reply!