harryzhangog / deep-rl-notes Goto Github PK

Hi,

First of all, thanks so much for the notes! They are extremely useful.

I just wanted to point out a few typos:

1. Last equation of Ch.7 (step size): missing J(\theta) after last grad on the denominator. Link
2. Algorithm 14 (DDPG), page 34: line 4 should not have a \max_a in front of Q (since we are finding the best state-action function by training a NN to learn the best action \mu_{\theta}). Also, although this is not a typo per se, I think after the summation in line 6 there should be a \frac{d \mu_{\theta}}{d \theta} rather that \frac{d a}}{d \theta}, as that "a" in that derivative is the output from the NN above. Link
3. Algorithm 5 (Online AC), page 22: line 5 does not have a \sum_i as we are updating one value at a time. Link

I hope this helps! Thanks!
Javier.

Hi Harry,

Thanks for taking these notes! They are clear and very helpful. I found a typo in 2.3.1, which should be $\pi(a_t | o_1, ..., o_t)$ rather than $\pi(a_t | o+1, ..., o_t)$

It should be a easy fix in L20 in https://github.com/harryzhangOG/Deep-RL-Notes/blob/master/imitation.tex

Thanks a lot!

The first equation seems to have its first parenthesis in the wrong place: https://youtu.be/VgdSubQN35g?list=PL_iWQOsE6TfU9DwANRsUZf0YUUJS_ySr0&t=152

Hi Harry, great note! Just found some small typos in chapter 10:

1. The sigma (page 60)

$p(x|z) = \mathcal{N}(\mu_{nn}(z),\mu_{nn}(z))$

And the correct should be this:

$p(x|z) = \mathcal{N}(\mu_{nn}(z),\sigma_{nn}(z))$

2. The theta display (page 60)

[
theta\leftarrow \argmaxA_\theta\frac{1}{N}\sum_i\mathbb{E}{z\sim p(z|x_i)}\log p\theta(x_i)
]

Hi, thanks for your great notes.
I just found a typo in Chapter 10.1.1. :
The KL-divergence $D_{KL}(q_i(x_i))||(p(z|x_i)$.
And the correct should be this: $D_{KL}(q_i(z)||p(z|x_i))$?