why rnnlm tool doesn't have parameter for syn0 --> hidden about kaldi

[danpovey]

Ilya edrenkin wrote the RNNLM tool. I don't understand what the question
means.
Ilya, if you could get a github account it would be helpful.
Dan

On Thu, Aug 27, 2015 at 8:41 AM, fedorajzf [email protected] wrote:

—
Reply to this email directly or view it on GitHub
#97.

[ilya-eder]

Here I am.
However I also don't get the question, frankly. syn0 matrix is used for word embeddings, they are just learned. What do you mean by a parameter in this context?

[danpovey]

This is more of a discussion than a real issue; closing the issue.

[fedorajzf]

hi, ilya
in rnnlm.c，hidden value is stored in neu1, neu1[t] = sigmoid(synRec * neu1[t-1] + syn0)
should it be neu1[t] = sigmoid(synRec * neu1[t-1] + syn0 * U) ? here U is another parameter for syn0 to hidden layer.

[ilya-eder]

It might be the case that carefully choosing this U would improve convergence; however I'd expect from the training procedure to learn the optimal scale for syn0 without making it a parameter.

[fedorajzf]

if not use U, hidden layer dimension must equal embedding dimension, maybe this is a problem