Bug: running optimize() multiple times produces different result compared to running it once about opentsne

[dkobak]

Hmm, does not look like it's due to gains: if I use 2 iterations instead of 100 in the snippet above, then E1 and E2 are still not identical, yet E1.optimizer.gains and E2.optimizer.gains are the same.

[dkobak]

So I figured out what is going on here. update vectors are not save between the optimize() calls, so momentum term has no effect if one uses optimize(n_iter=1). I guess it's debatable what is better, but I would say that if we keep the gains between the optimize() calls, then we should also keep the update vectors.

I prepared a quick PR that implements that but unfortunately it makes some tests fail, and I could not fix it yet.

[pavlin-policar]

Thanks for tracking this down. This is definitely a bug. Calling optimize once with iter=100 should definitely be the same as calling optimize 100 times with iter=1.