Make the system less chaotic & loss landscape less non-convex about causality-disentanglement-rl

[sergeivolodin]

Open questions

1. what is a good sparsity regularizer for the task? Sparse Sep? l1 is too brittle w.r.t. the coeff and the pruning is too restrictive. Maybe we need to use evolutionary algo to do the pruning? Somehow we need to consider "bringing back" pruned components...

2. how should we do training? one loss or many losses (with many optimizers)?

3. what is the number of constraints for n > 2?

4. what can we say from the general theory of differentiable games?

5. how does MuZero solve the problem of learning decoder + model? no details in the paper

6. can we solve the equation analytically somehow and see the number of fixed points (use Mathematica's Solve[...]?) -- should work as it's a fractional-polynomial equation

7. how can we make the training distributed so that we can use multi-start to battle non-convexity?

8. what does general theory says about it all (Polyak/Nesterov/Boyd/...)? -- write Polyak?

[sergeivolodin]

Some exercises from Gerstner's lectures:
https://lcnwww.epfl.ch/gerstner/VideoLecturesANN-Gerstner.html -> Week 3

Exercise 1: Sol_Lecture7.pdf

[sergeivolodin]

Johanni: small networks have local minima
Sergei: check if GANs break on 2 dimensions
Sergei: trying 5 and 3 for a start
Johanni: first, try 5 and then high-dimensional representation for observations

[sergeivolodin]

Johanni: GANs' loss is split so that D does not make G worse ("consensus algorithm instead of a battle")

Sergei: 3 optimizers (disentanglement, model fit, non-degeneracy)

Look at Goodfellow's talks with functional analysis -- why do we freeze G?
https://www.youtube.com/watch?v=HGYYEUSm-0Q

https://channel9.msdn.com/Events/Neural-Information-Processing-Systems-Conference/Neural-Information-Processing-Systems-Conference-NIPS-2016/Generative-Adversarial-Networks

[sergeivolodin]

Spiking networks -- built-in discreteness instead of l1

[nb from skype]

[sergeivolodin]

1. Try GAN with combined loss https://github.com/sergeivolodin/gans-playground . makes things worse
2. Try math in |XAX^{-1}|_0 -> min s.t. X non degenerate . NP-hard
3. Try all the loss splitting options in the optimizer and try more hyperparams

[sergeivolodin]

Splitting might be bad for efficiency.
https://discourse.julialang.org/t/a-hacky-guide-to-using-automatic-differentiation-in-nested-optimization-problems/39123/6

Goal for the week: understand what the problem is (formulate concisely and solve / send to ppl)

[sergeivolodin]

Related: https://en.wikipedia.org/wiki/Sparse_dictionary_learning

[sergeivolodin]

l1 projection + masking works ~ok in a toy setting (not the synth. experiment)

adapt? https://en.wikipedia.org/wiki/Matching_pursuit

Binary search for params in sparsity https://www.cs.virginia.edu/~bjc8c/class/cs6501-f18/papers/bhattacharya16sparsesep.pdf

[sergeivolodin]

Works with an observation model estimated separately, see the report