Vishu26/Optimization-Algorithms
Various Convex Optimization Algorithms
README
First Order Optimization Algorithms
1. Stochastic Weights Averaging
2. Batch Gradient Descent
3. Stochastic Gradient Descent
4. Mini Batch Stochastic Gradient Descent
5. Momentum
6. Nesterov Accelarated Gradient
7. AdaGrad
8. RMSProp
9. RMSProp with Nesterov Momentum
10. Adam
11. ADADelta
12. Linear Conjugate Gradient Method
13. Non Linear Conjugate Gradient Method
14. AdaMax
15. Nadam
16. AMSGrad
17. kSGD
18. Coordinate Descent
19. Polayak Averaging
Second Order Optimization Algorithms
1. Newton's Method
2. Quasi- Newton Method
3. Gauss-Newton Method
4. Broyden-Fletcher-Goldfarb-Shanno (BFGS)
5. Limited Memory BFGS
