This small project explored two approaches to improve the SAGA incremental gradient algorithm:
- Take gradients over mini-batches to reduce the memory requirement.
- Compute gradients in parallel on multiple CPU cores to speed it up.
See our small report for an exposition of the methods and some experimental results.
You'll also find a Python implementation of our mini-batch approach as well as a MATLAB implementation of our distributed approach.