anilesh-prajapati / data-structures-algorithms-and-machine-learning-optimization

Data Structures, Algorithms and Machine Learning Optimization

Data Structures, Algorithms and Machine Learning Optimization

This repo is home to the code that accompanies Data Structures, Algorithms and Machine Learning Optimization curriculum, which provides a comprehensive overview of all of the subjects — across Data Structures, Algorithms and Machine Learning Optimization — that underlie contemporary machine learning approaches, including deep learning and other artificial intelligence techniques:

Algorithms & Data Structures Curriculum

• Segment 1: Introduction to Data Structures and Algorithms

  • A Brief History of Data
  • A Brief History of Algorithms
  • “Big O” Notation for Time and Space Complexity

• Segment 2: Lists and Dictionaries

  • List-Based Data Structures: Arrays, Linked Lists, Stacks, Queues, and Deques
  • Searching and Sorting: Binary, Bubble, Merge, and Quick
  • Set-Based Data Structures: Maps and Dictionaries
  • Hashing: Hash Tables, Load Factors, and Hash Maps

• Segment 3: Trees and Graphs

  • Trees: Decision Trees, Random Forests, and Gradient-Boosting (XGBoost)
  • Graphs: Terminology, Directed Acyclic Graphs (DAGs)
  • Resources for Further Study of Data Structures & Algorithms

Machine Learning Optimization Curriculum

• Segment 1: The Machine Learning Approach to Optimization

  • The Statistical Approach to Regression: Ordinary Least Squares
  • When Statistical Approaches to Optimization Break Down
  • The Machine Learning Solution

• Segment 2: Gradient Descent

  • Objective Functions
  • Cost / Loss / Error Functions
  • Minimizing Cost with Gradient Descent
  • Learning Rate
  • Critical Points, incl. Saddle Points
  • Gradient Descent from Scratch with PyTorch
  • The Global Minimum and Local Minima
  • Mini-Batches and Stochastic Gradient Descent (SGD)
  • Learning Rate Scheduling
  • Maximizing Reward with Gradient Ascent

• Segment 3: Fancy Deep Learning Optimizers

  • A Layer of Artificial Neurons in PyTorch
  • Jacobian Matrices
  • Hessian Matrices and Second-Order Optimization
  • Momentum
  • Nesterov Momentum
  • AdaGrad
  • AdaDelta
  • RMSProp
  • Adam
  • Nadam
  • Training a Deep Neural Net
  • Resources for Further Study