Each student will be assigned a problem and an algorithm to implement on this problem. We will use the database MNIST http://yann.lecun.com/exdb/mnist/ where the goal is to classify digits between 0 and 9.
- Each student chooses a model and an algorithm to determine its parameters.
- By keeping part of the training set into a validation set, find a good value for the hyperparameters of the model.
- We will compare models and optimization algorithms in terms of classification performance, quality of the local optima returned by each method, the speed of convergence.
- The Chosen Model: The Multinomial logistic regression with squared 2-norm regularization .
- The Chosen Algorithm Stochastic Gradient Descent (SGD)