learn-co-students / dsc-4-42-01-introduction-seattle-ds-career-040119

Introduction

Introduction

This lesson summarizes the topics we'll be covering in section 42 and why they'll be important to you as a data scientist.

Objectives

You will be able to:

• Understand and explain what is covered in this section
• Understand and explain why the section will help you to become a data scientist

Regularization and Optimization

In the previous section you have been given quite some insight on how neural networks work, how deeper neural networks can lead to better model performance, and how to build neural networks in Keras. Now that we've covered the concept behind neural networks as well as some streamlined methods for building such models, you will begin to further explore concepts involved in tuning and optimizing the performance of these networks.

Tuning Neural Networks with Regularization

You've learned about the bias variance trade-off, and in this section you'll see these principles applied and explained in the deep learning contex. You'll learn how to use a training, test and validation set and why this 3-way split is particularly important in the deep learning context. Next, you'll learn about some regularization techniques which are particularly heplful when overfitting occurs. We'll explain the concepts of L1 and L2 regularization as well as dropout regularization. Next, you'll learn how to apply all these using Keras.

Normalization and Tuning Neural Networks

In this section, more neural networks concepts for optimization and training time reduction will be introduced. One of them is normalization of inputs: we've already seen this in several machine learning models, but input normalization in deep learning models can drastically decrease computation time, mitigate common issues such as vanishing or exploding gradients, and increase model performance. Next, you'll learn more about several optimization algorithms which can be used to tune neural networks. We'll also give you an overview of the hyperparameters that need to be tuned in neural networks.

Summary

In this section, you'll extend your deep learning knowledge by learning about regularizing and optimizing your neural network models.