shivambedi / dsc-2-23-14-section-recap-online-ds-ft-100118

Section Recap

Introduction

This short lesson summarizes the topics we covered in section 23 and why they'll be important to you as a data scientist.

Objectives

You will be able to:

• Understand and explain what was covered in this section
• Understand and explain why this section will help you become a data scientist

Key Takeaways

The key takeaways for this section include:

• The Kolmogorov-Smirnov Test can be used to test the "normality assumption" for a given data set
• More generally, the KS Test is a way to compare the similarity of two different distributions
• A one-sample KS Test (goodness of fit test) calculates the similarity between an observed data set and a theoretical distribution.
• A two-sample KS Test compares the similarity of two separate empirical distributions
• The make_blobs() method in sklearn is one way to generate a synthetic data set
• The make_moons() method allows for the generation of data for binary classification problems
• There are a number of other useful methods such as make_circles() and make_regression() for generating various types of data sets for testing your algorithms
• Resampling methods allow for improved precision in estimating sample statistics and validating models by using random subsets
• Common resampling techniques include bootstrapping, jackknifing and permutation tests
• Monte Carlo Simulations are a powerful tool for running large numbers of simulations with various inputs to provide distributions of possible output values