MLE with Normal Distributions - Lab
Introduction
In this lab, we shall put in practice, the mathematical formulas we saw in previous lesson to see how MLE works with normal distributions.
Objectives
You will be able to:
- Understand and describe how MLE works with normal distributions
- Fit a normal distribution to given data identifying mean and variance
- Visually compare the fitted distribution vs. real distribution
Note: *A detailed derivation of all MLE equations with proofs can be seen at this website. *
MLE in Python
Let's see an example of MLE and distribution fittings with Python below. Here scipy.stats.norm.fit
calculates the distribution parameters using Maximum Likelihood Estimation.
Import necessary libraries
from scipy.stats import norm # for generating sample data and fitting distributions
import matplotlib.pyplot as plt
plt.style.use('seaborn')
import numpy as np
Generate an array of 200 random sample from a normal dist with mean 0 and stdv 1
sample = None
Distribution fitting through MLE
stats.norm.fit(data)
to fit a distribution to above data.
- Use - This returns a list of two parameters : (mean,: parameters[0] and std: parameters[1]) via a MLE approach
param = None
#param[0], param[1]
# (0.08241224761452863, 1.002987490235812)
x = np.linspace(-5,5,100)
Calculate the PDF from a) actual data parameters b) fitted parameters with x = np.linspace(-5,5,100)
# Generate the pdf from fitted parameters (fitted distribution)
fitted_pdf = None
# Generate the pdf without fitting (normal distribution non fitted)
normal_pdf = None
Visualize both PDFs
# Your code here
# Your comments/observations
Summary
In this short lab, we looked at Bayesian setting in a Gaussian context i.e. when the underlying random variables are normally distributed. We learned that MLE can estimate the unknown parameters of a normal distribution, by maximizing the likelihood of expected mean. The expected mean comes very close to the mean of a non-fitted normal distribution within that parameter space. We shall move ahead with this understanding towards learning how such estimations are performed in estimating means of a number of classes present in the data distribution using Naive Bayes Classifier.