YWBAT
- define linear regression
- create an example of linear regression
- describe what the various parts of the linreg equation do
- calculate the error of a linear regression equation
What is linear regression?
![](linregimage.png)
Let's make an example with some data!!!!!
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
plt.xkcd()
<contextlib._GeneratorContextManager at 0x1c20065e80>
x_vals = np.linspace(0, 100, 50)
slope = np.random.randint(20, 50)
errors = np.random.normal(100, 200, 50)
bias = np.random.randint(20, 200)
y_vals = slope*x_vals + bias + errors
plt.scatter(x_vals, y_vals)
plt.xlabel("x values")
plt.ylabel("y values")
plt.title("data points")
plt.show()
![png](lesson-plan_files/lesson-plan_8_0.png)
Let's check the correlation coefficient
np.corrcoef(x_vals, y_vals)
array([[1. , 0.98366979],
[0.98366979, 1. ]])
### let's just guess a slope
slope_guess = 21
bias_guess = 0
### this yields a y_hat array of
y_hat = slope_guess*x_vals + bias_guess
plt.scatter(x_vals, y_vals)
plt.plot(x_vals, y_hat, c='r', label='y-hat')
plt.xlabel("x values")
plt.ylabel("y values")
plt.title("data points")
plt.legend()
plt.show()
![png](lesson-plan_files/lesson-plan_13_0.png)
![](rmse.png)
### Yikes! How bad is this?
### Let's create our RMSE equations
def RMSE(y_true, y_pred):
num = np.sum((y_true-y_pred)**2)
den = np.sum((y_true - y_true.mean())**2)
return 1 - 1.0*num / den
def RMSE2(y_true, y_pred):
num = np.sum((y_pred - y_true.mean())**2)
den = np.sum((y_true - y_true.mean())**2)
return 1.0 * num / den
which one will python use? Let's import from sklearn.metrics
from sklearn.metrics import r2_score
now, how can we do this using statsmodels?
import statsmodels.api as sm
x = sm.add_constant(x_vals)
linreg = sm.OLS(y_vals, x).fit()
summary = linreg.summary()
summary
OLS Regression Results
Dep. Variable: | y | R-squared: | 0.968 |
Model: | OLS | Adj. R-squared: | 0.967 |
Method: | Least Squares | F-statistic: | 1434. |
Date: | Tue, 07 May 2019 | Prob (F-statistic): | 2.07e-37 |
Time: | 21:19:13 | Log-Likelihood: | -331.45 |
No. Observations: | 50 | AIC: | 666.9 |
Df Residuals: | 48 | BIC: | 670.7 |
Df Model: | 1 | | |
Covariance Type: | nonrobust | | |
| coef | std err | t | P>|t| | [0.025 | 0.975] |
const | 53.5332 | 52.073 | 1.028 | 0.309 | -51.167 | 158.233 |
x1 | 33.9789 | 0.897 | 37.865 | 0.000 | 32.175 | 35.783 |
Omnibus: | 2.110 | Durbin-Watson: | 1.886 |
Prob(Omnibus): | 0.348 | Jarque-Bera (JB): | 1.445 |
Skew: | 0.172 | Prob(JB): | 0.486 |
Kurtosis: | 2.242 | Cond. No. | 114. |
# how close were we?
slope, bias
ols_slope, ols_bias = linreg.params
ols_y_hat = ols_slope*x_vals + ols_bias
ols_slope, ols_bias
(53.53323559323411, 33.97888956550166)
plt.scatter(x_vals, y_vals)
plt.plot(x_vals, ols_y_hat, c='r', label='ols y-hat')
plt.xlabel("x values")
plt.ylabel("y values")
plt.title("data points")
plt.legend()
plt.show()
![png](lesson-plan_files/lesson-plan_29_0.png)
best_y_hat = slope*x_vals + bias
plt.scatter(x_vals, y_vals)
plt.plot(x_vals, best_y_hat, c='r', label='best y-hat')
plt.xlabel("x values")
plt.ylabel("y values")
plt.title("data points")
plt.legend()
plt.show()
![png](lesson-plan_files/lesson-plan_31_0.png)