This project combines classification and regression tasks to determine whether a team should run, pass, punt, or kick a field goal on 4th down, emulating the duties of an NFL coach.
# clone project
git clone https://github.com/lderr4/Robo-NFL-Coach.git
# Switch to project directory
cd Robo-NFL-Coach
# Setup Python3.8 Virtual Env
python3.8 -m venv env_name
source env_name/bin/activate
pip install -r requirements.txt
# Run the Training Script (which saves the Models, Dataset, and Class):
python3 robo_coach.py
- Nailing down the correct approach to a problem (before writing any code) is the key to avoiding time waste
- Importance of feature engineering and preprocessing
- Recursive Feature Elimination and Bayesian Optimization
- Cleaning up messy Jupyter Notebook code into usable Python Modules
- Visual evaluation metrics are often more informative than scores
- Coming up with the approach to the question: What is the optimal 4th Down Play?
- Going from ~300 features in the raw dataset to ~30
- Poor Performance of Regression Models
- Evaluating the entire system (as apposed to each of its components separately)
This diagram depicts the flow of data from the initial API request to the final loading of each model. Data is pulled from the nfl_data_py library. In total, five models are trained:
- Classifier: a multi-class Decision Tree Classifier which classifies fourth down plays as Pass, Run, Punt, or Field Goal.
- Pass Regressor: Predict the change in win probability (Win Probability Added) for Passing plays on 4th down
- Run Regressor: Predict the Win Probability Added for Run plays on 4th down
- Punt Regressor: Predict the Win Probability Added for Punt plays on 4th down
- Field Goal Regressor: Predict the Win Probability Added for Field Goal plays on 4th down
This diagram illustrates the decision flow chart for the predict function of the robo coach class. The classifier's predict_proba function is used to determine the degree of certainty of the prediction. If the probability of the highest play is above the max probability threshold parameter, the classifier will be used for the prediction. Otherwise, for each play probability that is greater than the minimum probability threshold parameter, the corresponding regression model is used to predict Win Probability Added. The play with the highest Win Probability Added is chosen.
Feature correlation was one of the feature selection techniques used in this project.
Notice the proportion of passing plays shooting up at about the 1% mark. This indicates lots of 4th down plays have ~1% probability of being a passing play. This is because of fake punts and trick plays which happen in typical punt or field goal situations.
Metric | Score |
---|---|
0.525 | |
MSE | 0.00143 |
MAE | 0.0257 |
Correct Sign (+/-) % | 0.882 |
Metric | Score |
---|---|
0.392 | |
MSE | 0.00275 |
MAE | 0.0266 |
Correct Sign (+/-) % | 0.865 |
Metric | Score |
---|---|
-0.0573 | |
MSE | 0.00291 |
MAE | 0.0247 |
Correct Sign (+/-) % | 0.766 |
Note: Despite my best efforts, the field goal model still has a negative
Metric | Score |
---|---|
0.339 | |
MSE | 0.000908 |
MAE | 0.0151 |
Correct Sign (+/-) % | 0.742 |