dborrelli / ds-resources

Data Science Resources

Data Science Resources

This list started out as a way for me to keep track of data science resources I've found helpful. However, I frequently get asked for data science resource recommendations by other data scientists and friends looking to break into data science. So I've continued to add to this, with a focus on beginner- and intermediate-level resources. Where possible, I've included links to the (legitimate) free versions of books. One of the great things about the data science community is the willingness to open-source and make things available for free. Within each category or sub-category the resources are listed very loosely in order of usefulness/introductory level to more advanced (but not entirely).

This list is far from complete, but I'll try to continue to add to it. Hopefully you find it helpful.

Non-exhaustive list of additional topics to add:

• Spark
• time series forecasting
• docker

Technical Resources

1. Foundational

1.1 Python

• course Coursera - Introduction to Data Science in Python
• course codecademy - Learn Python 3
• ebook Python Like You Mean It
• book Automate the Boring Stuff with Python
• book Python for Everybody
• book Learn Python the Hard Way maybe not my favorite resource, but was still useful
• book Python Data Science Handbook
• video series Calm Code
• Google Python style guide

1.2 Statistics

• course Khan Academy - Statistics
• Stanford Experimental Design course and course notes
• course Coursera - Statistics with Python Specialization
• book Open Intro Statistics
• book Introduction to Empirical Bayes by David Robinson
• book Think Bayes
• book Think Stats

1.3 SQL

• Mode SQL tutorial
• SQL Zoo

1.4 Computer Science, data structures and algorithms

• course - MIT OCW - Introduction to Computer Science and Programming in Python
• ebook Problem Solving with Algorithms and Data Structures using Python
• course Khan Academy - Algorithms
• course Coursera - Algorithms Specialization
• course - MIT OCW - Introduction to Algorithms
• HackerRank 30 days of code
• github repo Awesome Algorithms
• book Introduction to Algorithms by Cormen, Leiserson, Rivest and Stein
• article Learn X in Y minutes: Bash
• article Bash scripting cheatsheet

2. General ML

2.1 ML overview

• course Coursera - Machine Learning by Andrew Ng foundational knowledge of machine learning
• course Applied Data Science with Python Specialization more immediately applicable than the previous course
• book An Introduction to Statistical Learning with Applications in R (ISLR), 2nd edition by James, Witten, Hastie, Tibshirani
• book The Hundred Page Machine Learning book
• book Approaching (Almost) Any Machine Learning Problem
• book Mining of Massive Datasets and course edX/Stanford - Mining Massive Datasets
• article series Machine Learning Mastery
• article How to Train a Final Machine Learning Model on Machine Learning Mastery
• book (advanced material) Probabilistic Machine Learning: An Introduction by Kevin Murphy
• book (advanced material) Elements of Statistical Learning
• Papers With Code
• ArXiv Sanity Preserver

2.2 University ML courses

• course - Harvard CS 109 Data Science
• course - Cornell CS 4780 Machine Learning lecture notes and lecture youtube videos
• course - MIT Intro to Machine Learning
• course - Wisconsin Machine Learning Sebastian Raschka

2.3 Dimensionality reduction

• paper UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction by McInnes et al.
• blog Understanding UMAP by Andy Coenen and Adam Pearce
• article How Exactly UMAP Works
• paper Visualizing Data using t-SNE by van der Maaten and Hinton
• blog How to Use t-SNE Effectively

2.4 Clustering

• blog Visualizing DBSCAN by Naftali Harris
• API documentation How HDBSCAN Works
• paper Accelerated Hierarchical Density Clustering by McInnes and Healy, 2017
• blog Understanding HDBSCAN and Density-Based Clustering by Pepe Berba
• stackoverflow How to select a clustering method? How to validate a cluster solution?
• stackoverflow Evaluation measures of goodness or validity of clustering
• paper What are the true clusters? by Christian Henning
• paper Density-Based Clustering Validation by Moulavi et al, 2014

2.5 Curse of dimensionality

• paper On the Surprising Behavior of Distance Metrics in High Dimensional Space by Aggarwal et al., 2001
• blog Escaping the Curse of Dimensionality by Peter Gleeson (FreeCodeCamp)

2.6 Data issues

• article Learning from Imbalanced Classes

3. ML in production

• github repo Curated papers, articles, and blogs on data science & machine learning in production
• course Stanford CS 329S: Machine Learning Systems Design
• article Overview of the different approaches to putting Machine Learning (ML) models in production
• article A Practical Guide to Maintaining Machine Learning in Production

4. MLOps

• github repo and tutorials Made With ML by Goku Mohandas
• github repo Awesome ML Ops
• article ML Ops: Machine Learning as an Engineering Discipline

5. Deep Learning

5.1 General DL

• course Coursera - deeplearning.ai Deep Learning Specialization
• book Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow and its associated github repo (the first ~200 pages are about general ML so this book could go under that section, but it's probably better suited for someone looking to learn about DL)
• course Google's Machine Learning Crash Course
• site Neural Network Playground
• blog post The Unreasonable Effectiveness of Recurrent Neural Networks by Andrej Kaparthy

5.2 University DL courses

• github repo Deep Learning Drizzle giant list of university DL courses
• course Stanford CS230 - Deep Learning
• course Stanford CS231n - Convolutional Neural Networks for Visual Recognition
• course Yann LeCun's NYU course - DS-GA 1008 · SPRING 2020
• course MIT Intro to Deep Learning

5.3 DL papers

• github repo Deep Learning Papers Reading Roadmap
• paper Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification by He et al, 2015
• paper A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay by Smith, 2018

5.4 TensorFlow

• book Deep Learning with Python, 2nd edition by François Chollet
• course Coursera Deeplearning.AI Tensorflow Developer Professional Certificate

5.5 PyTorch

• book Deep Learning with PyTorch

5.6 Reinforcement Learning

• course Coursera - Reinforcement Learning Specialization
• book Reinforcement Learning by Sutton and Barto

5.7 Graph Neural Networks

• article A Gentle Introduction to Graph Neural Networks

6. NLP

6.1 NLP overview

• course Coursera - deeplearning.ai Natural Language Processing Specialization
• course CS224n: Natural Language Processing with Deep Learning
• course Advanced NLP with spaCy
• course Hugging Face course
• book Natural Language Processing with Python by Bird, Klein and Loper
• course Michigan NLP course videos and github
• article FROM Pre-trained Word Embeddings TO Pre-trained Language Models — Focus on BERT
• book Speech and Language Processing (3rd ed. draft) by
• article How to get started in NLP

6.2 Embeddings

• article Introduction to Word Embeddings
• article Document Embedding Techniques
• paper word2vec: Efficient Estimation of Word Representations in Vector Space by Mikolov et al.
• paper GloVe: GloVe: Global Vectors for Word Representation by Pennington et al. and Stanford webiste for GloVe
• paper fastText: Bag of Tricks for Efficient Text Classification by Joulin et al.
• paper Universal Sentence Encoder by Cer et al., 2018

6.3 Topic modeling

• paper LDA: Latent Dirichlet Allocation by Blei et al.
• paper Anchored CorEx Anchored Correlation Explanation: Topic Modeling with Minimal Domain Knowledge by Gallagher et al., 2017 and github
• github Top2Vec and paper Top2Vec: Distributed Representations of Topics by Dimo Angelov
• github BERTopic and article Topic Modeling with BERT by Maarten Grootendorst
• blog post - StitchFix - Introducing our Hybrid lda2vec Algorithm by Chris Moody

6.4 Transformers

• paper transformers Attention Is All You Need by Vaswani et al, 2017
• article The Illustrated GPT-2 (Visualizing Transformer Language Models by Jay Alammar
• article How GPT3 Works - Visualizations and Animations by Jay Alammar

7. Experimentation

7.1 A/B testing

• book Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing by Kohavi, et al.
• course Microsoft Experimentation Platform
• Evan Miller's A/B test tools
• paper Three Key Checklists and Remedies for Trustworthy Analysis of Online Controlled Experiments at Scale
• paperTop Challenges from the first Practical Online Controlled Experiments Summit
• paper Controlled experiments on the web: survey and practical guide Kohavi et al, 2009
• article Guidelines for A/B Testing
• article A/B Testing: 29 Guidelines for Online Experiments (Plus a Checklist)
• course Udacity - A/B testing by Google
• article How Not To Run an A/B Test by Evan Miller
• article Simple Sequential A/B testing by Evan Miller
• paper Overlapping Experiment Infrastructure: More, Better, Faster Experimentation by Tang et al, 2010
• blog post A/B Testing Tutorial
• paper Controlled experiments on the web: survey and practical guide by Ron Kohavi et al
• presentation Online Controlled Experiments: Lessons from Running A/B/n Tests for 12 years by Ron Kohavi
• articles Microsoft's Experimentation Platform

7.2 Bayesian A/B testing

• article Understanding Bayesian A/B testing by David Robinson
• article Is Bayesian A/B Testing Immune to Peeking? Not Exactly by David Robinson
• article Agile A/B testing with Bayesian Statistics and Python by Chris Stucchio
• article The Power of Bayesian A/B Testing

7.3 Multi-Armed Bandits (MAB)

• paper Best arm identification in multi-armed bandits with delayed feedback
• paper Generalized Thompson Sampling for Contextual Bandits
• paper Analysis of Thompson Sampling for the Multi-armed Bandit Problem
• paper A Contextual-Bandit Approach to Personalized News Article Recommendation
• article A/B testing — Is there a better way? An exploration of multi-armed bandits

8. Coding best practices

8.1 GitHub

• course Coursera - Introduction to Git and GitHub
• ebook Pro Git

8.2 Structuring projects

• article Structuring Your Project: The Hitchhiker's Guide to Python
• github Cookiecutter data science
• blog post How to Set Up a Python Project For Automation and Collaboration by Eugene Yan

8.3 Code refactoring workflow

• article The importance of structure, coding style, and refactoring in notebooks
• tutorial Production Data Science
• article Coding habits for data scientists

8.4 Unit testing

• article Effective Python Testing With Pytest Real Python
• article Becoming a Better Data Scientist: Testing with pytest by Chang Hsin Lee
• article Unit Testing for Data Scientists

8.5 Creating PyPI packages

• tutorial PyPA Packaging Python projects tutorial
• e-book Python Packages e-book
• e-book The Joy of Packaging
• poetry
• article How to Build Your First Python Package

9. Helpful tools and packages

9.1 AWS

• course Coursera - Getting Started with AWS Machine Learning
• course Coursera - AWS Cloud Technical Essentials
• course Coursera - Practical Data Science Specialization
• AWS Ramp up guide

9.2 Flask

• Flask Mega-tutorial by Miguel Ginberg

9.3 Hyperopt

• article Parameter Tuning with Hyperopt by District Data Labs
• article On Using Hyperopt: Advanced Machine Learning by Tanay Agrawal
• article An Introductory Example of Bayesian Optimization in Python with Hyperopt by Will Koehrsen

10. Datasets

10.1 General

• Awesome Public Datasets
• Google Dataset search
• Registry of Open Data on AWS

10.2 NLP

• huggingface datasets
• nlp-datasets github repo

10.3 Time series

• UCR Time Series Classification Archive
• Time Series Classification Repository

11. Domain applications

11.1 Rewewable Energy

• paper Tackling Climate Change with Machine Learning
• ClimateChage AI

11.2 Healthcare

• course MIT OCW Machine Learning for Healthcare and lecture videos

12. Additional topics

12.1 Ethics

• github repo EthicalML Awesome Production ML

12.2 Bias and explanability

• package Fairlearn

13. Other learning resource lists

• article Data science learning resources by Microsoft Data Science team
• blog End-to-End Machine Learning by Brandon Rohrer (some good free resources, some paid)
• github repo Awesome Machine Learning
• blog Free online machine learning curriculum by Chip Huyen

14. Industry resources and trends

14.1 Company tech blogs

• AirBnb
• Coursera
• Stitch Fix
• Square Product Analytics at Square

14.2 Newsletters

• Data Science Weekly
• Papers With Code Newsletter

14.3 Podcasts

• Data Skeptic
• Linear Digressions
• Talking Machines

Career resources

Career advice

• book Build a Career in Data Science by Emily Robinson and Jacqueline Nolis
• article 80000 hours: Data Science career review
• Quora As a data scientist, what career advice changed your life?
• blog post A Framework for Career Decisions by Conor Dewey
• ApplyingML - Mentor interviews by Eugene Yan
• talks by Angela Bassa

Defining data science

• blog post Applied / Research Scientist, ML Engineer: What’s the Difference? by Eugene Yan
• reddit Difference between DS and MLE
• article Machine Learning Engineer vs Data Scientist (Is Data Science Over?)

Becoming a data scientist

• Open-Source Data Science Masters
• article How to Build a Data Science Portfolio
• github Awesome Data Science

Generalist vs specialist

• article Unpopular Opinion - Data Scientists Should be More End-to-End by Eugene Yan
• article - Stitch Fix Beware the data science pin factory: The power of the full-stack data science generalist and the perils of division of labor through function by Eric Colson

IC vs Management and career progression

• blog post Finding Answers to your Career Questions
• blog post Engieering Management: The Pendulum or the ladder by Charity Majors
• blog post The Engineer/Manager Pendulum by Charity Majors
• blog post Senior engineer and then what? by Ju Yang

Team structure

• article Models for integrating data science teams within organizations
• article - Coursera Analytics at Coursera: three years later
• article - Coursera What is the most effective way to structure a data science team?
• article - AirBnB At Airbnb, Data Science Belongs Everywhere
• article Embedding Data Science In Cross-Functional Teams

Data-driven culture

• blog Building a data team at a mid-stage startup: a short story by Erik Bernhardsson
• blog - StitchFix Let Curiosity Drive: Fostering Innovation in Data Science

Interviewing

• book Introduction to Machine Learning Interviews Book by Chip Huyen
• article Data science career advice to my younger self by Schaun Wheeler
• blog post How to Break Into the Tech Industry—a Guide to Job Hunting and Tech Interviews by Haseeb Qureshi
• article Mastering the Data Science Interview Loop by Andrei Lyskov
• blog post Reverse Interviewing Your Future Manager and Team by Gergely Orosz
• blog post Red Flags to Look Out for When Joining a Data Team by Eugene Yan
• blog post Red Flags in Data Science Interviews by Emily Robinson

Non-technical resources

Agile & Project management

• article How to manage Machine Learning and Data Science projects
• article Data Science and Agile (What works, and what doesn't) and Data Science and Agile (Frameworks for effectiveness)

Product

• article Jobs To Be Done Framework
• article series - Sequoia Data-Informed Product Building
• article Sequoia Data Science Team Measuring Product Health
• article Sequoia Data Science Team Retention

Business

• article 10 Reads for Data Scientists Getting Started with Business Models by Conor Dewey