todd-cook / ml-you-can-use

Practical ML and NLP with examples.

ML-You-Can-Use

Practical Machine Learning and Natural Language Processing with examples.

Featuring

• Interesting applications of ML, NLP, and Computer Vision
• Practical demonstration notebooks
• Reproducible experiments
• Illustrated best practices:
  • Code extracted from notebooks for:
    • automatic formatting with Black
    • Type checking via MyPy annotations
    • Linting via Pylint
    • Doctests whenever possible

Setup

Download this repo using git with the submodule command, e.g.:

git pull --recurse-submodules

Submodules are used to pull in some data and external data processing utilities that we'll use for preprocessing some of the data.

Install Python 3

Create Virtual Environment

mkdir p3
 `which python3` -m venv ./p3
 source setPythonHashSeed.sh
 source p3/bin/activate

Install Requirements

pip install -r requirements.txt

For running all notebook examples

pip install -r requirements-dev.txt

Note: some examples will have a conda environment.yaml file that you will want to use.

Installing Test Corpora

Many notebooks use data that needs to be installed, do so by running the install script.

install_corpora.sh

• installs Python ssl certificates
• installs CLTK data for Latin and Greek
• installs NLTK data

Testing

./runUnitTests.sh

Interactivity

juypter notebook

Notebooks

Getting data

• Extracting Occupation and Employer data from Wikidata

Labeling Data

• Labeling occupation data with Wikipedia and GoogleNews
• Correcting GoogleNews labels with Cleanlab
• Training to label with BERT and Cleanlab

Modeling Language

• Assessing Corpus Quality
• Making a Frequency Distribution
• Making a Word Trie Probability Model
• Word and Sentence Probability using BERT
• Comparing Collocation Extraction Methodologies

Detecting Duplicate Documents

• Merge corpora by detecting and filtering duplicate documents

Classifying Texts

• Benchmarking our classifier
• Boostrapping Document Classification

Detecting Loanwords

• Making a Frequency Distribution of Transliterated Greek
• Boosting Training Data
• The Problem of Loanwords, and a Solution
• Feature Engineering with the Loanwords matrix
• Detecting Loanwords with Keras

Wikipedia Corpus Processing

• English Wikipedia Corpus Cleaning
• English Wikipedia Corpus Processing
• Latin Corpus Processing
• Downsample or not

Quality Embeddings

• Generating an English Wikipedia word vector
• Generating a Latin word vector
• The Case for Using an Embedding Encoder
• Sentence Embeddings - A simple but effective baseline - using Seneca

Computer Vision - Object Detection

• Object detection as a multivariable regression using a custom Convnet
• Assessing the Noisy Circle detector

Summarizing Texts

• Assessing Headline Generation

Searching and Search Relevance

• Search Results Relevance using BERT

References and Acknowledgements

• A Sensitivity Analysis of (and Practitioners' Guide to) Convolutional Neural Networks for Sentence Classification by Ye Zhang, Byron Wallace
• Word2vec applied to Recommendation: Hyperparameters Matter byHugo Caselles-Dupré, Florian Lesaint, Jimena Royo-Letelier
• Exploiting Similarities among Languages for Machine Translation by Tomas Mikolov, Quoc V. Le, Ilya Sutskever
• Distributed Representations of Words and Phrases and their Compositionality by Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, Jeffrey Dean
• Deep Learning with Python by Francois Chollet
• Mining Massive Datasets
• Chris McCormick MinHash Tutorial with Python Code
• Convolutional Neural Networks for Text Classification by David S. Batista
• Convolutional Neural Networks for Sentence Classification by Yoon Kim
• The Unreasonable Effectiveness of Transformer Language Models in Grammatical Error Correction by Dimitris Alikaniotis, Vipul Raheja
• BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model by Alex Wang, Kyunghyun Cho
• An Exploration of Word Embedding Initialization in Deep-Learning Tasks by Tom Kocmi, Ondřej Bojar

ml-you-can-use's Issues

Training dataset

Hi @todd-cook!

I'm attempting to reproduce your results using BERT on the Crowdflower Search Results Relevance dataset, but I'm facing some issues. Your notebook says that the training set contains 20,571 labeled samples, but when I download the dataset from Kaggle it contains only 10,159 samples.

Am I missing something?

Why Sum of log probabilities

Why sometimes is the output of sum_log_probabilities positive and sometimes negative?
Log of probabilities are always negative because they are between 0 and 1; also sum of them must be negative, but why in some examples is this value positive?
in this link