theoviel / kaggle_quora

Solution for Kaggle's Quora Insincere Questions Classification competition

Kaggle : Quora Insincere Questions Classification

Introduction

On the competition

An existential problem for any major website today is how to handle toxic and divisive content. Quora wants to tackle this problem head-on to keep their platform a place where users can feel safe sharing their knowledge with the world. Quora is a platform that empowers people to learn from each other. On Quora, people can ask questions and connect with others who contribute unique insights and quality answers. A key challenge is to weed out insincere questions -- those founded upon false premises, or that intend to make a statement rather than look for helpful answers. In this competition, Kagglers will develop models that identify and flag insincere questions. To date, Quora has employed both machine learning and manual review to address this problem. With your help, they can develop more scalable methods to detect toxic and misleading content. Here's your chance to combat online trolls at scale. Help Quora uphold their policy of “Be Nice, Be Respectful” and continue to be a place for sharing and growing the world’s knowledge.

See https://www.kaggle.com/c/quora-insincere-questions-classification/overview

The metric was the F1-Score, as the problem was an unbalanced binary classification one. Note that all the training had to be made in the kaggle kernels, in less that 2 hours. No external data nor pretrianed models were allowed.

The competition took place from November, 6 2018 to February, 14 2019.

Results

This competiton was the first one I really invested in. I did it solo, and ended up 26th out of 4037. My best model achieved 0.700 on the public leaderboard, which ranked about 400th, but the 0.688 CV model I selected was robust enough to perform well on the private leaderboard.

Work overview

Preprocessing

• Some special characters cleaning
• Number processing
• Contractions & mispells replacement
• Latex tags cleaning.
• No lowering though

Embeddings

Concatenation of glove, fasttext and paragram. 4 embeddings were made available by the organisers, I kept those three.

Features

• Toxic words ratio
• Total length
• word vs unique words
• ratio of capital letters

Model

Single model, 5 folds, 4 epochs :

• Embedding layer + some noise
• LSTM, 64 Units (unidirectional)
• GRU, 32 Units (unidirectional)
• Attention, maxpool & average pool on the outputs of both rnns
• Concatenating them with features
• 32 units dense + reLu + Batchnorm + Dropout
• And the final layer

Repository

• solution_notebook : The exact notebook I submitted, and that scored 26th. Code is a bit dirty and easily improvable.
• cleaned_solution_notebook : Modification of the solution notebook (15/07/2019)
• capgemini_slides : Work presented to Data Scientists at Capgemini Invent. If you want to understand my code, this notebook is a bit more commented. The html slideshow is also available.

Both notebooks are available on Kaggle

Data

Data can be downloaded on the official Kaggle page. If you wish to rerun the notebook, the easiest way is to fork the Kaggle kernel

Additional work

Here are some kernels I made public during the competiton :

• Text processing for embeddings
• Text processing for embeddings with performance comparison
• Augmenting insincere texts with word embeddings
• Augmenting insincere texts with SMOTE
• Applying usual cleaning methods to our problem