mlh-fellowship / jamspam Goto Github PK

• Create a list of 50 Spam PRs (links) in the shared Google Sheet

• Add spam.csv to jam-spam-ml/data

• Initialise app in jam-spam-app with Probot
• Set up GitHub App on MLH-Fellowship Organization
• Flag off 🏁

Implement NLP to extract keywords from SPAM and HAM corpus.

A frequency vector of these keywords would be a great feature for our model. To make sure, we have keywords specific to SPAM and HAM characteristics of the PR, we decide to do the following.

N = complexity of the model (starting with 30, might change iteratively to achieve better results)

A = Top N keywords list from SPAM dataset
B = Top N keywords list from HAM dataset

SPAM_KEYWORDS = (A - B)
HAM_KEYWORDS = (B - A)

Suggest using multi-rake for rapid keyword extraction from corpus

• Create a list of 50 Ham PRs (links) in the shared Google Sheet

• Add ham.csv to jam-spam-ml/data

Create a script to scrape or possibly use APIs to get attributes of spam and ham PRs in order to build a complete dataset

Attributes

• PR Title
• PR Description
• Diffs
• Commit Messages
• .. discussion open for more attributes

• PR Title
• PR Body
• Diffs (actual changes)
• Commit Messages
• Files Changes
• Documentation File Changes (.md, .rst, .txt)
• Number of commits
• Diffs number (additions + deletions)

(Those ticked have been worked on in #12 (WIP) or earlier)

• Build Spam/Ham Classifier Model
• Decide on passing threshold for misclassification rate
• Test for Accuracy / Improve, if poor

Keep pull request open irrespective of contents if all the below conditions are met (skip spam detection)

• PR should have a linked issue.
• Linked issue should have label documentation
• Labelled issue should be assigned to the same user who has sent the Pull Request