15863004186sunchi / mongodb-datamining-shell

MongoDB shell implementation of the data mining algorithms

##MongoDB Data Mining Shell

MongoDB shell implementation of the data mining algorithms.

##Installation

git clone https://github.com/selvinsource/mongodb-datamining-shell.git
cd mongodb-datamining-shell
mongoimport --db mongodbdm --collection weatherData --type csv --headerline --file dataset/weatherData.csv
mongo mongodbdm --eval "var inputCollectionName = \"weatherData\", target = \"play\"" datamining/classification/oner.js
mongoimport --db mongodbdm --collection iris --type csv --headerline --file dataset/iris.csv
mongo mongodbdm --eval "var inputCollectionName = \"iris\", k = 3" datamining/clustering/kmeans.js

Follow this tutorial to compare the results to the Weka Data Mining Software.

##Documentation

Data mining or also called knowledge discovery is a set of activities aiming at analyzing large databases and extracting extra information meaningful for decision making or problem solving.

###Classification

Classification is one of the most common knowledge discovery task that consists in creating a model that predicts a target class based on explanatory variables.

####OneR OneR is a simple yet accurate classification algorithm that produces a one level decision tree.
For a visual description of the algorithm see OneR pseudocode.
Its oner.js MongoDB implementation takes as input two parameters:

• inputCollectionName - the collection used as training dataset
• target - the target class of the collection

Usage:

mongo yourdatabase --eval "var inputCollectionName = \"yourcollection\", target = \"yourtargetclass\"" datamining/classification/oner.js

Example of a collection and its target class play: weather data.

Limitation:

• the target class must be a categorical variable with values Yes and No
• the explanatory variables must be categorical variables, numerical variables should be discretized in a small number of distinct ranges before running the algorithm

###Clustering

Clustering is the task of identifying and segmenting the instances into a finite number (k) of categories (clusters) which are not predefined (unlike classification).

####K-Means

K-Means is the classic clustering technique that partitions the instances into k clusters whereas k is predefined.
For an high level description of the algorithm see K-Means pseudocode.
Its kmeans.js MongoDB implementation takes as input two parameters:

• inputCollectionName - the collection used as training dataset
• k - the number of predefined clusters

Usage:

mongo mongodbdm --eval "var inputCollectionName = \"yourcollection\", k = numberofclusters" datamining/clustering/kmeans.js

Example of a collection: iris data.

Limitation:

• the variables must be all numerical

Note:

• If a field in the collection is called "class", this is excluded from the computation, instead it will be printed in the result with the assigned cluster

###References

• Hartigan, J. A. (1975) Clustering Algorithms, Probability & Mathematical Statistics, John Wiley & Sons Inc.
• Holte, R. C. (1993) Very simple classification rules perform well on most commonly used datasets, Machine Learning, 11, pp 63-91
• Selvaggio, V. (2011) Customer Churn prediction for an Automotive Dealership using computational Data Mining, MSc dissertation, City University London
• UCI Machine Learning Repository University of California, School of Information and Computer Science