crimeanalysis-mapreduce's Introduction

CrimeAnalysis-MapReduce

Map reduce code for Crime analyssi of Large POLICE data collected . Studied Performance and working of Hadoop HDFS

Cloud Computing - project1 Raja jaya chandra Mannem [email protected] Hadoop-1.1.0

Contents:

Project1 Report.doc - Contains report of performances and jobs
outputs.txt
Job tracker log for single job.txt
graphs.xlsx - contains performance graphs
fsimage.txt - FS Image
proj1 - project description
data_distr.txt - Input data distribution.
READ ME.txt - Readme file
Crime - Folder contains source including hadoop libraries . java location - Crime\src\org\apache\hadoop\examples

IDE USED - Eclipse 1.Install hadoop plugin compatible with 1.1.0 in Eclipse IDE 2.Import java file to a new Hadoop project : File->import 3. Export the compiled code as Jar file , with class file linked to this jar . 4. Copy this jar file to a local unix directory. 5. SCP this jar file to Hadoop cluster scp Crime.jar rxm124330@cluster05master:/home/cloud2014/rxm124330/Crime.jar 6. Open the Hadoop cluster and run the jar file for input located at /user/cloud/full_input/single/

hadoop jar /home/cloud2014/rxm124330/Crime.jar /user/cloud/full_input/single/ /user/cloud2014/rxm124330/Crime/test1

See the output at /user/cloud2014/rxm124330/Crime/test1 hadoop fs -cat /user/cloud2014/rxm124330/Crime/test1/part-r-00000

8.see the hadoop logs at /home/cloud/hadoop_logs/

Recommend Projects

mannem / crimeanalysis-mapreduce Goto Github PK