Map reduce code for Crime analyssi of Large POLICE data collected . Studied Performance and working of Hadoop HDFS
Cloud Computing - project1 Raja jaya chandra Mannem [email protected] Hadoop-1.1.0
Contents:
- Project1 Report.doc - Contains report of performances and jobs
- outputs.txt
- Job tracker log for single job.txt
- graphs.xlsx - contains performance graphs
- fsimage.txt - FS Image
- proj1 - project description
- data_distr.txt - Input data distribution.
- READ ME.txt - Readme file
- Crime - Folder contains source including hadoop libraries . java location - Crime\src\org\apache\hadoop\examples
IDE USED - Eclipse 1.Install hadoop plugin compatible with 1.1.0 in Eclipse IDE 2.Import java file to a new Hadoop project : File->import 3. Export the compiled code as Jar file , with class file linked to this jar . 4. Copy this jar file to a local unix directory. 5. SCP this jar file to Hadoop cluster scp Crime.jar rxm124330@cluster05master:/home/cloud2014/rxm124330/Crime.jar 6. Open the Hadoop cluster and run the jar file for input located at /user/cloud/full_input/single/
hadoop jar /home/cloud2014/rxm124330/Crime.jar /user/cloud/full_input/single/ /user/cloud2014/rxm124330/Crime/test1
- See the output at /user/cloud2014/rxm124330/Crime/test1 hadoop fs -cat /user/cloud2014/rxm124330/Crime/test1/part-r-00000
8.see the hadoop logs at /home/cloud/hadoop_logs/