$ hadoop jar hw2_dblp_mapred.jar 2 input
2021-05-07 18:22:02,295 INFO hw2.RunJobs$: Starting up MapReduce job..
2021-05-07 18:22:02,354 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at /0.0.0.0:8032
2021-05-07 18:22:02,583 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2021-05-07 18:22:02,669 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/hduser/.staging/job_1620420770486_0002
2021-05-07 18:22:05,246 INFO input.FileInputFormat: Total input files to process : 1
2021-05-07 18:22:05,810 INFO mapreduce.JobSubmitter: number of splits:24
2021-05-07 18:22:06,035 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1620420770486_0002
2021-05-07 18:22:06,035 INFO mapreduce.JobSubmitter: Executing with tokens: []
2021-05-07 18:22:06,158 INFO conf.Configuration: resource-types.xml not found
2021-05-07 18:22:06,158 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
2021-05-07 18:22:06,204 INFO impl.YarnClientImpl: Submitted application application_1620420770486_0002
2021-05-07 18:22:06,233 INFO mapreduce.Job: The url to track the job: http://[MYUSER].localdomain:8088/proxy/application_1620420770486_0002/
2021-05-07 18:22:06,234 INFO mapreduce.Job: Running job: job_1620420770486_0002
2021-05-07 18:22:12,284 INFO mapreduce.Job: Job job_1620420770486_0002 running in uber mode : false
2021-05-07 18:22:12,284 INFO mapreduce.Job: map 0% reduce 0%
2021-05-07 18:22:36,479 INFO mapreduce.Job: map 1% reduce 0%
2021-05-07 18:22:48,545 INFO mapreduce.Job: map 2% reduce 0%
2021-05-07 18:23:06,653 INFO mapreduce.Job: map 3% reduce 0%
2021-05-07 18:23:18,751 INFO mapreduce.Job: map 4% reduce 0%
2021-05-07 18:23:30,821 INFO mapreduce.Job: map 5% reduce 0%
2021-05-07 18:23:42,913 INFO mapreduce.Job: map 6% reduce 0%
2021-05-07 18:24:01,016 INFO mapreduce.Job: map 7% reduce 0%
2021-05-07 18:24:13,079 INFO mapreduce.Job: map 8% reduce 0%
2021-05-07 18:24:25,130 INFO mapreduce.Job: map 9% reduce 0%
2021-05-07 18:24:37,168 INFO mapreduce.Job: map 10% reduce 0%
2021-05-07 18:24:49,211 INFO mapreduce.Job: map 11% reduce 0%
2021-05-07 18:25:01,260 INFO mapreduce.Job: map 12% reduce 0%
2021-05-07 18:25:19,327 INFO mapreduce.Job: map 13% reduce 0%
2021-05-07 18:25:31,393 INFO mapreduce.Job: map 14% reduce 0%
2021-05-07 18:25:43,422 INFO mapreduce.Job: map 15% reduce 0%
2021-05-07 18:25:55,465 INFO mapreduce.Job: map 16% reduce 0%
2021-05-07 18:25:59,493 INFO mapreduce.Job: map 19% reduce 0%
2021-05-07 18:26:06,523 INFO mapreduce.Job: map 20% reduce 0%
2021-05-07 18:26:07,534 INFO mapreduce.Job: map 22% reduce 0%
2021-05-07 18:26:18,576 INFO mapreduce.Job: map 24% reduce 0%
2021-05-07 18:26:19,581 INFO mapreduce.Job: map 25% reduce 0%
2021-05-07 18:26:28,619 INFO mapreduce.Job: map 26% reduce 0%
2021-05-07 18:26:35,660 INFO mapreduce.Job: map 26% reduce 8%
2021-05-07 18:26:38,680 INFO mapreduce.Job: Task Id : attempt_1620420770486_0002_m_000010_0, Status : FAILED
Error: java.util.NoSuchElementException: head of empty list
at scala.collection.immutable.Nil$.head(List.scala:629)
at scala.collection.immutable.Nil$.head(List.scala:628)
at com.samujjwaal.hw2.mappers.VenueOneAuthorMapper.map(VenueOneAuthorMapper.scala:31)
at com.samujjwaal.hw2.mappers.VenueOneAuthorMapper.map(VenueOneAuthorMapper.scala:8)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:178)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172)
2021-05-07 18:26:47,724 INFO mapreduce.Job: map 27% reduce 8%
2021-05-07 18:26:58,778 INFO mapreduce.Job: Task Id : attempt_1620420770486_0002_m_000010_1, Status : FAILED
Error: java.util.NoSuchElementException: head of empty list
at scala.collection.immutable.Nil$.head(List.scala:629)
at scala.collection.immutable.Nil$.head(List.scala:628)
at com.samujjwaal.hw2.mappers.VenueOneAuthorMapper.map(VenueOneAuthorMapper.scala:31)
at com.samujjwaal.hw2.mappers.VenueOneAuthorMapper.map(VenueOneAuthorMapper.scala:8)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:178)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172)
2021-05-07 18:27:09,836 INFO mapreduce.Job: map 28% reduce 8%
2021-05-07 18:27:27,913 INFO mapreduce.Job: map 29% reduce 8%
2021-05-07 18:27:45,966 INFO mapreduce.Job: map 30% reduce 8%
2021-05-07 18:28:04,032 INFO mapreduce.Job: map 31% reduce 8%
2021-05-07 18:28:21,072 INFO mapreduce.Job: map 32% reduce 8%
2021-05-07 18:28:39,139 INFO mapreduce.Job: map 33% reduce 8%
2021-05-07 18:28:59,777 INFO mapreduce.Job: map 34% reduce 8%
2021-05-07 18:29:17,832 INFO mapreduce.Job: map 35% reduce 8%
2021-05-07 18:29:35,891 INFO mapreduce.Job: map 36% reduce 8%
2021-05-07 18:29:40,907 INFO mapreduce.Job: Task Id : attempt_1620420770486_0002_m_000009_0, Status : FAILED
Error: java.util.NoSuchElementException: head of empty list
at scala.collection.immutable.Nil$.head(List.scala:629)
at scala.collection.immutable.Nil$.head(List.scala:628)
at com.samujjwaal.hw2.mappers.VenueOneAuthorMapper.map(VenueOneAuthorMapper.scala:31)
at com.samujjwaal.hw2.mappers.VenueOneAuthorMapper.map(VenueOneAuthorMapper.scala:8)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:178)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172)
2021-05-07 18:29:41,921 INFO mapreduce.Job: map 33% reduce 8%
2021-05-07 18:29:47,940 INFO mapreduce.Job: map 34% reduce 8%
2021-05-07 18:29:53,283 INFO mapreduce.Job: map 37% reduce 8%
2021-05-07 18:29:54,292 INFO mapreduce.Job: map 37% reduce 10%
2021-05-07 18:30:00,313 INFO mapreduce.Job: map 37% reduce 11%
2021-05-07 18:30:03,340 INFO mapreduce.Job: Task Id : attempt_1620420770486_0002_m_000010_2, Status : FAILED
Error: java.util.NoSuchElementException: head of empty list
at scala.collection.immutable.Nil$.head(List.scala:629)
at scala.collection.immutable.Nil$.head(List.scala:628)
at com.samujjwaal.hw2.mappers.VenueOneAuthorMapper.map(VenueOneAuthorMapper.scala:31)
at com.samujjwaal.hw2.mappers.VenueOneAuthorMapper.map(VenueOneAuthorMapper.scala:8)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:178)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172)
2021-05-07 18:30:06,346 INFO mapreduce.Job: map 38% reduce 11%
2021-05-07 18:30:12,370 INFO mapreduce.Job: map 38% reduce 13%
2021-05-07 18:30:19,385 INFO mapreduce.Job: map 39% reduce 13%
2021-05-07 18:30:25,396 INFO mapreduce.Job: map 100% reduce 100%
2021-05-07 18:30:27,402 INFO mapreduce.Job: Job job_1620420770486_0002 failed with state FAILED due to: Task failed task_1620420770486_0002_m_000010
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0
2021-05-07 18:30:27,463 INFO mapreduce.Job: Counters: 43
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=38575671
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=1207962496
HDFS: Number of bytes written=0
HDFS: Number of read operations=27
HDFS: Number of large read operations=0
HDFS: Number of write operations=0
HDFS: Number of bytes read erasure-coded=0
Job Counters
Failed map tasks=5
Killed map tasks=14
Killed reduce tasks=1
Launched map tasks=18
Launched reduce tasks=1
Other local map tasks=4
Data-local map tasks=14
Total time spent by all maps in occupied slots (ms)=2679662
Total time spent by all reduces in occupied slots (ms)=245579
Total time spent by all map tasks (ms)=2679662
Total time spent by all reduce tasks (ms)=245579
Total vcore-milliseconds taken by all map tasks=2679662
Total vcore-milliseconds taken by all reduce tasks=245579
Total megabyte-milliseconds taken by all map tasks=2743973888
Total megabyte-milliseconds taken by all reduce tasks=251472896
Map-Reduce Framework
Map input records=2364905
Map output records=417710
Map output bytes=35339828
Map output materialized bytes=36191841
Input split bytes=1017
Combine input records=0
Spilled Records=417710
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=25146
CPU time spent (ms)=2043870
Physical memory (bytes) snapshot=5030256640
Virtual memory (bytes) snapshot=23105875968
Total committed heap usage (bytes)=4161798144
Peak Map Physical memory (bytes)=567414784
Peak Map Virtual memory (bytes)=2582102016
File Input Format Counters
Bytes Read=1207961479