Hi there, <div class="snippet-clipboard-content notranslate position-relative over

Here is some high-level deion: I have a 5-node vm-based cluster, including one m

Comments (15)

adrian-wang commented on July 19, 2024

HiBench 3.0 should support hadoop 1.2.1, thought we may not tested that.

from hibench.

echozyw commented on July 19, 2024

Thank you for the quick response. I appreciate it!
Here are some warnings which occur very often when I ran ./run.sh for the Bayes workload. Could you shed light on what might go wrong here:

15/04/03 02:58:41 INFO mapred.JobClient: map 0% reduce 0%
15/04/03 03:12:09 INFO mapred.JobClient: Task Id : attempt_201504022127_0012_m_000003_0, Status : FAILED
Error launching task
15/04/03 03:12:09 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0012_m_000003_0&filter=stdout
15/04/03 03:12:09 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0012_m_000003_0&filter=stderr
15/04/03 03:12:09 INFO mapred.JobClient: Task Id : attempt_201504022127_0012_m_000004_0, Status : FAILED
Error launching task
15/04/03 03:12:09 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0012_m_000004_0&filter=stdout
15/04/03 03:12:09 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0012_m_000004_0&filter=stderr
15/04/03 03:12:57 INFO mapred.JobClient: map 6% reduce 0%
....

from hibench.

adrian-wang commented on July 19, 2024

Can you share your hibench-config?

from hibench.

echozyw commented on July 19, 2024

No problem.

this="${BASH_SOURCE-$0}"
bin=$(cd -P -- "$(dirname -- "$this")" && pwd -P)
script="$(basename -- "$this")"
this="$bin/$script"

export HIBENCH_VERSION="3.0"

###################### Global Paths ##################

#export JAVA_HOME=
#export HADOOP_HOME=
#export HADOOP_EXECUTABLE=
#export HADOOP_CONF_DIR=
#export HADOOP_EXAMPLES_JAR=
#export MAPRED_EXECUTABLE=
##Set the varaible below only in YARN mode
#export HADOOP_JOBCLIENT_TESTS_JAR=

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk.x86_64
export HADOOP_HOME=/home/fedora/hadoop-1.2.1
export HADOOP_EXECUTABLE=/home/fedora/hadoop-1.2.1/bin/hadoop
export HADOOP_CONF_DIR=/home/fedora/hadoop-1.2.1/conf
export HADOOP_EXAMPLES_JAR=/home/fedora/hadoop-1.2.1/hadoop-examples-1.2.1.jar
export MAPRED_EXECUTABLE=
#Set the varaible below only in YARN mode
export HADOOP_JOBCLIENT_TESTS_JAR=

export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_VERSION=hadoop2 # set it to hadoop1 to enable MR1, hadoop2 to enable MR2

if $HADOOP_EXECUTABLE version|grep -i -q cdh4; then
HADOOP_RELEASE=cdh4
HADOOP_VERSION=cdh4
elif $HADOOP_EXECUTABLE version|grep -i -q cdh5; then
HADOOP_RELEASE=cdh5
HADOOP_VERSION=cdh5
elif $HADOOP_EXECUTABLE version|grep -i -q "hadoop 2"; then
HADOOP_RELEASE=hadoop2
HADOOP_VERSION=hadoop2
else
HADOOP_RELEASE=hadoop1
HADOOP_VERSION=hadoop1
fi

if [ "x"$HADOOP_VERSION == "xhadoop1" ]; then

CONFIG_REDUCER_NUMBER=mapred.reduce.tasks
CONFIG_MAP_NUMBER=mapred.map.tasks
else

CONFIG_REDUCER_NUMBER=mapreduce.job.reduces
CONFIG_MAP_NUMBER=mapreduce.job.maps
fi

echo JAVA_HOME=${JAVA_HOME:? "ERROR: Please set paths in $this before using HiBench."}
echo HADOOP_HOME=${HADOOP_HOME:? "ERROR: Please set paths in $this before using HiBench."}
echo HADOOP_EXECUTABLE=${HADOOP_EXECUTABLE:? "ERROR: Please set paths in $this before using HiBench."}
echo HADOOP_CONF_DIR=${HADOOP_CONF_DIR:? "ERROR: Please set paths in $this before using HiBench."}
echo HADOOP_EXAMPLES_JAR=${HADOOP_EXAMPLES_JAR:? "ERROR: Please set paths in $this before using HiBench."}
#echo MAPRED_EXECUTABLE=${MAPRED_EXECUTABLE:? "ERROR: Please set paths in $this before using HiBench."}

if [ -z "$HIBENCH_HOME" ]; then
export HIBENCH_HOME=dirname "$this"/..
fi

if [ -z "$HIBENCH_CONF" ]; then
export HIBENCH_CONF=${HIBENCH_HOME}/conf
fi

if [ -f "${HIBENCH_CONF}/funcs.sh" ]; then
. "${HIBENCH_CONF}/funcs.sh"
fi

if [ -z "$DEPENDENCY_DIR" ]; then
export DEPENDENCY_DIR=${HIBENCH_HOME}/common/hibench
fi

if [ -z "$HIVE_HOME" ]; then
export HIVE_RELEASE=hive-0.12.0-bin
export HIVE_HOME=${DEPENDENCY_DIR}/hivebench/target/${HIVE_RELEASE}
fi

if [ -z "$MAHOUT_HOME" ]; then
export MAHOUT_RELEASE=mahout-distribution-0.7
export MAHOUT_EXAMPLE_JOB="mahout-examples-0.7-job.jar"
export MAHOUT_HOME=${DEPENDENCY_DIR}/mahout/target/${MAHOUT_RELEASE}
fi

if [ -z "$NUTCH_HOME" ]; then
export NUTCH_RELEASE=nutch-1.2
export NUTCH_HOME=${DEPENDENCY_DIR}/nutchindexing/target/${NUTCH_RELEASE}
fi

if [ -z "$DATATOOLS" ]; then
export DATATOOLS=${HIBENCH_HOME}/common/autogen/dist/datatools.jar
fi

if [ $# -gt 1 ]
then
if [ "--hadoop_config" = "$1" ]
then
shift
confdir=$1
shift
HADOOP_CONF_DIR=$confdir
fi
fi

base dir HDFS

#export DATA_HDFS=/HiBench
export DATA_HDFS=/home/fedora

local report

export HIBENCH_REPORT=${HIBENCH_HOME}/hibench.report

################# Compress Options #################

swith on/off compression: 0-off, 1-on.

Switch it off (COMPRESS_GLOBAL=0) for better performance

export COMPRESS_GLOBAL=1
export COMPRESS_CODEC_GLOBAL=org.apache.hadoop.io.compress.DefaultCodec
export COMPRESS_CODEC_MAP=org.apache.hadoop.io.compress.DefaultCodec

Set COMPRESS_CODEC_MAP to SnappyCodec (as shown below) for better performance

#export COMPRESS_CODEC_MAP=org.apache.hadoop.io.compress.SnappyCodec

#export COMPRESS_CODEC_GLOBAL=com.hadoop.compression.lzo.LzoCodec
#export COMPRESS_CODEC_GLOBAL=org.apache.hadoop.io.compress.SnappyCodec

from hibench.

echozyw commented on July 19, 2024

Also, here is how it crashed when I ./run.sh for the Bayes workload. Thanks a lot!

[fedora@newjobtestmaster bin]$ ./run.sh
========== running bayes bench ==========
Warning: $HADOOP_HOME is deprecated.

Warning: $HADOOP_HOME is deprecated.

JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk.x86_64
HADOOP_HOME=/home/fedora/hadoop-1.2.1
HADOOP_EXECUTABLE=/home/fedora/hadoop-1.2.1/bin/hadoop
HADOOP_CONF_DIR=/home/fedora/hadoop-1.2.1/conf
HADOOP_EXAMPLES_JAR=/home/fedora/hadoop-1.2.1/hadoop-examples-1.2.1.jar
###done from hibench-config.sh###

Warning: $HADOOP_HOME is deprecated.

Deleted hdfs://192.168.111.240:9000/home/fedora/Bayes/Output-comp
Warning: $HADOOP_HOME is deprecated.

MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /home/fedora/hadoop-1.2.1/bin/hadoop and HADOOP_CONF_DIR=/home/fedora/hadoop-1.2.1/conf
MAHOUT-JOB: /home/fedora/HiBench/common/hibench/mahout/target/mahout-distribution-0.7/mahout-examples-0.7-job.jar
Warning: $HADOOP_HOME is deprecated.

15/04/03 03:17:57 INFO vectorizer.SparseVectorsFromSequenceFiles: Maximum n-gram size is: 3
15/04/03 03:17:57 INFO vectorizer.SparseVectorsFromSequenceFiles: Minimum LLR value: 1.0
15/04/03 03:17:57 INFO vectorizer.SparseVectorsFromSequenceFiles: Number of reduce tasks: 24
15/04/03 03:22:02 INFO input.FileInputFormat: Total input paths to process : 48
15/04/03 03:22:03 INFO mapred.JobClient: Running job: job_201504022127_0013
15/04/03 03:22:04 INFO mapred.JobClient: map 0% reduce 0%
15/04/03 03:22:07 INFO mapred.JobClient: Task Id : attempt_201504022127_0013_m_000049_0, Status : FAILED
Error launching task
15/04/03 03:22:07 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave3:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000049_0&filter=stdout
15/04/03 03:22:07 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave3:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000049_0&filter=stderr
15/04/03 03:22:18 INFO mapred.JobClient: Task Id : attempt_201504022127_0013_m_000001_0, Status : FAILED
Error launching task
15/04/03 03:22:18 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000001_0&filter=stdout
15/04/03 03:22:18 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000001_0&filter=stderr
15/04/03 03:22:18 INFO mapred.JobClient: Task Id : attempt_201504022127_0013_m_000003_0, Status : FAILED
Error launching task
15/04/03 03:22:18 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000003_0&filter=stdout
15/04/03 03:22:18 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000003_0&filter=stderr
15/04/03 03:22:18 INFO mapred.JobClient: Task Id : attempt_201504022127_0013_m_000004_0, Status : FAILED
Error launching task
15/04/03 03:22:18 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave0:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000004_0&filter=stdout
15/04/03 03:22:18 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave0:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000004_0&filter=stderr
15/04/03 03:22:18 INFO mapred.JobClient: Task Id : attempt_201504022127_0013_m_000005_0, Status : FAILED
Error launching task
15/04/03 03:22:18 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave0:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000005_0&filter=stdout
15/04/03 03:22:18 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave0:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000005_0&filter=stderr
15/04/03 03:22:18 INFO mapred.JobClient: Task Id : attempt_201504022127_0013_m_000006_0, Status : FAILED
Error launching task
15/04/03 03:22:18 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000006_0&filter=stdout
15/04/03 03:22:18 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000006_0&filter=stderr
15/04/03 03:22:18 INFO mapred.JobClient: Task Id : attempt_201504022127_0013_m_000007_0, Status : FAILED
Error launching task
15/04/03 03:22:18 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000007_0&filter=stdout
15/04/03 03:22:18 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000007_0&filter=stderr
15/04/03 03:22:30 INFO mapred.JobClient: map 2% reduce 0%
15/04/03 03:22:39 INFO mapred.JobClient: Task Id : attempt_201504022127_0013_m_000001_1, Status : FAILED
Error launching task
15/04/03 03:22:39 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000001_1&filter=stdout
15/04/03 03:22:39 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000001_1&filter=stderr
15/04/03 03:22:42 INFO mapred.JobClient: map 3% reduce 0%
15/04/03 03:22:45 INFO mapred.JobClient: map 4% reduce 0%
15/04/03 03:22:50 INFO mapred.JobClient: Task Id : attempt_201504022127_0013_m_000004_1, Status : FAILED
Error launching task
15/04/03 03:22:50 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000004_1&filter=stdout
15/04/03 03:22:50 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000004_1&filter=stderr
15/04/03 03:23:03 INFO mapred.JobClient: map 5% reduce 0%
15/04/03 03:23:03 INFO mapred.JobClient: Task Id : attempt_201504022127_0013_m_000004_2, Status : FAILED
Error launching task
15/04/03 03:23:03 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000004_2&filter=stdout
15/04/03 03:23:03 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000004_2&filter=stderr
15/04/03 03:23:04 INFO mapred.JobClient: Task Id : attempt_201504022127_0013_m_000003_1, Status : FAILED
Error launching task
15/04/03 03:23:04 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave0:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000003_1&filter=stdout
15/04/03 03:23:04 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave0:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000003_1&filter=stderr
15/04/03 03:23:07 INFO mapred.JobClient: Task Id : attempt_201504022127_0013_m_000005_1, Status : FAILED
Error launching task
15/04/03 03:23:07 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000005_1&filter=stdout
15/04/03 03:23:07 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000005_1&filter=stderr
15/04/03 03:23:14 INFO mapred.JobClient: map 6% reduce 0%
15/04/03 03:23:17 INFO mapred.JobClient: Task Id : attempt_201504022127_0013_m_000006_1, Status : FAILED
Error launching task
15/04/03 03:23:17 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave0:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000006_1&filter=stdout
15/04/03 03:23:17 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave0:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0013_m_000006_1&filter=stderr
15/04/03 03:23:26 INFO mapred.JobClient: map 7% reduce 0%
15/04/03 03:23:30 INFO mapred.JobClient: map 8% reduce 0%
15/04/03 03:23:44 INFO mapred.JobClient: map 9% reduce 0%
15/04/03 03:23:50 INFO mapred.JobClient: map 10% reduce 0%
15/04/03 03:23:56 INFO mapred.JobClient: map 11% reduce 0%
15/04/03 03:24:05 INFO mapred.JobClient: map 12% reduce 0%
15/04/03 03:24:16 INFO mapred.JobClient: map 13% reduce 0%
15/04/03 03:24:28 INFO mapred.JobClient: map 14% reduce 0%
15/04/03 03:24:52 INFO mapred.JobClient: map 15% reduce 0%
15/04/03 03:25:01 INFO mapred.JobClient: map 16% reduce 0%
15/04/03 03:25:04 INFO mapred.JobClient: map 17% reduce 0%
15/04/03 03:25:13 INFO mapred.JobClient: map 18% reduce 0%
15/04/03 03:25:27 INFO mapred.JobClient: map 19% reduce 0%
15/04/03 03:25:34 INFO mapred.JobClient: map 20% reduce 0%
15/04/03 03:25:39 INFO mapred.JobClient: map 21% reduce 0%
15/04/03 03:25:46 INFO mapred.JobClient: map 22% reduce 0%
15/04/03 03:26:07 INFO mapred.JobClient: map 24% reduce 0%
15/04/03 03:26:10 INFO mapred.JobClient: map 25% reduce 0%
15/04/03 03:26:22 INFO mapred.JobClient: map 26% reduce 0%
15/04/03 03:26:25 INFO mapred.JobClient: map 27% reduce 0%

15/04/03 03:26:41 INFO mapred.JobClient: map 28% reduce 0%

15/04/03 03:26:53 INFO mapred.JobClient: map 30% reduce 0%
15/04/03 03:27:08 INFO mapred.JobClient: map 31% reduce 0%
15/04/03 03:27:11 INFO mapred.JobClient: map 32% reduce 0%
15/04/03 03:27:20 INFO mapred.JobClient: map 33% reduce 0%
15/04/03 03:27:31 INFO mapred.JobClient: map 34% reduce 0%
15/04/03 03:27:36 INFO mapred.JobClient: map 35% reduce 0%
15/04/03 03:27:43 INFO mapred.JobClient: map 36% reduce 0%
15/04/03 03:27:48 INFO mapred.JobClient: map 37% reduce 0%
15/04/03 03:28:06 INFO mapred.JobClient: map 38% reduce 0%
15/04/03 03:28:13 INFO mapred.JobClient: map 39% reduce 0%
15/04/03 03:28:18 INFO mapred.JobClient: map 40% reduce 0%
15/04/03 03:28:25 INFO mapred.JobClient: map 41% reduce 0%
15/04/03 03:28:39 INFO mapred.JobClient: map 42% reduce 0%
15/04/03 03:28:45 INFO mapred.JobClient: map 43% reduce 0%
15/04/03 03:28:51 INFO mapred.JobClient: map 44% reduce 0%
15/04/03 03:28:57 INFO mapred.JobClient: map 45% reduce 0%
15/04/03 03:29:11 INFO mapred.JobClient: map 46% reduce 0%
15/04/03 03:29:23 INFO mapred.JobClient: map 47% reduce 0%
15/04/03 03:29:30 INFO mapred.JobClient: map 49% reduce 0%
15/04/03 03:30:00 INFO mapred.JobClient: map 51% reduce 0%
15/04/03 03:30:15 INFO mapred.JobClient: map 53% reduce 0%
15/04/03 03:30:30 INFO mapred.JobClient: map 54% reduce 0%
15/04/03 03:30:33 INFO mapred.JobClient: map 55% reduce 0%
15/04/03 03:30:45 INFO mapred.JobClient: map 56% reduce 0%
15/04/03 03:31:04 INFO mapred.JobClient: map 57% reduce 0%
15/04/03 03:31:06 INFO mapred.JobClient: map 58% reduce 0%
15/04/03 03:31:16 INFO mapred.JobClient: map 59% reduce 0%
15/04/03 03:31:19 INFO mapred.JobClient: map 60% reduce 0%
15/04/03 03:31:33 INFO mapred.JobClient: map 61% reduce 0%
15/04/03 03:31:42 INFO mapred.JobClient: map 62% reduce 0%
15/04/03 03:31:45 INFO mapred.JobClient: map 63% reduce 0%
15/04/03 03:31:54 INFO mapred.JobClient: map 64% reduce 0%
15/04/03 03:32:12 INFO mapred.JobClient: map 65% reduce 0%
15/04/03 03:32:19 INFO mapred.JobClient: map 66% reduce 0%
15/04/03 03:32:24 INFO mapred.JobClient: map 67% reduce 0%
15/04/03 03:32:31 INFO mapred.JobClient: map 68% reduce 0%
15/04/03 03:32:43 INFO mapred.JobClient: map 69% reduce 0%
15/04/03 03:32:49 INFO mapred.JobClient: map 70% reduce 0%
15/04/03 03:32:55 INFO mapred.JobClient: map 71% reduce 0%
15/04/03 03:33:00 INFO mapred.JobClient: map 72% reduce 0%
15/04/03 03:33:17 INFO mapred.JobClient: map 74% reduce 0%
15/04/03 03:33:28 INFO mapred.JobClient: map 75% reduce 0%
15/04/03 03:33:32 INFO mapred.JobClient: map 76% reduce 0%
15/04/03 03:33:49 INFO mapred.JobClient: map 77% reduce 0%
15/04/03 03:34:04 INFO mapred.JobClient: map 78% reduce 0%
15/04/03 03:34:09 INFO mapred.JobClient: map 79% reduce 0%
15/04/03 03:34:16 INFO mapred.JobClient: map 80% reduce 0%
15/04/03 03:34:28 INFO mapred.JobClient: map 81% reduce 0%
15/04/03 03:34:34 INFO mapred.JobClient: map 82% reduce 0%
15/04/03 03:34:43 INFO mapred.JobClient: map 83% reduce 0%
15/04/03 03:34:53 INFO mapred.JobClient: map 84% reduce 0%
15/04/03 03:34:59 INFO mapred.JobClient: map 85% reduce 0%
15/04/03 03:35:08 INFO mapred.JobClient: map 86% reduce 0%
15/04/03 03:35:17 INFO mapred.JobClient: map 87% reduce 0%
15/04/03 03:35:23 INFO mapred.JobClient: map 88% reduce 0%
15/04/03 03:35:32 INFO mapred.JobClient: map 89% reduce 0%
15/04/03 03:35:41 INFO mapred.JobClient: map 90% reduce 0%
15/04/03 03:35:50 INFO mapred.JobClient: map 91% reduce 0%
15/04/03 03:35:53 INFO mapred.JobClient: map 92% reduce 0%
15/04/03 03:36:03 INFO mapred.JobClient: map 93% reduce 0%
15/04/03 03:36:13 INFO mapred.JobClient: map 94% reduce 0%
15/04/03 03:36:25 INFO mapred.JobClient: map 95% reduce 0%
15/04/03 03:36:50 INFO mapred.JobClient: map 96% reduce 0%
15/04/03 03:37:01 INFO mapred.JobClient: map 97% reduce 0%
15/04/03 03:37:46 INFO mapred.JobClient: map 99% reduce 0%
15/04/03 03:38:54 INFO mapred.JobClient: map 100% reduce 0%
15/04/03 03:38:56 INFO mapred.JobClient: Job complete: job_201504022127_0013
15/04/03 03:38:56 INFO mapred.JobClient: Counters: 20
15/04/03 03:38:56 INFO mapred.JobClient: Job Counters
15/04/03 03:38:56 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=1839963
15/04/03 03:38:56 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
15/04/03 03:38:56 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
15/04/03 03:38:56 INFO mapred.JobClient: Rack-local map tasks=15
15/04/03 03:38:56 INFO mapred.JobClient: Launched map tasks=60
15/04/03 03:38:56 INFO mapred.JobClient: Data-local map tasks=45
15/04/03 03:38:56 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=6580
15/04/03 03:38:56 INFO mapred.JobClient: File Output Format Counters
15/04/03 03:38:56 INFO mapred.JobClient: Bytes Written=87165324
15/04/03 03:38:56 INFO mapred.JobClient: FileSystemCounters
15/04/03 03:38:56 INFO mapred.JobClient: HDFS_BYTES_READ=181494754
15/04/03 03:38:56 INFO mapred.JobClient: FILE_BYTES_WRITTEN=2710790
15/04/03 03:38:56 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=87165324
15/04/03 03:38:56 INFO mapred.JobClient: File Input Format Counters
15/04/03 03:38:56 INFO mapred.JobClient: Bytes Read=181488418
15/04/03 03:38:56 INFO mapred.JobClient: Map-Reduce Framework
15/04/03 03:38:56 INFO mapred.JobClient: Map input records=40000
15/04/03 03:38:56 INFO mapred.JobClient: Physical memory (bytes) snapshot=3853488128
15/04/03 03:38:56 INFO mapred.JobClient: Spilled Records=0
15/04/03 03:38:56 INFO mapred.JobClient: CPU time spent (ms)=134550
15/04/03 03:38:56 INFO mapred.JobClient: Total committed heap usage (bytes)=1525678080
15/04/03 03:38:56 INFO mapred.JobClient: Virtual memory (bytes) snapshot=40598532096
15/04/03 03:38:56 INFO mapred.JobClient: Map output records=40000
15/04/03 03:38:56 INFO mapred.JobClient: SPLIT_RAW_BYTES=6336
15/04/03 03:42:26 INFO input.FileInputFormat: Total input paths to process : 48
15/04/03 03:42:27 INFO mapred.JobClient: Running job: job_201504022127_0014
15/04/03 03:42:28 INFO mapred.JobClient: map 0% reduce 0%
15/04/03 03:42:32 INFO mapred.JobClient: Task Id : attempt_201504022127_0014_m_000049_0, Status : FAILED
Error launching task
15/04/03 03:42:34 INFO mapred.JobClient: Task Id : attempt_201504022127_0014_m_000049_1, Status : FAILED
Error launching task
15/04/03 03:42:34 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave0:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_m_000049_1&filter=stdout
15/04/03 03:42:34 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave0:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_m_000049_1&filter=stderr
15/04/03 03:42:43 INFO mapred.JobClient: Task Id : attempt_201504022127_0014_m_000004_0, Status : FAILED
Error launching task
15/04/03 03:42:43 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_m_000004_0&filter=stdout
15/04/03 03:42:43 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_m_000004_0&filter=stderr
15/04/03 03:42:43 INFO mapred.JobClient: Task Id : attempt_201504022127_0014_m_000005_0, Status : FAILED
Error launching task
15/04/03 03:42:43 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_m_000005_0&filter=stdout
15/04/03 03:42:43 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_m_000005_0&filter=stderr
15/04/03 03:42:43 INFO mapred.JobClient: Task Id : attempt_201504022127_0014_m_000007_0, Status : FAILED
Error launching task
15/04/03 03:42:43 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_m_000007_0&filter=stdout
15/04/03 03:42:43 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_m_000007_0&filter=stderr
15/04/03 03:42:43 INFO mapred.JobClient: Task Id : attempt_201504022127_0014_m_000008_0, Status : FAILED
Error launching task
15/04/03 03:42:43 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_m_000008_0&filter=stdout
15/04/03 03:42:43 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_m_000008_0&filter=stderr
15/04/03 03:42:53 INFO mapred.JobClient: map 1% reduce 0%
15/04/03 03:42:55 INFO mapred.JobClient: map 2% reduce 0%
15/04/03 03:43:02 INFO mapred.JobClient: map 3% reduce 0%
15/04/03 03:43:07 INFO mapred.JobClient: map 4% reduce 0%
15/04/03 03:43:14 INFO mapred.JobClient: map 5% reduce 0%
15/04/03 03:43:16 INFO mapred.JobClient: map 6% reduce 0%
15/04/03 03:43:23 INFO mapred.JobClient: map 7% reduce 0%
15/04/03 03:43:30 INFO mapred.JobClient: map 8% reduce 0%
15/04/03 03:43:54 INFO mapred.JobClient: Task Id : attempt_201504022127_0014_r_000002_0, Status : FAILED
Error launching task
15/04/03 03:43:54 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_r_000002_0&filter=stdout
15/04/03 03:43:54 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_r_000002_0&filter=stderr
15/04/03 03:43:54 INFO mapred.JobClient: Task Id : attempt_201504022127_0014_r_000006_0, Status : FAILED
Error launching task
15/04/03 03:43:54 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_r_000006_0&filter=stdout
15/04/03 03:43:54 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_r_000006_0&filter=stderr
15/04/03 03:44:03 INFO mapred.JobClient: map 9% reduce 0%
15/04/03 03:44:12 INFO mapred.JobClient: map 10% reduce 0%
15/04/03 03:44:16 INFO mapred.JobClient: map 11% reduce 0%
15/04/03 03:44:17 INFO mapred.JobClient: map 12% reduce 0%
15/04/03 03:44:25 INFO mapred.JobClient: map 13% reduce 0%
15/04/03 03:44:29 INFO mapred.JobClient: map 15% reduce 0%
15/04/03 03:44:34 INFO mapred.JobClient: map 16% reduce 0%
15/04/03 03:44:37 INFO mapred.JobClient: map 17% reduce 0%
15/04/03 03:44:41 INFO mapred.JobClient: map 18% reduce 0%
15/04/03 03:44:43 INFO mapred.JobClient: map 19% reduce 0%
15/04/03 03:44:52 INFO mapred.JobClient: map 20% reduce 0%
15/04/03 03:45:11 INFO mapred.JobClient: map 21% reduce 0%
15/04/03 03:45:24 INFO mapred.JobClient: map 22% reduce 0%
15/04/03 03:45:25 INFO mapred.JobClient: map 22% reduce 1%
15/04/03 03:45:26 INFO mapred.JobClient: map 23% reduce 1%
15/04/03 03:45:29 INFO mapred.JobClient: map 24% reduce 1%
15/04/03 03:45:33 INFO mapred.JobClient: map 25% reduce 1%
15/04/03 03:45:38 INFO mapred.JobClient: map 26% reduce 1%
15/04/03 03:45:43 INFO mapred.JobClient: map 27% reduce 1%
15/04/03 03:45:44 INFO mapred.JobClient: map 28% reduce 1%
15/04/03 03:45:50 INFO mapred.JobClient: map 29% reduce 1%
15/04/03 03:45:52 INFO mapred.JobClient: map 30% reduce 1%
15/04/03 03:45:57 INFO mapred.JobClient: map 31% reduce 1%
15/04/03 03:46:04 INFO mapred.JobClient: map 32% reduce 1%
15/04/03 03:46:16 INFO mapred.JobClient: map 33% reduce 1%
15/04/03 03:46:30 INFO mapred.JobClient: map 34% reduce 1%
15/04/03 03:46:36 INFO mapred.JobClient: map 35% reduce 1%
15/04/03 03:46:42 INFO mapred.JobClient: map 35% reduce 2%
15/04/03 03:46:43 INFO mapred.JobClient: map 36% reduce 2%
15/04/03 03:46:48 INFO mapred.JobClient: map 37% reduce 2%
15/04/03 03:46:52 INFO mapred.JobClient: map 38% reduce 2%
15/04/03 03:46:54 INFO mapred.JobClient: map 39% reduce 2%
15/04/03 03:46:55 INFO mapred.JobClient: map 40% reduce 2%
15/04/03 03:46:57 INFO mapred.JobClient: map 41% reduce 2%
15/04/03 03:47:01 INFO mapred.JobClient: map 42% reduce 2%
15/04/03 03:47:04 INFO mapred.JobClient: map 44% reduce 2%
15/04/03 03:47:10 INFO mapred.JobClient: map 45% reduce 2%
15/04/03 03:47:16 INFO mapred.JobClient: map 46% reduce 2%
15/04/03 03:47:17 INFO mapred.JobClient: map 47% reduce 2%
15/04/03 03:47:22 INFO mapred.JobClient: map 48% reduce 2%
15/04/03 03:47:29 INFO mapred.JobClient: map 49% reduce 2%
15/04/03 03:47:36 INFO mapred.JobClient: map 50% reduce 2%
15/04/03 03:47:40 INFO mapred.JobClient: map 51% reduce 2%
15/04/03 03:47:47 INFO mapred.JobClient: map 51% reduce 3%
15/04/03 03:47:52 INFO mapred.JobClient: map 52% reduce 3%
15/04/03 03:47:59 INFO mapred.JobClient: map 53% reduce 3%
15/04/03 03:48:02 INFO mapred.JobClient: map 54% reduce 3%
15/04/03 03:48:08 INFO mapred.JobClient: map 55% reduce 3%
15/04/03 03:48:09 INFO mapred.JobClient: map 56% reduce 3%
15/04/03 03:48:11 INFO mapred.JobClient: map 57% reduce 3%
15/04/03 03:48:13 INFO mapred.JobClient: map 58% reduce 3%
15/04/03 03:48:20 INFO mapred.JobClient: map 59% reduce 3%
15/04/03 03:48:24 INFO mapred.JobClient: map 60% reduce 3%
15/04/03 03:48:33 INFO mapred.JobClient: map 61% reduce 3%
15/04/03 03:48:35 INFO mapred.JobClient: map 62% reduce 3%
15/04/03 03:48:43 INFO mapred.JobClient: map 63% reduce 3%
15/04/03 03:48:44 INFO mapred.JobClient: map 63% reduce 4%
15/04/03 03:48:51 INFO mapred.JobClient: map 65% reduce 4%
15/04/03 03:49:03 INFO mapred.JobClient: map 66% reduce 4%
15/04/03 03:49:04 INFO mapred.JobClient: map 67% reduce 4%
15/04/03 03:49:12 INFO mapred.JobClient: map 68% reduce 4%
15/04/03 03:49:15 INFO mapred.JobClient: map 70% reduce 4%
15/04/03 03:49:24 INFO mapred.JobClient: map 71% reduce 4%
15/04/03 03:49:26 INFO mapred.JobClient: map 72% reduce 4%
15/04/03 03:49:34 INFO mapred.JobClient: map 73% reduce 4%
15/04/03 03:49:38 INFO mapred.JobClient: map 74% reduce 5%
15/04/03 03:49:40 INFO mapred.JobClient: map 75% reduce 5%
15/04/03 03:49:48 INFO mapred.JobClient: map 76% reduce 5%
15/04/03 03:49:49 INFO mapred.JobClient: map 77% reduce 5%
15/04/03 03:49:51 INFO mapred.JobClient: map 78% reduce 5%
15/04/03 03:49:59 INFO mapred.JobClient: map 79% reduce 5%
15/04/03 03:50:00 INFO mapred.JobClient: map 80% reduce 5%
15/04/03 03:50:03 INFO mapred.JobClient: map 81% reduce 5%
15/04/03 03:50:15 INFO mapred.JobClient: map 83% reduce 5%
15/04/03 03:50:25 INFO mapred.JobClient: map 84% reduce 5%
15/04/03 03:50:27 INFO mapred.JobClient: map 85% reduce 5%
15/04/03 03:50:28 INFO mapred.JobClient: map 86% reduce 5%
15/04/03 03:50:36 INFO mapred.JobClient: map 87% reduce 5%
15/04/03 03:50:38 INFO mapred.JobClient: map 88% reduce 5%
15/04/03 03:50:46 INFO mapred.JobClient: map 89% reduce 5%
15/04/03 03:50:47 INFO mapred.JobClient: map 90% reduce 5%
15/04/03 03:50:50 INFO mapred.JobClient: map 91% reduce 5%
15/04/03 03:50:53 INFO mapred.JobClient: map 92% reduce 5%
15/04/03 03:50:54 INFO mapred.JobClient: map 92% reduce 6%
15/04/03 03:51:00 INFO mapred.JobClient: map 93% reduce 6%
15/04/03 03:51:06 INFO mapred.JobClient: map 94% reduce 6%
15/04/03 03:51:09 INFO mapred.JobClient: map 95% reduce 6%
15/04/03 03:51:14 INFO mapred.JobClient: map 96% reduce 6%
15/04/03 03:51:28 INFO mapred.JobClient: map 97% reduce 6%
15/04/03 03:51:29 INFO mapred.JobClient: map 98% reduce 6%
15/04/03 03:51:31 INFO mapred.JobClient: map 98% reduce 7%
15/04/03 03:51:47 INFO mapred.JobClient: map 99% reduce 7%
15/04/03 03:51:59 INFO mapred.JobClient: map 100% reduce 7%
15/04/03 03:52:59 INFO mapred.JobClient: map 100% reduce 8%
15/04/03 03:53:02 INFO mapred.JobClient: map 100% reduce 9%
15/04/03 03:53:05 INFO mapred.JobClient: map 100% reduce 10%
15/04/03 03:53:08 INFO mapred.JobClient: map 100% reduce 12%
15/04/03 03:53:11 INFO mapred.JobClient: map 100% reduce 13%
15/04/03 03:53:20 INFO mapred.JobClient: map 100% reduce 15%
15/04/03 03:53:23 INFO mapred.JobClient: map 100% reduce 16%
15/04/03 03:53:46 INFO mapred.JobClient: map 100% reduce 17%
15/04/03 03:53:50 INFO mapred.JobClient: map 100% reduce 20%
15/04/03 03:53:53 INFO mapred.JobClient: map 100% reduce 22%
15/04/03 03:53:56 INFO mapred.JobClient: map 100% reduce 24%
15/04/03 03:53:59 INFO mapred.JobClient: map 100% reduce 25%
15/04/03 03:54:01 INFO mapred.JobClient: map 100% reduce 26%
15/04/03 03:54:13 INFO mapred.JobClient: Task Id : attempt_201504022127_0014_r_000011_0, Status : FAILED
Error launching task
15/04/03 03:54:13 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_r_000011_0&filter=stdout
15/04/03 03:54:13 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_r_000011_0&filter=stderr
15/04/03 03:54:15 INFO mapred.JobClient: Task Id : attempt_201504022127_0014_r_000012_0, Status : FAILED
Error launching task
15/04/03 03:54:15 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_r_000012_0&filter=stdout
15/04/03 03:54:15 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0014_r_000012_0&filter=stderr
15/04/03 03:54:28 INFO mapred.JobClient: map 100% reduce 27%
15/04/03 03:54:41 INFO mapred.JobClient: map 100% reduce 28%
15/04/03 03:55:10 INFO mapred.JobClient: map 100% reduce 29%
15/04/03 03:55:44 INFO mapred.JobClient: map 100% reduce 30%
15/04/03 03:56:20 INFO mapred.JobClient: map 100% reduce 31%
15/04/03 03:56:47 INFO mapred.JobClient: map 100% reduce 32%
15/04/03 03:56:50 INFO mapred.JobClient: map 100% reduce 33%
15/04/03 03:56:53 INFO mapred.JobClient: map 100% reduce 34%
15/04/03 03:57:17 INFO mapred.JobClient: map 100% reduce 35%
15/04/03 03:58:08 INFO mapred.JobClient: map 100% reduce 36%
15/04/03 03:58:35 INFO mapred.JobClient: map 100% reduce 38%
15/04/03 03:58:41 INFO mapred.JobClient: map 100% reduce 39%
15/04/03 03:58:51 INFO mapred.JobClient: map 100% reduce 40%
15/04/03 03:58:54 INFO mapred.JobClient: map 100% reduce 41%
15/04/03 03:58:56 INFO mapred.JobClient: map 100% reduce 42%
15/04/03 03:59:02 INFO mapred.JobClient: map 100% reduce 43%
15/04/03 03:59:05 INFO mapred.JobClient: map 100% reduce 44%
15/04/03 03:59:08 INFO mapred.JobClient: map 100% reduce 45%
15/04/03 03:59:40 INFO mapred.JobClient: map 100% reduce 46%
15/04/03 04:00:07 INFO mapred.JobClient: map 100% reduce 47%
15/04/03 04:00:25 INFO mapred.JobClient: map 100% reduce 49%
15/04/03 04:00:28 INFO mapred.JobClient: map 100% reduce 50%
15/04/03 04:01:07 INFO mapred.JobClient: map 100% reduce 51%
15/04/03 04:01:20 INFO mapred.JobClient: map 100% reduce 52%
15/04/03 04:01:21 INFO mapred.JobClient: map 100% reduce 53%
15/04/03 04:01:26 INFO mapred.JobClient: map 100% reduce 54%
15/04/03 04:02:01 INFO mapred.JobClient: map 100% reduce 56%
15/04/03 04:02:04 INFO mapred.JobClient: map 100% reduce 57%
15/04/03 04:02:19 INFO mapred.JobClient: map 100% reduce 58%
15/04/03 04:02:54 INFO mapred.JobClient: map 100% reduce 59%
15/04/03 04:03:18 INFO mapred.JobClient: map 100% reduce 61%
15/04/03 04:03:24 INFO mapred.JobClient: map 100% reduce 62%
15/04/03 04:03:42 INFO mapred.JobClient: map 100% reduce 63%
15/04/03 04:04:30 INFO mapred.JobClient: map 100% reduce 64%
15/04/03 04:05:28 INFO mapred.JobClient: map 100% reduce 65%
15/04/03 04:05:51 INFO mapred.JobClient: map 100% reduce 67%
15/04/03 04:05:54 INFO mapred.JobClient: map 100% reduce 68%
15/04/03 04:06:43 INFO mapred.JobClient: map 100% reduce 69%
15/04/03 04:07:16 INFO mapred.JobClient: map 100% reduce 71%
15/04/03 04:07:32 INFO mapred.JobClient: map 100% reduce 73%
15/04/03 04:07:34 INFO mapred.JobClient: map 100% reduce 74%
15/04/03 04:07:35 INFO mapred.JobClient: map 100% reduce 75%
15/04/03 04:08:04 INFO mapred.JobClient: map 100% reduce 77%
15/04/03 04:08:07 INFO mapred.JobClient: map 100% reduce 78%
15/04/03 04:08:24 INFO mapred.JobClient: map 100% reduce 79%
15/04/03 04:09:01 INFO mapred.JobClient: map 100% reduce 80%
15/04/03 04:09:18 INFO mapred.JobClient: map 100% reduce 82%
15/04/03 04:09:22 INFO mapred.JobClient: map 100% reduce 83%
15/04/03 04:09:50 INFO mapred.JobClient: map 100% reduce 84%
15/04/03 04:10:03 INFO mapred.JobClient: map 100% reduce 85%
15/04/03 04:10:11 INFO mapred.JobClient: map 100% reduce 86%
15/04/03 04:10:35 INFO mapred.JobClient: map 100% reduce 87%
15/04/03 04:11:32 INFO mapred.JobClient: map 100% reduce 88%
15/04/03 04:11:46 INFO mapred.JobClient: map 100% reduce 90%
15/04/03 04:11:59 INFO mapred.JobClient: map 100% reduce 91%
15/04/03 04:12:32 INFO mapred.JobClient: map 100% reduce 93%
15/04/03 04:12:35 INFO mapred.JobClient: map 100% reduce 94%
15/04/03 04:13:30 INFO mapred.JobClient: map 100% reduce 96%
15/04/03 04:13:33 INFO mapred.JobClient: map 100% reduce 97%
15/04/03 04:13:44 INFO mapred.JobClient: map 100% reduce 98%
15/04/03 04:13:47 INFO mapred.JobClient: map 100% reduce 99%
15/04/03 04:14:05 INFO mapred.JobClient: map 100% reduce 100%
15/04/03 04:14:14 INFO mapred.JobClient: Job complete: job_201504022127_0014
15/04/03 04:14:14 INFO mapred.JobClient: Counters: 32
15/04/03 04:14:14 INFO mapred.JobClient: Job Counters
15/04/03 04:14:14 INFO mapred.JobClient: Launched reduce tasks=31
15/04/03 04:14:14 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=3712107
15/04/03 04:14:14 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
15/04/03 04:14:14 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
15/04/03 04:14:14 INFO mapred.JobClient: Rack-local map tasks=3
15/04/03 04:14:14 INFO mapred.JobClient: Launched map tasks=54
15/04/03 04:14:14 INFO mapred.JobClient: Data-local map tasks=51
15/04/03 04:14:14 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=10240649
15/04/03 04:14:14 INFO mapred.JobClient: org.apache.mahout.vectorizer.collocations.llr.CollocMapper$Count
15/04/03 04:14:14 INFO mapred.JobClient: NGRAM_TOTAL=44741934
15/04/03 04:14:14 INFO mapred.JobClient: File Output Format Counters
15/04/03 04:14:14 INFO mapred.JobClient: Bytes Written=26509282
15/04/03 04:14:14 INFO mapred.JobClient: org.apache.mahout.vectorizer.collocations.llr.CollocReducer$Skipped
15/04/03 04:14:14 INFO mapred.JobClient: LESS_THAN_MIN_SUPPORT=68210076
15/04/03 04:14:14 INFO mapred.JobClient: FileSystemCounters
15/04/03 04:14:14 INFO mapred.JobClient: FILE_BYTES_READ=15324597526
15/04/03 04:14:14 INFO mapred.JobClient: HDFS_BYTES_READ=87173148
15/04/03 04:14:14 INFO mapred.JobClient: FILE_BYTES_WRITTEN=21404924440
15/04/03 04:14:14 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=26509282
15/04/03 04:14:14 INFO mapred.JobClient: File Input Format Counters
15/04/03 04:14:14 INFO mapred.JobClient: Bytes Read=87165324
15/04/03 04:14:14 INFO mapred.JobClient: Map-Reduce Framework
15/04/03 04:14:14 INFO mapred.JobClient: Map output materialized bytes=6112297187
15/04/03 04:14:14 INFO mapred.JobClient: Map input records=40000
15/04/03 04:14:14 INFO mapred.JobClient: Reduce shuffle bytes=6112297187
15/04/03 04:14:14 INFO mapred.JobClient: Spilled Records=424321293
15/04/03 04:14:14 INFO mapred.JobClient: Map output bytes=7393388643
15/04/03 04:14:14 INFO mapred.JobClient: Total committed heap usage (bytes)=11110109184
15/04/03 04:14:14 INFO mapred.JobClient: CPU time spent (ms)=1960340
15/04/03 04:14:14 INFO mapred.JobClient: Combine input records=422967584
15/04/03 04:14:14 INFO mapred.JobClient: SPLIT_RAW_BYTES=7824
15/04/03 04:14:14 INFO mapred.JobClient: Reduce input records=100254078
15/04/03 04:14:14 INFO mapred.JobClient: Reduce input groups=16732774
15/04/03 04:14:14 INFO mapred.JobClient: Combine output records=331241009
15/04/03 04:14:14 INFO mapred.JobClient: Physical memory (bytes) snapshot=14863052800
15/04/03 04:14:14 INFO mapred.JobClient: Reduce output records=3504184
15/04/03 04:14:14 INFO mapred.JobClient: Virtual memory (bytes) snapshot=61045489664
15/04/03 04:14:14 INFO mapred.JobClient: Map output records=191980653
15/04/03 04:18:42 INFO input.FileInputFormat: Total input paths to process : 24
15/04/03 04:18:43 INFO mapred.JobClient: Running job: job_201504022127_0015
15/04/03 04:18:44 INFO mapred.JobClient: map 0% reduce 0%
15/04/03 04:18:48 INFO mapred.JobClient: Task Id : attempt_201504022127_0015_m_000025_0, Status : FAILED
Error launching task
15/04/03 04:18:48 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave3:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0015_m_000025_0&filter=stdout
15/04/03 04:18:48 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave3:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0015_m_000025_0&filter=stderr
15/04/03 04:18:50 INFO mapred.JobClient: Task Id : attempt_201504022127_0015_m_000025_1, Status : FAILED
Error launching task
15/04/03 04:18:50 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0015_m_000025_1&filter=stdout
15/04/03 04:18:50 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0015_m_000025_1&filter=stderr
15/04/03 04:18:53 INFO mapred.JobClient: Task Id : attempt_201504022127_0015_r_000025_0, Status : FAILED
Error launching task
15/04/03 04:18:53 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0015_r_000025_0&filter=stdout
15/04/03 04:18:53 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504022127_0015_r_000025_0&filter=stderr
15/04/03 04:18:56 INFO mapred.JobClient: Task Id : attempt_201504022127_0015_m_000025_2, Status : FAILED
Error launching task
15/04/03 04:19:02 INFO mapred.JobClient: Job complete: job_201504022127_0015
15/04/03 04:19:02 INFO mapred.JobClient: Counters: 4
15/04/03 04:19:02 INFO mapred.JobClient: Job Counters
15/04/03 04:19:02 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=3263
15/04/03 04:19:02 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
15/04/03 04:19:02 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
15/04/03 04:19:02 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
Exception in thread "main" java.lang.IllegalStateException: Job failed!
at org.apache.mahout.vectorizer.collocations.llr.CollocDriver.computeNGramsPruneByLLR(CollocDriver.java:281)
at org.apache.mahout.vectorizer.collocations.llr.CollocDriver.generateAllGrams(CollocDriver.java:191)
at org.apache.mahout.vectorizer.DictionaryVectorizer.createTermFrequencyVectors(DictionaryVectorizer.java:183)
at org.apache.mahout.vectorizer.SparseVectorsFromSequenceFiles.run(SparseVectorsFromSequenceFiles.java:271)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.mahout.vectorizer.SparseVectorsFromSequenceFiles.main(SparseVectorsFromSequenceFiles.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
Warning: $HADOOP_HOME is deprecated.

15/04/03 04:19:07 WARN driver.MahoutDriver: No trainnb.props found on classpath, will use command-line arguments only
15/04/03 04:19:07 INFO common.AbstractJob: Command line arguments: {--alphaI=[1.0], --endPhase=[2147483647], --extractLabels=null, --input=[/home/fedora/Bayes/Output-comp/vectors/tfidf-vectors], --labelIndex=[/home/fedora/Bayes/Output-comp/labelindex], --output=[/home/fedora/Bayes/Output-comp/model], --overwrite=null, --startPhase=[0], --tempDir=[/home/fedora/Bayes/Output-comp/temp]}
15/04/03 04:23:55 INFO mapred.JobClient: Cleaning up the staging area hdfs://192.168.111.240:9000/tmp/hadoop-fedora/mapred/staging/fedora/.staging/job_201504022127_0016
15/04/03 04:23:55 ERROR security.UserGroupInformation: PriviledgedActionException as:fedora cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: /home/fedora/Bayes/Output-comp/vectors/tfidf-vectors
Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: /home/fedora/Bayes/Output-comp/vectors/tfidf-vectors
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:235)
at org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:55)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:252)
at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1054)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1071)
at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:983)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:550)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:580)
at org.apache.mahout.classifier.naivebayes.training.TrainNaiveBayesJob.run(TrainNaiveBayesJob.java:105)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.mahout.classifier.naivebayes.training.TrainNaiveBayesJob.main(TrainNaiveBayesJob.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)

(The following are the main command in the run.sh that are executed)
/home/fedora/HiBench/bin/../common/hibench/mahout/target/mahout-distribution-0.7/bin/mahout seq2sparse -Dmapred.map.output.compress=true -Dmapred.map.output.compress.codec=org.apache.hadoop.io.compress.DefaultCodec -Dmapred.output.compress=true -Dmapred.output.compression.codec=org.apache.hadoop.io.compress.DefaultCodec -Dmapred.output.compression.type=BLOCK -i /home/fedora/Bayes/Input-comp -o /home/fedora/Bayes/Output-comp/vectors -lnorm -nv -wt tfidf -ng 3 --numReducers 24
/home/fedora/HiBench/bin/../common/hibench/mahout/target/mahout-distribution-0.7/bin/mahout trainnb -Dmapred.map.output.compress=true -Dmapred.map.output.compress.codec=org.apache.hadoop.io.compress.DefaultCodec -Dmapred.output.compress=true -Dmapred.output.compression.codec=org.apache.hadoop.io.compress.DefaultCodec -Dmapred.output.compression.type=BLOCK -i /home/fedora/Bayes/Output-comp/vectors/tfidf-vectors -el -o /home/fedora/Bayes/Output-comp/model -li /home/fedora/Bayes/Output-comp/labelindex -ow --tempDir /home/fedora/Bayes/Output-comp/temp

from hibench.

adrian-wang commented on July 19, 2024

if you are using hadoop 1.2.1, then hadoop version should be set as hadoop1 instead of hadoop2.
export HADOOP_VERSION=hadoop1

from hibench.

echozyw commented on July 19, 2024

Thanks for your insight. But that is already set via the following statement:
export HADOOP_VERSION=hadoop2 # set it to hadoop1 to enable MR1, hadoop2 to enable MR2

Any other speculations? I am kindof stuck here. Any input will be greatly appreciated!

Thanks a lot,
Gina

from hibench.

adrian-wang commented on July 19, 2024

Can you get the log of tasktracker?

from hibench.

echozyw commented on July 19, 2024

Here is some high-level description: I have a 5-node vm-based cluster, including one master and 4 data/compute nodes. For the bayes workload, the following error shows up at the same place during ./run.sh. I checked all the 4 slave nodes (logs/tasktracker*.log and logs/userlogs/JOBFOLDER) and I found that only one slave has job logs (in logs/userlogs/JOBFOLDER) with "stderr" information which I also pasted below, along with the tasktracker.log of the same slave node. Thanks a lot in advance!
!!!!!!0) The Error output while performing: ./run.sh
15/04/07 03:15:03 INFO mapred.JobClient: map 84% reduce 0%
15/04/07 03:15:18 INFO mapred.JobClient: map 85% reduce 0%
15/04/07 03:15:20 INFO mapred.JobClient: map 86% reduce 0%
15/04/07 03:15:21 INFO mapred.JobClient: map 87% reduce 0%
15/04/07 03:15:39 INFO mapred.JobClient: map 88% reduce 0%
15/04/07 03:15:43 INFO mapred.JobClient: map 89% reduce 0%
15/04/07 03:15:51 INFO mapred.JobClient: map 90% reduce 0%
15/04/07 03:16:01 INFO mapred.JobClient: map 91% reduce 0%
15/04/07 03:16:16 INFO mapred.JobClient: map 93% reduce 0%
15/04/07 03:16:22 INFO mapred.JobClient: map 94% reduce 0%
15/04/07 03:16:31 INFO mapred.JobClient: map 95% reduce 0%
15/04/07 03:17:14 INFO mapred.JobClient: map 97% reduce 0%
15/04/07 03:17:30 INFO mapred.JobClient: map 99% reduce 0%
15/04/07 03:17:53 INFO mapred.JobClient: map 100% reduce 0%
15/04/07 03:17:55 INFO mapred.JobClient: Job complete: job_201504031414_0032
15/04/07 03:17:55 INFO mapred.JobClient: Counters: 20
15/04/07 03:17:55 INFO mapred.JobClient: Job Counters
15/04/07 03:17:55 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=1467609
15/04/07 03:17:55 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
15/04/07 03:17:55 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
15/04/07 03:17:55 INFO mapred.JobClient: Rack-local map tasks=13
15/04/07 03:17:55 INFO mapred.JobClient: Launched map tasks=60
15/04/07 03:17:55 INFO mapred.JobClient: Data-local map tasks=47
15/04/07 03:17:55 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=5694
15/04/07 03:17:55 INFO mapred.JobClient: File Output Format Counters
15/04/07 03:17:55 INFO mapred.JobClient: Bytes Written=87165324
15/04/07 03:17:55 INFO mapred.JobClient: FileSystemCounters
15/04/07 03:17:55 INFO mapred.JobClient: HDFS_BYTES_READ=181494754
15/04/07 03:17:55 INFO mapred.JobClient: FILE_BYTES_WRITTEN=2710790
15/04/07 03:17:55 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=87165324
15/04/07 03:17:55 INFO mapred.JobClient: File Input Format Counters
15/04/07 03:17:55 INFO mapred.JobClient: Bytes Read=181488418
15/04/07 03:17:55 INFO mapred.JobClient: Map-Reduce Framework
15/04/07 03:17:55 INFO mapred.JobClient: Map input records=40000
15/04/07 03:17:55 INFO mapred.JobClient: Physical memory (bytes) snapshot=3825807360
15/04/07 03:17:55 INFO mapred.JobClient: Spilled Records=0
15/04/07 03:17:55 INFO mapred.JobClient: CPU time spent (ms)=117680
15/04/07 03:17:55 INFO mapred.JobClient: Total committed heap usage (bytes)=1525678080
15/04/07 03:17:55 INFO mapred.JobClient: Virtual memory (bytes) snapshot=40598466560
15/04/07 03:17:55 INFO mapred.JobClient: Map output records=40000
15/04/07 03:17:55 INFO mapred.JobClient: SPLIT_RAW_BYTES=6336

15/04/07 03:21:59 INFO input.FileInputFormat: Total input paths to process : 48
15/04/07 03:22:00 INFO mapred.JobClient: Running job: job_201504031414_0033
15/04/07 03:22:01 INFO mapred.JobClient: map 0% reduce 0%
15/04/07 03:22:04 INFO mapred.JobClient: Task Id : attempt_201504031414_0033_m_000049_0, Status : FAILED
Error launching task
15/04/07 03:22:04 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504031414_0033_m_000049_0&filter=stdout
15/04/07 03:22:04 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504031414_0033_m_000049_0&filter=stderr
15/04/07 03:22:07 INFO mapred.JobClient: Task Id : attempt_201504031414_0033_m_000049_1, Status : FAILED
Error launching task
15/04/07 03:22:07 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave3:50060/tasklog?plaintext=true&attemptid=attempt_201504031414_0033_m_000049_1&filter=stdout
15/04/07 03:22:07 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave3:50060/tasklog?plaintext=true&attemptid=attempt_201504031414_0033_m_000049_1&filter=stderr
15/04/07 03:22:09 INFO mapred.JobClient: Task Id : attempt_201504031414_0033_r_000025_0, Status : FAILED
Error launching task
15/04/07 03:22:09 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave3:50060/tasklog?plaintext=true&attemptid=attempt_201504031414_0033_r_000025_0&filter=stdout
15/04/07 03:22:09 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave3:50060/tasklog?plaintext=true&attemptid=attempt_201504031414_0033_r_000025_0&filter=stderr
15/04/07 03:22:12 INFO mapred.JobClient: Task Id : attempt_201504031414_0033_m_000049_2, Status : FAILED
Error launching task
15/04/07 03:22:12 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504031414_0033_m_000049_2&filter=stdout
15/04/07 03:22:12 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave1:50060/tasklog?plaintext=true&attemptid=attempt_201504031414_0033_m_000049_2&filter=stderr
15/04/07 03:22:21 INFO mapred.JobClient: Job complete: job_201504031414_0033
15/04/07 03:22:21 INFO mapred.JobClient: Counters: 4
15/04/07 03:22:21 INFO mapred.JobClient: Job Counters
15/04/07 03:22:21 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=5062
15/04/07 03:22:21 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
15/04/07 03:22:21 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
15/04/07 03:22:21 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
Exception in thread "main" java.lang.IllegalStateException: Job failed!
at org.apache.mahout.vectorizer.collocations.llr.CollocDriver.generateCollocations(CollocDriver.java:239)
at org.apache.mahout.vectorizer.collocations.llr.CollocDriver.generateAllGrams(CollocDriver.java:188)
at org.apache.mahout.vectorizer.DictionaryVectorizer.createTermFrequencyVectors(DictionaryVectorizer.java:183)
at org.apache.mahout.vectorizer.SparseVectorsFromSequenceFiles.run(SparseVectorsFromSequenceFiles.java:271)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.mahout.vectorizer.SparseVectorsFromSequenceFiles.main(SparseVectorsFromSequenceFiles.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
Warning: $HADOOP_HOME is deprecated.

15/04/07 03:22:27 WARN driver.MahoutDriver: No trainnb.props found on classpath, will use command-line arguments only
15/04/07 03:22:27 INFO common.AbstractJob: Command line arguments: {--alphaI=[1.0], --endPhase=[2147483647], --extractLabels=null, --input=[/home/fedora/Bayes/Output-comp/vectors/tfidf-vectors], --labelIndex=[/home/fedora/Bayes/Output-comp/labelindex], --output=[/home/fedora/Bayes/Output-comp/model], --overwrite=null, --startPhase=[0], --tempDir=[/home/fedora/Bayes/Output-comp/temp]}
15/04/07 03:26:33 INFO mapred.JobClient: Cleaning up the staging area hdfs://192.168.111.240:9000/tmp/hadoop-fedora/mapred/staging/fedora/.staging/job_201504031414_0034
15/04/07 03:26:33 ERROR security.UserGroupInformation: PriviledgedActionException as:fedora cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: /home/fedora/Bayes/Output-comp/vectors/tfidf-vectors
Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: /home/fedora/Bayes/Output-comp/vectors/tfidf-vectors
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:235)
at org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:55)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:252)
at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1054)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1071)
at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:983)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:550)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:580)
at org.apache.mahout.classifier.naivebayes.training.TrainNaiveBayesJob.run(TrainNaiveBayesJob.java:105)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.mahout.classifier.naivebayes.training.TrainNaiveBayesJob.main(TrainNaiveBayesJob.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)

/home/fedora/HiBench/bin/../common/hibench/mahout/target/mahout-distribution-0.7/bin/mahout seq2sparse -Dmapred.map.output.compress=true -Dmapred.map.output.compress.codec=org.apache.hadoop.io.compress.DefaultCodec -Dmapred.output.compress=true -Dmapred.output.compression.codec=org.apache.hadoop.io.compress.DefaultCodec -Dmapred.output.compression.type=BLOCK -i /home/fedora/Bayes/Input-comp -o /home/fedora/Bayes/Output-comp/vectors -lnorm -nv -wt tfidf -ng 3 --numReducers 24
/home/fedora/HiBench/bin/../common/hibench/mahout/target/mahout-distribution-0.7/bin/mahout trainnb -Dmapred.map.output.compress=true -Dmapred.map.output.compress.codec=org.apache.hadoop.io.compress.DefaultCodec -Dmapred.output.compress=true -Dmapred.output.compression.codec=org.apache.hadoop.io.compress.DefaultCodec -Dmapred.output.compression.type=BLOCK -i /home/fedora/Bayes/Output-comp/vectors/tfidf-vectors -el -o /home/fedora/Bayes/Output-comp/model -li /home/fedora/Bayes/Output-comp/labelindex -ow --tempDir /home/fedora/Bayes/Output-comp/temp
Warning: $HADOOP_HOME is deprecated.

!!!!!!1) userlogs/JOBID/attemptID
1 log4j:WARN Failed to set property [conversionPattern] to value "%d{ISO8601} %p %c: %m%n".
2 java.lang.reflect.InvocationTargetException
3 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
4 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
5 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
6 at java.lang.reflect.Method.invoke(Method.java:606)
7 at org.apache.log4j.config.PropertySetter.setProperty(PropertySetter.java:206)
8 at org.apache.log4j.config.PropertySetter.setProperty(PropertySetter.java:165)
9 at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:130)
10 at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:97)
11 at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:684)
12 at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
13 at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:544)
14 at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:440)
15 at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:476)
16 at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:471)
17 at org.apache.log4j.LogManager.(LogManager.java:125)
18 at org.apache.log4j.Logger.getLogger(Logger.java:105)
19 at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:289)
20 at org.apache.commons.logging.impl.Log4JLogger.(Log4JLogger.java:109)
21 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
22 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
23 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
24 at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
25 at org.apache.commons.logging.impl.LogFactoryImpl.createLogFromClass(LogFactoryImpl.java:1116)
26 at org.apache.commons.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:914)
27 at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:604)
28 at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:336)
29 at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:310)
30 at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:685)
31 at org.apache.hadoop.mapred.Child.(Child.java:56)
32 Caused by: java.lang.ExceptionInInitializerError
33 at java.nio.file.FileSystems.getDefault(FileSystems.java:176)
34 at sun.util.calendar.ZoneInfoFile$1.run(ZoneInfoFile.java:490)
35 at sun.util.calendar.ZoneInfoFile$1.run(ZoneInfoFile.java:481)
36 at java.security.AccessController.doPrivileged(Native Method)
37 at sun.util.calendar.ZoneInfoFile.(ZoneInfoFile.java:480)
38 at sun.util.calendar.ZoneInfo.getTimeZone(ZoneInfo.java:663)
39 at java.util.TimeZone.getTimeZone(TimeZone.java:566)
40 at java.util.TimeZone.setDefaultZone(TimeZone.java:663)
41 at java.util.TimeZone.getDefaultRef(TimeZone.java:630)
42 at java.util.Calendar.getInstance(Calendar.java:968)
43 at org.apache.log4j.helpers.AbsoluteTimeDateFormat.(AbsoluteTimeDateFormat.java:62)
44 at org.apache.log4j.helpers.ISO8601DateFormat.(ISO8601DateFormat.java:46)
45 at org.apache.log4j.helpers.PatternParser.finalizeConverter(PatternParser.java:257)
46 at org.apache.log4j.helpers.PatternParser.parse(PatternParser.java:187)
47 at org.apache.log4j.PatternLayout.setConversionPattern(PatternLayout.java:446)
48 ... 29 more
49 Caused by: java.security.PrivilegedActionException: java.security.PrivilegedActionException: sun.nio.fs.UnixException: No such file or directory
50 at java.security.AccessController.doPrivileged(Native Method)
51 at java.nio.file.FileSystems$DefaultFileSystemHolder.defaultFileSystem(FileSystems.java:95)
52 at java.nio.file.FileSystems$DefaultFileSystemHolder.(FileSystems.java:90)
53 ... 44 more
54 Caused by: java.security.PrivilegedActionException: sun.nio.fs.UnixException: No such file or directory
55 at java.security.AccessController.doPrivileged(Native Method)
56 at sun.nio.fs.DefaultFileSystemProvider.createProvider(DefaultFileSystemProvider.java:42)
57 at sun.nio.fs.DefaultFileSystemProvider.create(DefaultFileSystemProvider.java:70)
58 at java.nio.file.FileSystems$DefaultFileSystemHolder.getDefaultProvider(FileSystems.java:108)
59 at java.nio.file.FileSystems$DefaultFileSystemHolder.access$000(FileSystems.java:89)
60 at java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:98)
61 at java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:96)
62 ... 47 more
63 Caused by: sun.nio.fs.UnixException: No such file or directory
64 at sun.nio.fs.UnixNativeDispatcher.getcwd(Native Method)
65 at sun.nio.fs.UnixFileSystem.(UnixFileSystem.java:67)
66 at sun.nio.fs.LinuxFileSystem.(LinuxFileSystem.java:39)
67 at sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:44)
68 at sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:37)
69 at sun.nio.fs.UnixFileSystemProvider.(UnixFileSystemProvider.java:56)
70 at sun.nio.fs.LinuxFileSystemProvider.(LinuxFileSystemProvider.java:39)
71 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
72 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
73 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
74 at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
75 at java.lang.Class.newInstance(Class.java:379)
76 at sun.nio.fs.DefaultFileSystemProvider$1.run(DefaultFileSystemProvider.java:52)
77 at sun.nio.fs.DefaultFileSystemProvider$1.run(DefaultFileSystemProvider.java:43)
78 ... 54 more
79 Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/xerces/xinclude/XIncludeHandler
80 at org.apache.xerces.parsers.XIncludeAwareParserConfiguration.configurePipeline(Unknown Source)
81 at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
82 at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
83 at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
84 at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
85 at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
86 at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:177)
87 at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1156)
88 at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1107)
89 at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1053)
90 at org.apache.hadoop.conf.Configuration.get(Configuration.java:397)
91 at org.apache.hadoop.mapred.JobConf.checkAndWarnDeprecation(JobConf.java:1899)
92 at org.apache.hadoop.mapred.JobConf.(JobConf.java:343)
93 at org.apache.hadoop.mapred.Child.main(Child.java:72)
94 Caused by: java.lang.ClassNotFoundException: org.apache.xerces.xinclude.XIncludeHandler
95 at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
96 at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
97 at java.security.AccessController.doPrivileged(Native Method)
98 at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
99 at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
100 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
101 at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
102 ... 14 more

!!!!!!2) logs/hadoop-fedora-tasktracker-newjobtestslave3.log
1 2015-04-07 03:04:57,360 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201504031414_0032_m_000003_0 task's state:UNASSIGNED
2 2015-04-07 03:04:57,361 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201504031414_0032_m_000003_0 which needs 1 slots
3 2015-04-07 03:04:57,362 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201504031414_0032_m_000003_0 which needs 1 slots
4 2015-04-07 03:04:57,364 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201504031414_0032_m_000006_0 task's state:UNASSIGNED
5 2015-04-07 03:04:57,364 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201504031414_0032_m_000006_0 which needs 1 slots
6 2015-04-07 03:04:57,365 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 1 and trying to launch attempt_201504031414_0032_m_000006_0 which needs 1 slots
7 2015-04-07 03:04:57,432 INFO org.apache.hadoop.mapred.JobLocalizer: Initializing user fedora on this TT.
8 2015-04-07 03:05:02,107 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201504031414_0032_m_-1006288762
9 2015-04-07 03:05:02,107 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201504031414_0032_m_-1006288762 spawned.
10 2015-04-07 03:05:02,109 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201504031414_0032_m_-1207767792
11 2015-04-07 03:05:02,111 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201504031414_0032_m_-1207767792 spawned.
12 2015-04-07 03:05:02,114 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /tmp/hadoop-fedora/mapred/local/ttprivate/taskTracker/fedora/jobcache/job_201504 031414_0032/attempt_201504031414_0032_m_000006_0/taskjvm.sh
13 2015-04-07 03:05:02,116 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /tmp/hadoop-fedora/mapred/local/ttprivate/taskTracker/fedora/jobcache/job_201504 031414_0032/attempt_201504031414_0032_m_000003_0/taskjvm.sh
14 2015-04-07 03:05:05,855 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201504031414_0032_m_-1207767792 given task: attempt_201504031414_0032_m_000003_0
15 2015-04-07 03:05:05,866 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201504031414_0032_m_-1006288762 given task: attempt_201504031414_0032_m_000006_0
16 2015-04-07 03:05:14,233 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000003_0 0.53208405%
17 2015-04-07 03:05:14,258 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000006_0 0.8006324%
18 2015-04-07 03:05:17,261 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000003_0 0.53208405%
19 2015-04-07 03:05:17,282 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000006_0 1.0%
20 2015-04-07 03:05:18,084 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201504031414_0032_m_000006_0 is in commit-pending, task state:COMMIT_PENDING
21 2015-04-07 03:05:18,084 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000006_0 1.0%
22 2015-04-07 03:05:18,277 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201504031414_0032_m_000004_1 task's state:UNASSIGNED
23 2015-04-07 03:05:18,277 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201504031414_0032_m_000004_1 which needs 1 slots
24 2015-04-07 03:05:18,277 INFO org.apache.hadoop.mapred.TaskTracker: TaskLauncher : Waiting for 1 to launch attempt_201504031414_0032_m_000004_1, currently we have 0 free s lots
25 2015-04-07 03:05:20,325 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000006_0 1.0%
26 2015-04-07 03:05:23,284 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000003_0 1.0%
27 2015-04-07 03:05:23,344 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000006_0 1.0%
28 2015-04-07 03:05:26,328 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000003_0 1.0%
29 2015-04-07 03:05:26,367 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000006_0 1.0%
30 2015-04-07 03:05:29,347 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000003_0 1.0%
31 2015-04-07 03:05:29,385 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000006_0 1.0%
32 2015-04-07 03:05:31,929 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201504031414_0032_m_000003_0 is in commit-pending, task state:COMMIT_PENDING
33 2015-04-07 03:05:31,929 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000003_0 1.0%
34 2015-04-07 03:05:32,220 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201504031414_0032_m_000005_1 task's state:UNASSIGNED
35 2015-04-07 03:05:32,402 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000006_0 1.0%
36 2015-04-07 03:05:35,485 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000006_0 1.0%
37 2015-04-07 03:05:38,369 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000003_0 1.0%
38 2015-04-07 03:05:38,504 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000006_0 1.0%
39 2015-04-07 03:05:41,394 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000003_0 1.0%
40 2015-04-07 03:05:41,524 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000006_0 1.0%
41 2015-04-07 03:05:44,411 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000003_0 1.0%
42 2015-04-07 03:05:44,541 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000006_0 1.0%
.............
523 2015-04-07 03:17:51,318 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000006_0 1.0%
524 2015-04-07 03:17:51,477 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000003_0 1.0%
525 2015-04-07 03:17:54,327 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000006_0 1.0%
526 2015-04-07 03:17:54,487 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201504031414_0032_m_000003_0 1.0%
527 2015-04-07 03:17:55,850 INFO org.apache.hadoop.mapred.TaskTracker: Received 'KillJobAction' for job: job_201504031414_0032
528 2015-04-07 03:17:55,857 INFO org.apache.hadoop.util.ProcessTree: Killing process group17828 with signal TERM. Exit code 0
529 2015-04-07 03:17:55,857 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 1
530 2015-04-07 03:17:55,858 INFO org.apache.hadoop.mapred.IndexCache: Map ID attempt_201504031414_0032_m_000003_0 not found in cache
531 2015-04-07 03:17:55,858 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 1 and trying to launch attempt_201504031414_0032_m_000004_1 which needs 1 slots
532 2015-04-07 03:17:55,862 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201504031414_0032_m_000005_1 which needs 1 slots
533 2015-04-07 03:17:55,862 INFO org.apache.hadoop.mapred.TaskTracker: TaskLauncher : Waiting for 1 to launch attempt_201504031414_0032_m_000005_1, currently we have 0 free s lots
534 2015-04-07 03:17:55,872 INFO org.apache.hadoop.util.ProcessTree: Killing process group17827 with signal TERM. Exit code 0
535 2015-04-07 03:17:55,872 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 1
536 2015-04-07 03:17:55,872 INFO org.apache.hadoop.mapred.IndexCache: Map ID attempt_201504031414_0032_m_000006_0 not found in cache
537 2015-04-07 03:17:55,873 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 1 and trying to launch attempt_201504031414_0032_m_000005_1 which needs 1 slots
538 2015-04-07 03:17:55,878 WARN org.apache.hadoop.mapred.DefaultTaskController: Exit code from task is : 143
539 2015-04-07 03:17:55,878 INFO org.apache.hadoop.mapred.DefaultTaskController: Output from DefaultTaskController's launchTask follows:
540 2015-04-07 03:17:55,878 INFO org.apache.hadoop.mapred.TaskController:
541 2015-04-07 03:17:55,878 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201504031414_0032_m_-1207767792 exited with exit code 143. Number of tasks it ran: 0
542 2015-04-07 03:17:55,879 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201504031414_0032 for user-log deletion with retainTimeStamp:1428463075878
543 2015-04-07 03:17:55,927 INFO org.apache.hadoop.io.nativeio.NativeIO: Got UserName fedora for UID 1000 from the native implementation
544 2015-04-07 03:17:55,933 WARN org.apache.hadoop.mapred.DefaultTaskController: Exit code from task is : 143
545 2015-04-07 03:17:55,933 INFO org.apache.hadoop.mapred.DefaultTaskController: Output from DefaultTaskController's launchTask follows:
546 2015-04-07 03:17:55,933 INFO org.apache.hadoop.mapred.TaskController:
547 2015-04-07 03:17:55,933 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201504031414_0032_m_-1006288762 exited with exit code 143. Number of tasks it ran: 0
548 2015-04-07 03:17:55,929 WARN org.apache.hadoop.mapred.TaskTracker: Error initializing attempt_201504031414_0032_m_000005_1:
549 java.io.FileNotFoundException: File does not exist: hdfs://192.168.111.240:9000/tmp/hadoop-fedora/mapred/system/job_201504031414_0032/jobToken
550 at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:558)
551 at org.apache.hadoop.mapred.TaskTracker.localizeJobTokenFile(TaskTracker.java:4489)
552 at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1285)
553 at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
554 at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2603)
555 at java.lang.Thread.run(Thread.java:745)
556
557 2015-04-07 03:17:55,934 ERROR org.apache.hadoop.mapred.TaskStatus: Trying to set finish time for task attempt_201504031414_0032_m_000005_1 when no start time is set, stac kTrace is : java.lang.Exception
558 at org.apache.hadoop.mapred.TaskStatus.setFinishTime(TaskStatus.java:145)
559 at org.apache.hadoop.mapred.TaskTracker$TaskInProgress.kill(TaskTracker.java:3326)
560 at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2613)
561 at java.lang.Thread.run(Thread.java:745)
562
563 2015-04-07 03:17:55,934 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 1
564 2015-04-07 03:17:55,943 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201504031414_0032_m_708372940
565 2015-04-07 03:17:55,946 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201504031414_0032_m_708372940 spawned.
566 2015-04-07 03:17:55,955 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /tmp/hadoop-fedora/mapred/local/ttprivate/taskTracker/fedora/jobcache/job_201504 031414_0032/attempt_201504031414_0032_m_000004_1/taskjvm.sh
567 2015-04-07 03:17:57,345 WARN org.apache.hadoop.mapred.DefaultTaskController: Exit code from task is : 1
568 2015-04-07 03:17:57,345 INFO org.apache.hadoop.mapred.DefaultTaskController: Output from DefaultTaskController's launchTask follows:
569 2015-04-07 03:17:57,345 INFO org.apache.hadoop.mapred.TaskController:
570 2015-04-07 03:17:57,345 INFO org.apache.hadoop.mapred.JvmManager: JVM Not killed jvm_201504031414_0032_m_708372940 but just removed
571 2015-04-07 03:17:57,345 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201504031414_0032_m_708372940 exited with exit code 1. Number of tasks it ran: 0
572 2015-04-07 03:17:57,346 WARN org.apache.hadoop.mapred.TaskRunner: attempt_201504031414_0032_m_000004_1 : Child Error
573 java.io.IOException: Task process exit with nonzero status of 1.
574 at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
575 2015-04-07 03:18:00,348 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
576 2015-04-07 03:22:04,053 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201504031414_0033_m_000049_1 task's state:UNASSIGNED
577 2015-04-07 03:22:04,053 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201504031414_0033_m_000049_1 which needs 1 slots
578 2015-04-07 03:22:04,053 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201504031414_0033_m_000049_1 which needs 1 slots
579 2015-04-07 03:22:04,141 INFO org.apache.hadoop.mapred.JobLocalizer: Initializing user fedora on this TT.
580 2015-04-07 03:22:06,774 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201504031414_0033_r_000025_0 task's state:UNASSIGNED
581 2015-04-07 03:22:06,775 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201504031414_0033_r_000025_0 which needs 1 slots
582 2015-04-07 03:22:06,775 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 4 and trying to launch attempt_201504031414_0033_r_000025_0 which needs 1 slots
583 2015-04-07 03:22:21,913 INFO org.apache.hadoop.mapred.TaskTracker: Cleanup for id job_201504031414_0033 skipped as its localizing.
584 2015-04-07 03:22:25,383 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201504031414_0033_r_284599715
585 2015-04-07 03:22:25,383 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201504031414_0033_r_284599715 spawned.
586 2015-04-07 03:22:25,384 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201504031414_0033_m_-1040099847
587 2015-04-07 03:22:25,387 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201504031414_0033_m_-1040099847 spawned.
588 2015-04-07 03:22:25,390 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /tmp/hadoop-fedora/mapred/local/ttprivate/taskTracker/fedora/jobcache/job_201504 031414_0033/attempt_201504031414_0033_r_000025_0/taskjvm.sh
589 2015-04-07 03:22:25,392 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /tmp/hadoop-fedora/mapred/local/ttprivate/taskTracker/fedora/jobcache/job_201504 031414_0033/attempt_201504031414_0033_m_000049_1/taskjvm.sh
590 2015-04-07 03:22:25,914 INFO org.apache.hadoop.mapred.TaskTracker: Received 'KillJobAction' for job: job_201504031414_0033
591 2015-04-07 03:22:25,914 INFO org.apache.hadoop.mapred.JvmManager: JVM Not killed jvm_201504031414_0033_m_-1040099847 but just removed
592 2015-04-07 03:22:25,914 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
593 2015-04-07 03:22:25,915 INFO org.apache.hadoop.mapred.IndexCache: Map ID attempt_201504031414_0033_m_000049_1 not found in cache
594 2015-04-07 03:22:25,915 INFO org.apache.hadoop.mapred.JvmManager: JVM Not killed jvm_201504031414_0033_r_284599715 but just removed
595 2015-04-07 03:22:25,915 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 4
596 2015-04-07 03:22:25,929 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201504031414_0033 for user-log deletion with retainTimeStamp:1428463345916
597 2015-04-07 03:22:27,391 WARN org.apache.hadoop.mapred.DefaultTaskController: Exit code from task is : 1
598 2015-04-07 03:22:27,391 INFO org.apache.hadoop.mapred.DefaultTaskController: Output from DefaultTaskController's launchTask follows:
599 2015-04-07 03:22:27,391 INFO org.apache.hadoop.mapred.TaskController:
600 2015-04-07 03:22:27,391 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201504031414_0033_r_284599715 exited with exit code 1. Number of tasks it ran: 0
601 2015-04-07 03:22:27,405 WARN org.apache.hadoop.mapred.DefaultTaskController: Exit code from task is : 1
602 2015-04-07 03:22:27,405 INFO org.apache.hadoop.mapred.DefaultTaskController: Output from DefaultTaskController's launchTask follows:
603 2015-04-07 03:22:27,405 INFO org.apache.hadoop.mapred.TaskController:
604 2015-04-07 03:22:27,405 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201504031414_0033_m_-1040099847 exited with exit code 1. Number of tasks it ran: 0
605 2015-04-07 03:22:28,917 WARN org.apache.hadoop.mapred.TaskTracker: Unknown job job_201504031414_0033 being deleted.
606 2015-04-07 03:22:28,917 WARN org.apache.hadoop.mapred.TaskTracker: Unknown job job_201504031414_0033 being deleted.

from hibench.

adrian-wang commented on July 19, 2024

It seems the input path does not exists or cannot be found. Did you run bayes/bin/prepare.sh before you run run.sh or did you get any exception during prepare?

from hibench.

echozyw commented on July 19, 2024

I did run prepare.sh and it succeeded.
When I run run.sh, it successfully completed several jobs and then failed with the following output errors. I have 2 questions here:

What caused the "15/04/07 03:22:04 WARN mapred.JobClient: Error reading task outputhttp://newjobtestSlave2:50060/tasklog?plaintext=true&attemptid=attempt_201504031414_0033_m_000049_0&filter=stderr
15/04/07 03:22:07 INFO mapred.JobClient: Task Id : attempt_201504031414_0033_m_000049_1, Status : FAILED
Error launching task
15/04/07 03:22:07 WARN mapred.JobClient: Error reading task "?
It seems that the run.sh failed at "Exception in thread "main" java.lang.IllegalStateException: Job failed!
at org.apache.mahout.vectorizer.collocations.llr.CollocDriver.generateCollocations(CollocDriver.java:239)
at org.apache.mahout.vectorizer.collocations.llr.CollocDriver.generateAllGrams(CollocDriver.java:188)
at ". What should I check for the debugging?

Thanks a lot!

======================here is the error output==============================
$./run.sh
...... (successful jobs)
15/04/07 03:16:31 INFO mapred.JobClient: map 95% reduce 0%
15/04/07 03:17:14 INFO mapred.JobClient: map 97% reduce 0%
15/04/07 03:17:30 INFO mapred.JobClient: map 99% reduce 0%
15/04/07 03:17:53 INFO mapred.JobClient: map 100% reduce 0%
15/04/07 03:17:55 INFO mapred.JobClient: Job complete: job_201504031414_0032
15/04/07 03:17:55 INFO mapred.JobClient: Counters: 20
15/04/07 03:17:55 INFO mapred.JobClient: Job Counters
15/04/07 03:17:55 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=1467609
15/04/07 03:17:55 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
15/04/07 03:17:55 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
15/04/07 03:17:55 INFO mapred.JobClient: Rack-local map tasks=13
15/04/07 03:17:55 INFO mapred.JobClient: Launched map tasks=60
15/04/07 03:17:55 INFO mapred.JobClient: Data-local map tasks=47
15/04/07 03:17:55 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=5694
15/04/07 03:17:55 INFO mapred.JobClient: File Output Format Counters
15/04/07 03:17:55 INFO mapred.JobClient: Bytes Written=87165324
15/04/07 03:17:55 INFO mapred.JobClient: FileSystemCounters
15/04/07 03:17:55 INFO mapred.JobClient: HDFS_BYTES_READ=181494754
15/04/07 03:17:55 INFO mapred.JobClient: FILE_BYTES_WRITTEN=2710790
15/04/07 03:17:55 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=87165324
15/04/07 03:17:55 INFO mapred.JobClient: File Input Format Counters
15/04/07 03:17:55 INFO mapred.JobClient: Bytes Read=181488418
15/04/07 03:17:55 INFO mapred.JobClient: Map-Reduce Framework
15/04/07 03:17:55 INFO mapred.JobClient: Map input records=40000
15/04/07 03:17:55 INFO mapred.JobClient: Physical memory (bytes) snapshot=3825807360
15/04/07 03:17:55 INFO mapred.JobClient: Spilled Records=0
15/04/07 03:17:55 INFO mapred.JobClient: CPU time spent (ms)=117680
15/04/07 03:17:55 INFO mapred.JobClient: Total committed heap usage (bytes)=1525678080
15/04/07 03:17:55 INFO mapred.JobClient: Virtual memory (bytes) snapshot=40598466560
15/04/07 03:17:55 INFO mapred.JobClient: Map output records=40000
15/04/07 03:17:55 INFO mapred.JobClient: SPLIT_RAW_BYTES=6336

from hibench.

adrian-wang commented on July 19, 2024

You HiBench complains Input path does not exist: /home/fedora/Bayes/Output-comp/vectors/tfidf-vectors, which should be hdfs://HiBench/Bayes/Output-comp/vectors/tfidf-vectors
This is the problem.
Please try to set HADOOP_HOME in your bash environment explicitly, it may be required by earlier mahout versions.

from hibench.

adrian-wang commented on July 19, 2024

Did your problem get resolved?

from hibench.

echozyw commented on July 19, 2024

Thank you for checking. But it still has the same issue. I am pretty sure that I already set HADOOP_HOME in my bash environment explicitly, when I ran into those errors. I am giving it another try to see how it goes. Will report back once it is done/dead...

from hibench.

echozyw commented on July 19, 2024

It is dead as before. The issue seems to be that: The 3rd job (the first 2 jobs seem to have passed) that came along when I do ./run.sh complains that "ERROR security.UserGroupInformation: PriviledgedActionException as:fedora cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: /HiBench/Bayes/Output-comp/vectors/tfidf-vectors".
And here is the output when I ls the hadoop folder:
[fedora@newjobtestmaster ~]$ hadoop dfs -ls /HiBench/Bayes/Output-comp/vectors
Found 2 items
drwxr-xr-x - fedora supergroup 0 2015-05-05 04:26 /HiBench/Bayes/Output-comp/vectors/tokenized-documents
drwxr-xr-x - fedora supergroup 0 2015-05-05 05:54 /HiBench/Bayes/Output-comp/vectors/wordcount

When/where is the missing file "tfidf-vectors" supposed to be generated?
(Is there any way that I can attach my error trace file here? It seems a bit too long to store as an image.)

Thank you very much for your help in advance,
Z

from hibench.

Hadoop 1.2.1, HiBench 3.0.0 and Mahout 0.7 compatible? about hibench HOT 15 OPEN

Comments (15)

base dir HDFS

local report

swith on/off compression: 0-off, 1-on.

Switch it off (COMPRESS_GLOBAL=0) for better performance

Set COMPRESS_CODEC_MAP to SnappyCodec (as shown below) for better performance

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

Jobs