dfsioe yarn error about hibench

[colorant]

Does not encounter this issue on my cluster, please check your hadoop settings, is there any privilege issues etc.

[atupal]

I fix it by change some code in the IOMapperBase.java. And I am using the cdh4.2,,
I just move the code in the generate method IOMapperBase to the configure method , otherwise the job will not get the correct configuration of the hdfs . it workds for me.

[jason-dai]

Maybe you can send a pull request for this, so that we can take a look at the fix and potentially integrate it into the project? Thanks.

[viplav]

Hi,
We have been seeing the same issue with Hortonworks HDP 2 (at least since HDP 2.2). We really like dfsioe compared to standard Hadoop DFSIO, and I found some time to debug the issue. Tested a fix. Should I send a Pull request so that others who want to use dfsioe with HDP can benefit? Thanks.

[viplav]

I have sent a pull request ( #110 ) to fix the issue with using wrong FS resulting in errors like "Mkdirs failed to create" etc. Please take a look and merge so that people who run into this issue (e.g. using Hortonworks HDP 2) will benefit.

[somerandomsequence]

I'm still getting this error with the current master branch. Has it been merged in to master?

[adrian-wang]

cc @carsonwang

[carsonwang]

The fix mentioned in this issue was included in the master branch. We also fixed another issue in DFSIOe #276 . Please check the command that submits the application and what is the value of test.build.data.

[kumarshirish]

I get the following error while running dfsioe:

17/05/16 16:30:11 INFO dfsioe.TestDFSIOEnh: creating control file: 1 mega bytes, 16 files
17/05/16 16:30:11 INFO dfsioe.DfsioeConfig: Found 'test.build.data' in System properties, value is 'hdfs://nameservice1:8020/HiBench/Dfsioe/Input'
17/05/16 16:30:11 INFO dfsioe.DfsioeConfig: Setting testRootDir to 'hdfs://nameservice1:8020/HiBench/Dfsioe/Input'
java.lang.IllegalArgumentException: Wrong FS: hdfs://nameservice1:8020/HiBench/Dfsioe/Input/io_control, expected: hdfs://nameservice1
        at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:657)
        at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:194)
        at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:106)
        at org.apache.hadoop.hdfs.DistributedFileSystem$12.doCall(DistributedFileSystem.java:646)
        at org.apache.hadoop.hdfs.DistributedFileSystem$12.doCall(DistributedFileSystem.java:642)
        at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
        at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:642)
        at org.apache.hadoop.fs.dfsioe.TestDFSIOEnh.createControlFile(TestDFSIOEnh.java:640)
        at org.apache.hadoop.fs.dfsioe.TestDFSIOEnh.run(TestDFSIOEnh.java:602)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at org.apache.hadoop.fs.dfsioe.TestDFSIOEnh.main(TestDFSIOEnh.java:628)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Here is the output on the console:

patching args=
Parsing conf: /var/lib/hadoop-hdfs/HiBench/conf/99-user_defined_properties.conf
Parsing conf: /var/lib/hadoop-hdfs/HiBench/conf/hadoop.conf
Parsing conf: /var/lib/hadoop-hdfs/HiBench/conf/hibench.conf
Parsing conf: /var/lib/hadoop-hdfs/HiBench/conf/spark.conf
Parsing conf: /var/lib/hadoop-hdfs/HiBench/conf/workloads/micro/dfsioe.conf
probe sleep jar: /opt/cloudera/parcels/CDH/lib/hadoop/../../jars/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.8.0-tests.jar
start HadoopDfsioe-write bench
/var/lib/hadoop-hdfs/HiBench/bin/functions/workload_functions.sh: line 115: /dev/stderr: Permission denied
rm: `hdfs://nameservice1:8020/HiBench/Dfsioe/Output': No such file or directory
/var/lib/hadoop-hdfs/HiBench/bin/functions/workload_functions.sh: line 152: /dev/stderr: Permission denied
du: `hdfs://nameservice1:8020/HiBench/Dfsioe/Input': No such file or directory
Submit MapReduce Job: /opt/cloudera/parcels/CDH/lib/hadoop/bin/hadoop --config /opt/cloudera/parcels/CDH/lib/hadoop/etc/hadoop jar /var/lib/hadoop-hdfs/HiBench/autogen/target/autogen-6.1-SNAPSHOT-jar-with-dependencies.jar org.apache.hadoop.fs.dfsioe.TestDFSIOEnh -Dmapreduce.map.java.opts="-Dtest.build.data=hdfs://nameservice1:8020/HiBench/Dfsioe/Input -Djava.net.preferIPv4Stack=true" -Dmapreduce.reduce.java.opts="-Dtest.build.data=hdfs://nameservice1:8020/HiBench/Dfsioe/Input -Djava.net.preferIPv4Stack=true" -write -nrFiles 16 -fileSize 1 -bufferSize 4096 -plotInteval 1000 -sampleUnit m -sampleInteval 200 -sumThreshold 0.5 -tputReportTotal -Dtest.build.data=hdfs://nameservice1:8020/HiBench/Dfsioe/Input -resFile /var/lib/hadoop-hdfs/HiBench/report/dfsioe/hadoop/conf/../result_write.txt -tputFile /var/lib/hadoop-hdfs/HiBench/report/dfsioe/hadoop/conf/../throughput_write.csv
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
ERROR: Hadoop job /var/lib/hadoop-hdfs/HiBench/autogen/target/autogen-6.1-SNAPSHOT-jar-with-dependencies.jar org.apache.hadoop.fs.dfsioe.TestDFSIOEnh failed to run successfully.
Hint: You can goto /var/lib/hadoop-hdfs/HiBench/report/dfsioe/hadoop/conf/../bench.log to check for detailed log.
Opening log tail for you:

	at org.apache.hadoop.fs.dfsioe.TestDFSIOEnh.run(TestDFSIOEnh.java:602)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
	at org.apache.hadoop.fs.dfsioe.TestDFSIOEnh.main(TestDFSIOEnh.java:628)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

This happens whether while running prepare, read, or run_all.sh.