Presto query fails when querying on S3 external table about emr-bootstrap-actions

[8maki]

me too.

[croblee]

Facing a similar issue, but I get the error even with the sample commands in the README:

presto:default> select * from apachelog;
Query 20150121_233928_00009_axsg9, FAILED, 1 node
http://localhost:8080/v1/query/20150121_233928_00009_axsg9?pretty
Splits: 1 total, 0 done (0.00%)
CPU Time: 0.0s total, 0 rows/s, 0B/s, 0% active
Per Node: 0.0 parallelism, 0 rows/s, 0B/s
Parallelism: 0.0
0:00 [0 rows, 0B] [0 rows/s, 0B/s]

Query 20150121_233928_00009_axsg9 failed: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3 URL, or by setting the fs.s3.awsAccessKeyId or fs.s3.awsSecretAccessKey properties (respectively).
com.facebook.presto.spi.PrestoException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3 URL, or by setting the fs.s3.awsAccessKeyId or fs.s3.awsSecretAccessKey properties (respectively).
at com.facebook.presto.hive.HiveSplitSourceProvider$HiveSplitSource.propagatePrestoException(HiveSplitSourceProvider.java:599)
at com.facebook.presto.hive.HiveSplitSourceProvider$HiveSplitSource.getNextBatch(HiveSplitSourceProvider.java:542)
at com.facebook.presto.execution.ConnectorAwareSplitSource.getNextBatch(ConnectorAwareSplitSource.java:46)
at com.facebook.presto.execution.SqlStageExecution.scheduleSourcePartitionedNodes(SqlStageExecution.java:646)
at com.facebook.presto.execution.SqlStageExecution.startTasks(SqlStageExecution.java:565)
at com.facebook.presto.execution.SqlStageExecution.access$200(SqlStageExecution.java:97)
at com.facebook.presto.execution.SqlStageExecution$4.run(SqlStageExecution.java:537)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3 URL, or by setting the fs.s3.awsAccessKeyId or fs.s3.awsSecretAccessKey properties (respectively).
at org.apache.hadoop.fs.s3.S3Credentials.initialize(S3Credentials.java:70)
at com.facebook.presto.hive.PrestoS3FileSystem.getAwsCredentials(PrestoS3FileSystem.java:447)
at com.facebook.presto.hive.PrestoS3FileSystem.initialize(PrestoS3FileSystem.java:145)
at org.apache.hadoop.fs.PrestoFileSystemCache.createFileSystem(PrestoFileSystemCache.java:74)
at org.apache.hadoop.fs.PrestoFileSystemCache.getInternal(PrestoFileSystemCache.java:61)
at org.apache.hadoop.fs.PrestoFileSystemCache.get(PrestoFileSystemCache.java:43)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:287)
at com.facebook.presto.hive.HdfsEnvironment.getFileSystem(HdfsEnvironment.java:60)
at com.facebook.presto.hive.HiveSplitSourceProvider.loadPartitionSplits(HiveSplitSourceProvider.java:223)
at com.facebook.presto.hive.HiveSplitSourceProvider.access$100(HiveSplitSourceProvider.java:81)
at com.facebook.presto.hive.HiveSplitSourceProvider$2.run(HiveSplitSourceProvider.java:170)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at com.facebook.presto.hive.util.BoundedExecutor.executeOrMerge(BoundedExecutor.java:82)
at com.facebook.presto.hive.util.BoundedExecutor.access$000(BoundedExecutor.java:41)
at com.facebook.presto.hive.util.BoundedExecutor$1.run(BoundedExecutor.java:53)
... 3 more

I am using a custom instance profile. Does the PrestoS3FileSystem support instance profile credentials?

[pmanwatkar]

@achiku were you able to resolve this issue. I am facing this issue even after providing all s3 access to the instance. It worked fine with Hive though.

[pmanwatkar]

@achiku I got this resolved by creating a EMR cluster without any service-role or instance-profile-role.
When you use IAM roles, core-site.xml will not have these things:
fs.s3n.awsAccessKeyId

fs.s3.awsAccessKeyId
fs.s3bfs.awsAccessKeyId

And presto only get access creds from these properties.

[achiku]

@pmanwatkar Yeah, we did the same. But I really don't want to use accesskey/secret since we have to store them in somewhere. I talked to AWS support, but this behavior will not be fixed soon.

[pmanwatkar]

I raised a ticket with AWS and they have now changed the presto bootstrap location to:
s3://support.elasticmapreduce/bootstrap-actions/presto/install-presto.rb

Try this out it will work on IAM roles as well.

[achiku]

@pmanwatkar I haven't noticed but this is great news! Thanks! I'll close this issue.