Support for AssumeRoleWithWebIdentity? about collector

[lfittl]

@booleanbetrayal Wasn't too much work :)

PR here: #181

You can try this out by using the special Docker image quay.io/pganalyze/collector:hotfix-aws-web-identity-support that I just built - could you give this a try and see whether it works as expected?

[lfittl]

Hi @booleanbetrayal - thanks for opening this issue, I think its a good idea to add direct support for this.

That said, I'm wondering if you can utilize the environment variable based configuration to verify whether this works as intended today, without any code changes:

The SDK supports the same variables as the CLI (see https://docs.aws.amazon.com/cli/latest/topic/config-vars.html#assume-role-with-web-identity at the end), so I think you can set the following:

• AWS_ROLE_ARN: The ARN of the role you want to assume.
• AWS_WEB_IDENTITY_TOKEN_FILE: The path to the web identity token file.

If I understand the AWS documentation correctly, you can simply set AWS_WEB_IDENTITY_TOKEN_FILE to /var/run/secrets/eks.amazonaws.com/serviceaccount/token, and the AWS_ROLE_ARN to the correct role, and assuming you have the annotations set correctly, that should just work.

One more thing you might have to do is permit the pganalyze user to access the identity token file, as documented here: https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-technical-overview.html#pod-configuration

Let me know if that works, I'd be curious :)

[booleanbetrayal]

So I know this "just works" in the Ruby SDK -

irb(main):008:0> require 'aws-sdk-s3'
=> true
irb(main):009:0> ENV['AWS_WEB_IDENTITY_TOKEN_FILE']
=> "/var/run/secrets/eks.amazonaws.com/serviceaccount/token"
irb(main):010:0> ENV['AWS_ROLE_ARN'].length
=> 50
irb(main):011:0> 
irb(main):012:0> s3 = Aws::S3::Resource.new
=> #<Aws::S3::Resource:0x00007f79c4f1d688 @client=#<Aws::S3::Client>>
irb(main):013:0> s3.bucket(ENV['AWS_LOGGING_BUCKET']).object("test/#{Time.now.strftime('%Y-%m-%d_%H-%M-%S-%12N')}").put(body: "test")
=> #<struct Aws::S3::Types::PutObjectOutput expiration=nil, etag="\"*************\"", server_side_encryption=nil, version_id=nil, sse_customer_algorithm=nil, sse_customer_key_md5=nil, ssekms_key_id=nil, ssekms_encryption_context=nil, bucket_key_enabled=nil, request_charged=nil>

So I'm a bit surprised it's not working in the Go SDK. I'll try to revisit the configuration (reverted to IAM user access credentials) and will let you know if it's still reproducible or if there's some obvious misconfiguration, @lfittl . I was pretty confident I had dotted all my "i's" and crossed all my "t's", but maybe I had missed something.

[booleanbetrayal]

Actually, @lfittl - I'm wondering if the issue is in the creation of the provider chain via: https://github.com/pganalyze/collector/blob/master/util/awsutil/amazon.go#L56-L66

According to the documentation, EnvProvider only supports:

• Access Key ID: AWS_ACCESS_KEY_ID or AWS_ACCESS_KEY
• Secret Access Key: AWS_SECRET_ACCESS_KEY or AWS_SECRET_KEY

So maybe by explicitly configuring the provider chain for credentials creation, the legs are being cut out from some other automated WebIdentity provider support (if it even exists)?

[lfittl]

Actually, @lfittl - I'm wondering if the issue is in the creation of the provider chain via: https://github.com/pganalyze/collector/blob/master/util/awsutil/amazon.go#L56-L66

According to the documentation, EnvProvider only supports:

• Access Key ID: AWS_ACCESS_KEY_ID or AWS_ACCESS_KEY
• Secret Access Key: AWS_SECRET_ACCESS_KEY or AWS_SECRET_KEY

So maybe by explicitly configuring the provider chain for credentials creation, the legs are being cut out from some other automated WebIdentity provider support (if it even exists)?

Yeah, you might be right, looking at the code again.

I think its relatively straightforward to add this support, let me give that a try & will share a PR in a bit.

[booleanbetrayal]

Thanks @lfittl ! Not urgent, but appreciated!