S3 Upload - Bucket [BUCKET_NAME] does not exist or you do not have access about aws-toolkit-azure-devops

[jsancho]

I've just deleted an earlier comment that wasn't that accurate.
In order to avoid confusion, I'll summarising it all here.

When I narrow down all the permissions that are required for the upload.
I end up with the following:

{
    "Version": "2012-10-17",
    "Statement": [
      {
            "Sid": "LetMeSeeTheBucketSoICanUploadToIt",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:PutObject"
            ],
            "Resource": [
                "arn:aws:s3:::bucket-name"
            ]
       }
    ]
}

With this configuration I can even create subfolders and overwrite files in the bucket.

The takeaways are:

• All 3 actions above are required
• The bucket needs to be specified as the plain root path, the version with the wildcard at the end "/*" is not necessary

Thanks so much for looking into this and pointing me in the right direction
top work! 👍

[jsancho]

I've been doing some further testing today.
With a fresh AWS account even, and I can reproduce the issue with a minimal subset of tasks.

The upload does actually work fine I use the root AWS account.
However, the upload fails with the "bucket name does not exist or you do not have access..." whenever I use an account other than the root one.

This limited account has the following policy attached.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "Stmt1503241903611", "Action": "s3:*", "Effect": "Allow", "Resource": "arn:aws:s3:::fake-bucket-name/*" } ] }

I have also tried using a bucket policy and assigning the principal, but that's made no difference.

As with my main organisations' account, I can use "aws s3 cp" with the limited account key and secret.

[stevejroberts]

Thanks for digging into this. I've repro'd with the policy shown and am investigating further.

The task performs a HEAD operation before running, which, according to the linked S3 documentation on permissions should need more permissions than your policy shows. However, even after adding ListBucket and GetBucketLocation permissions I didn't see a change in behavior. The CLI's S3 cp command presumably doesn't do a HEAD request to test bucket existence, so it works fine.

It's possible I missed something in the permissions so am continuing to look. If that doesn't yield any results, maybe there is a bug in the underlying AWS SDK for Node.js, so I'll query with that team to see if they have any ideas too.

[jsancho]

Alright, that makes sense.
By the way, it's absolutely understandable that things like these come up on such early days.

Thanks a lot for the effort on putting this project together, it's going to make our delivery workflows so much easier moving forward :)

[stevejroberts]

Following the documentation here, this is the policy I crafted for a limited account that allowed upload (only) to my test bucket, 'steve-limited' -

{ "Version": "2012-10-17", "Statement": [ { "Sid": "LetMyLimitedAccountSeeTheBucket", "Effect": "Allow", "Action": [ "s3:GetBucketLocation", "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::*" ] }, { "Sid": "LetMyLimitedAccountUploadToTheBucket", "Effect": "Allow", "Action": [ "s3:PutObject" ], "Resource": [ "arn:aws:s3:::steve-limited", "arn:aws:s3:::steve-limited/*" ] } ] }

Let me know if this works for you too? Using the information on the linked page, you might want to customize further.

Thanks for the comments too!

[stevejroberts]

Great to hear you are up and running. Did you also manage to confirm the AWS CLI task worked when you had this policy in place?

[jsancho]

Afraid that I've not managed to get the CLI task to work, I'll add the comments in that issue

[stevejroberts]

Ok. I'll close this issue and we can move over to the CLI one.