cloudgraphdev / cloudgraph-provider-aws Goto Github PK

Although several (generally older) services do not support the Cloud Control API, the newer ones tend to do so. Implementing this would extend the supported services to most of AWS without having to write per-service implementations. Please see the list of supported resource types.

Describe the bug

When running cg scan aws for SNS topics, the CLI seems to correctly follow the page tokens and make multiple queries.

However, it only returns the first page of results to Cloudgraph. Since the maximum page size from the AWS API is 100, it means it will only ever save 100 SNS topics to Cloudgraph/Dgraph.

The output I get always looks like:

ℹ Printing scan report...
┌─────────┬─────────────────┬────────┐
│ Service │ Resources Found │ Status │
├─────────┼─────────────────┼────────┤
│ sns     │ 100             │ ✔      │
├─────────┼─────────────────┼────────┤
│ total   │ 100             │ N/A    │
└─────────┴─────────────────┴────────┘

Despite my AWS account containing more than 100 topics.

Also, for some reason the command only works with CG_DEBUG=5 set. Without that flag, it always fails with:

✖ Unknown type "AddawsSnsInput".
✖ Unknown type "AddawsAccountInput".
✖ Unknown type "AddawsTagInput".

To Reproduce
Steps to reproduce the behavior:

1. Set "resources": "sns" in .cloud-graphrc.json.
2. Run CG_DEBUG=5 cg scan aws

Expected behavior
To return and save all SNS topics, even if the total count exceeds one page (i.e. 100 topics).

Environment (please complete the following information):

• CLI version (installed via NPM): @cloudgraph/cli/0.21.4 darwin-x64 node-v18.1.0
• Provider versions: [email protected].
• Context: Local machine. MacOS 12.3.1.

Additional context

I think this might be caused by this line:
https://github.com/cloudgraphdev/cloudgraph-provider-aws/blob/main/src/services/sns/data.ts#L77-L81

I changed this locally to resolve only if the nextToken is empty and that seemed to fix the issue (though it then fails later when fetching the subscriptions, so other changes may be needed):

if (nextToken) {
    listTopicArns(nextToken)
} else {
    resolve(topicArnList)
}

Thank you for filling out a bug report, we really appreciate any help in improving the CloudGraph CLI and providers!

Describe the bug

I have accounts that I access using roles that require MFA. This works fine for the aws cli, but CloudGraph doesn't use or prompt for the MFA and thus has access errors.

To Reproduce
Steps to reproduce the behavior:

1. Configure cg with a base profile, and a role arn to assume
2. Run cg scan
3. See error:
   ℹ Searching for AWS credentials...
   ✖ No valid credentials found for roleARN: arn:aws:iam::[DELETED]:role/sec_admin
   ℹ No AWS Credentials found for scan, please enter them manually
   ? Please input a valid accessKeyId [DELETED]
   ? Please input a valid secretAccessKey [DELETED]
   ✔ Found and using the following AWS credentials
   ✔ roleARN: arn:aws:iam::[DELETED]:role/sec_admin
   ✔ accessKeyId: AKIOER
   ✔ secretAccessKey: Q2S****************hlUdzIR
   ⠏ SCANNING data for aws⚠ There was a problem getting data for service ALB, CG encountered an error calling elbv2:describeLoadBalancers
   ⚠ User: arn:aws:iam::[DELETED]:user/ames.cornish is not authorized to perform: elasticloadbalancing:DescribeLoadBalancers with an explicit deny in an identity-based policy
   ✔ alb scan completed
   ⠹ SCANNING data for aws⚠ There was a problem getting data for service API Gateway Rest API, CG encountered an error calling apiGw:getDomainNames
   ⚠ User: arn:aws:iam::[DELETED]:user/ames.cornish is not authorized to perform: apigateway:GET on resource: arn:aws:apigateway:us-west-2::/domainnames with an explicit deny in an identity-based policy

ℹ AccessDenied: User: arn:aws:iam::[DELETED]:user/ames.cornish is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::626601142497:role/sec_admin
at Request.extractError (/home/ames/Projects/cloudgraph-provider-aws/node_modules/aws-sdk/lib/protocol/query.js:50:29)
at Request.callListeners (/home/ames/Projects/cloudgraph-provider-aws/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
at Request.emit (/home/ames/Projects/cloudgraph-provider-aws/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
at Request.emit (/home/ames/Projects/cloudgraph-provider-aws/node_modules/aws-sdk/lib/request.js:686:14)
at Request.transition (/home/ames/Projects/cloudgraph-provider-aws/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/home/ames/Projects/cloudgraph-provider-aws/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /home/ames/Projects/cloudgraph-provider-aws/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request. (/home/ames/Projects/cloudgraph-provider-aws/node_modules/aws-sdk/lib/request.js:38:9)
at Request. (/home/ames/Projects/cloudgraph-provider-aws/node_modules/aws-sdk/lib/request.js:688:12)
at Request.callListeners (/home/ames/Projects/cloudgraph-provider-aws/node_modules/aws-sdk/lib/sequential_executor.js:116:18)

Please include the cg-debug.log file if applicable

Expected behavior

scan to connect and download data.

Environment (please complete the following information):

• CLI version 0.21.3
• Provider versions 0.79.0-alpha.22
• Context local machine

Additional context

For the aws sdk cli, I can create a credentials profile that looks like:

[profile-name]
source_profile = base-profile
role_arn = arn:aws:iam::[DELETED]:role/sec_admin
mfa_serial = arn:aws:iam::[DELETED]:mfa/ames.cornish

When I use this profile with the aws sdk, it prompts me for MFA the first time I use it. The MFA is then cached, and I don't need to re-enter.

I tried to use this with cg scan two ways. First, using the profile-name with MFA cached. Second, using the base-profile and the desired role_arn. Both cases gave the same error -- credentials not found -- and didn't authenticate or prompt for MFA.

Is MFA supported for roles?

Currently when I attempt to use a profile via my ~/.aws/config file it does not work. We use SAML for our AWS roles so my files look like something below. The Credentials file is populated via https://github.com/Versent/saml2aws. I've attempted to select the saml profile while running cg init aws and then using the same roleArn defined in my config file(arn:aws:iam::<corp-account-ID>:role/corp-saml), but that ends up with the following error:

ℹ Searching for AWS credentials...
✖ No credentials found for roleARN: arn:aws:iam::<corp-account-ID>:role/corp-saml
ℹ No AWS Credentials found, please enter them manually

~/.aws/credentials

[saml]
aws_access_key_id        = <AccessKeyId>
aws_secret_access_key    = <SecretAccessKey>
aws_session_token        = <SessionToken>
aws_security_token       = <SecurityToken>
x_principal_arn          = arn:aws:sts::<IdentiyAccountID>:assumed-role/corp-saml/<emailaddress>
x_security_token_expires = 2021-12-07T05:19:13-08:00

~/.aws/config/

[profile corp]
source_profile = saml
role_arn = arn:aws:iam::<corp-account-ID>:role/corp-saml
region=us-west-1