jckuester / awsrm

A remove command for AWS resources

awsrm

A remove command for AWS resources

This command line tool follows the Unix Philosophy of doing only one thing and doing it well:

It simplifies deleting over 250 AWS resource types across multiple accounts and regions.

Like other Unix-like tools, awsrm reveals its full power when combining it via pipes with other tools, such as awsls for listing AWS resources and grep for filtering by resource attributes.

Example

Delete resources by tags (or other attributes)

To delete, for example, all EC2 instances with tag Name=foo, run

  awsls instance -a tags | grep Name=foo | awsrm

To filter on multiple attributes, display them with awsls via the -a (--attributes) <comma-separated list> flag. Every attribute in the Terraform documentation (here are the attributes for aws_instance) can be used:

1. List resources via awsls with the attributes you want to filter on (here: -a tags)
2. Use standard tools like grep to filter resources
3. Pipe result to awsrm (nothing is deleted until you confirm)

Note: awsls output passes on profile and region information, so that awsrm knows for each resource in what account and region to delete it.

Depending on the type of resource, deletion can take some time. This GIF runs faster than EC2 instances are actually terminated; the shell prompt shows the real execution times in seconds.

Delete across multiple accounts and regions

List all instances with awsls in the AWS accounts associated with profile myaccount1 and myaccount2 in both regions us-west-2 and us-east-1 and pipe the result to awsrm:

awsls -p myaccount1,myaccount2 -r us-west-2,us-east-1 instance | awsrm

Delete by IDs

Delete specific resources by ID, for example, some IAM roles

awsrm iam_role db-cluster elb nginx

or VPCs

awsrm vpc vpc-1234 vpc-3456 vpc-7689

1. List resources via awsls to find out what resources to delete.
2. Use awsrm to delete the resources by resource type and ID(s)

Usage

Input via arguments:

awsrm [flags] <resource_type> <id> [<id>...]

Input via pipe:

awsls [flags] <resource_type> | awsrm

or

echo "<resource_type> <id> <profile> <region>" | awsrm

To see options available run awsrm --help.

Installation

Binary Releases

You can download a specific version on the releases page or use the following way to install to ./bin/:

curl -sSfL https://raw.githubusercontent.com/jckuester/awsrm/master/install.sh | sh -s v0.2.0

Homebrew

Homebrew users can install by:

brew install jckuester/tap/awsrm

For more information on Homebrew taps please see the tap documentation.

Supported resources

This tool can not only delete any resource type that is supported by awsls, but any resource type covered by the Terraform AWS Provider.

Note: the prefix aws_ for resource types is optional. This means, for example, awsrm aws_instance <id> and awsrm instance <id> are both valid commands.

Disclaimer

You are using this tool at your own risk! I will not take responsibility if you delete any critical resources in your production environments.

awsrm's Issues

Add a way to view teraform debug logs

Terraform has a lot of useful debug logs, including logging the AWS API requests it makes.
With the usual terraform CLI it is possible to set TF_LOG env variable to get verbose debug logs info.

I propose awsrm to mimic terraform in this way and use TF_LOG env variable for viewing terraform and AWS SDK logs, or use some other kind of switch if it makes sense.

As of now having TF_LOG env var set doesn't influence awsrm behavior.
It only exposes --debug CLI flag which provides pretty terse logs of awsrm wrapper itself, which is not enough

Timeout when deleting s3 buckets

When I run:
awsls "s3*" -p my_profile -r us-east-1 | awsrm --force --debug
output:
• failed to delete resource error=destroy timed out (1m0s)

Can I set the time limit from the cli?

Attempting to delete multiple resources results in "failed to decode instruction" error

Hello,

Thanks for making awsls and awsrm, they're very useful.

I have an issue when trying to delete multiple cloudformation stacks. When I try to delete multiple stacks from a list of profiles/accounts, awsrm fails with an error. However, I can successfully delete each stack if I run the command against a single profile/account. Any idea why this happens?

$ awsls --version

version: 0.11.0
commit: 7612fb3
built at: 2021-06-13T18:19:16Z
using: go1.16.3

$ awsrm --version

version: 0.2.0
commit: 8d547d9
built at: 2021-06-14T07:04:08Z
using: go1.15.13

$ awsls -p master-organizationaccountaccessrole,identity-organizationaccountaccessrole,backup-organizationaccountaccessrole,sharedservices-organizationaccountaccessrole,sandbox-organizationaccountaccessrole,prod-organizationaccountaccessrole "aws_cloudformation_stack" -r us-west-2 | grep guard
aws_cloudformation_stack   arn:aws:cloudformation:us-west-2:111111111111:stack/org-guardduty/81da7e60-dfca-11eb-8b77-06ed2dedafa1                                        sharedservices-organizationaccountaccessrole   us-west-2   2021-07-08 08:57:22   
aws_cloudformation_stack   arn:aws:cloudformation:us-west-2:222222222222:stack/org-guardduty/81c548b0-dfca-11eb-be43-066f7c61cd37                                        backup-organizationaccountaccessrole           us-west-2   2021-07-08 08:57:21   
aws_cloudformation_stack   arn:aws:cloudformation:us-west-2:333333333333:stack/org-guardduty/81dc0500-dfca-11eb-a07f-06b8c52235b5                                        prod-organizationaccountaccessrole             us-west-2   2021-07-08 08:57:22   
aws_cloudformation_stack   arn:aws:cloudformation:us-west-2:444444444444:stack/org-guardduty/81519f50-dfca-11eb-b0e8-06f3e52c1a19                                        master-organizationaccountaccessrole           us-west-2   2021-07-08 08:57:21   
aws_cloudformation_stack   arn:aws:cloudformation:us-west-2:555555555555:stack/org-guardduty/81d99400-dfca-11eb-92c0-0abfe5dcf375                                        identity-organizationaccountaccessrole         us-west-2   2021-07-08 08:57:22   
aws_cloudformation_stack   arn:aws:cloudformation:us-west-2:666666666666:stack/org-guardduty/81d09350-dfca-11eb-a0e3-025e58c7cc1d                                        sandbox-organizationaccountaccessrole          us-west-2   2021-07-08 08:57:21   

$ awsls -p master-organizationaccountaccessrole,identity-organizationaccountaccessrole,backup-organizationaccountaccessrole,sharedservices-organizationaccountaccessrole,sandbox-organizationaccountaccessrole,prod-organizationaccountaccessrole "aws_cloudformation_stack" -r us-west-2 | grep guard | awsrm --debug

   • input via pipe           
   • found already installed Terraform provider name=aws path=/Users/hurley/.awsrm/terraform-provider-aws_v3.42.0_x5 version=3.42.0
   • start launching new instance of Terraform AWS Provider profile=sandbox-organizationaccountaccessrole region=us-west-2
   • start launching new instance of Terraform AWS Provider profile=prod-organizationaccountaccessrole region=us-west-2
   • start launching new instance of Terraform AWS Provider profile=identity-organizationaccountaccessrole region=us-west-2
   • start launching new instance of Terraform AWS Provider profile=backup-organizationaccountaccessrole region=us-west-2
   • start launching new instance of Terraform AWS Provider profile=master-organizationaccountaccessrole region=us-west-2
   • start launching new instance of Terraform AWS Provider profile=sharedservices-organizationaccountaccessrole region=us-west-2
assertion failed [inst.has_value()]: failed to decode instruction: 0x0
(StateRecovery.cpp:336 determine_state_recovery_action_forward_branches)
Trace/BPT trap: 5

Introduce --force flag

I suggest adding a --force flag to the CLI so that when awsrm is run in automation no confirmation is required from the calling process.

Btw, if the calling process tries to work around this by setting a pipe and writing Yes to it, this doesn't work because awsrm expects the resources passed in a special format when it is invoked via a pipe

Timeout when deleting VPCs

When attempting to delete a single VPC the resource is not deleted due to a 1 minute timeout:

awsls aws_vpc | grep 'vpc-0e9361240ff2d80e9' | awsrm 

   • SHOWING RESOURCES THAT WOULD BE DELETED (DRY RUN)
      • aws_vpc                                  id=vpc-0e9361240ff2d80e9 profile=saml-core-govcloud region=us-gov-west-1
   • TOTAL NUMBER OF RESOURCES THAT WOULD BE DELETED: 1
      • Are you sure you want to delete these resources (cannot be undone)? Only YES will be accepted.
        Enter a value: YES
   • STARTING TO DELETE RESOURCES
      • will retry to delete resource                      resource_id=vpc-0e9361240ff2d80e9 type=aws_vpc
   • FAILED TO DELETE THE FOLLOWING RESOURCES (RETRIES EXCEEDED): 1
      • aws_vpc                                            error=destroy timed out (1m0s) id=vpc-0e9361240ff2d80e9
   • TOTAL NUMBER OF DELETED RESOURCES: 0

This command is being executed against us-gov-west-1. Any assistance would be greatly appreciated!

This version of Terraform has an outdated GPG key

Issue:

I followed the instructions as per the README on how to install the project's binary file, proceeding to place it on my $PATH, returning this following error afterward:

Full logs:

sh awsls vpc -a tags  | grep Name=vpc | awsrm                                                 

failed to install provider (aws): 2 problems:
- registry.terraform.io: This version of Terraform has an outdated GPG key and is unable to verify new provider releases. Please upgrade Terraform to at least 0.12.31 to receive new provider updates. For details see: https://discuss.hashicorp.com/t/hcsec-2021-12-codecov-security-event-and-hashicorp-gpg-key-exposure/23512
- open ~/.awsrm/terraform-provider-aws_v3.16.0_x5: text file busy
Error: failed to launch provider (): fork/exec : no such file or directory

This bug seems to refer to this thread on Hashicorp's forums

Something curious to be noted is that my machine's Terraform is at v15.x already. So it might have something to do with the project dependencies.

All the best.

Is there a way to run this cmd line utility in a forced fashion for pipeline deployments?

Hi,

Is there a way to make the cmd utility not behave in a Dry Run mode by default? I want to add it as part of the CI/CD automation process but I wanted to know if there was an ability to force delete without a prompt for the Dry Run.

Using `awsls ... | awsrm` pipe spawns terraform provider process for each resource and crashes

Here is what I see when I use a pipe:

Output logs

$ awsls aws_lambda_function | awsrm --debug

   • input via pipe           
   • found already installed Terraform provider name=aws path=/home/veetaha/.awsrm/terraform-provider-aws_v3.42.0_x5 version=3.42.0
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2
   • start launching new instance of Terraform AWS Provider profile=N/A region=us-east-2

Error: failed to configure provider (name=aws, version=3.42.0): error configuring Terraform AWS Provider: no valid credential sources for Terraform AWS Provider found.

Please see https://registry.terraform.io/providers/hashicorp/aws
for more information about providing credentials.

Error: NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors

This also leaks all the terraform provider processes, so they are left hanging and eating my PC RAM:

However, if I invoke the same command, but with all the resource identifiers as CLI parameters, everything works fine, only one terraform provider process is created and no NoCredentialProviders error is triggered, and no processes are leaked