GithubHelp home page GithubHelp logo

turbot / steampipe-mod-aws-tags Goto Github PK

View Code? Open in Web Editor NEW
15.0 12.0 8.0 3.66 MB

Is your AWS tagging strategy following best practice? This mod checks if your AWS resource tags are set correctly to help you manage them effectively using Powerpipe and Steampipe.

Home Page: https://hub.powerpipe.io/mods/turbot/aws_tags

License: Apache License 2.0

HCL 100.00%
steampipe-mod steampipe aws aws-tags tagging tags hacktoberfest powerpipe powerpipe-mod sql

steampipe-mod-aws-tags's Introduction

AWS Tags Mod for Powerpipe

Important

Powerpipe is now the preferred way to run this mod! Migrating from Steampipe →

All v0.x versions of this mod will work in both Steampipe and Powerpipe, but v1.0.0 onwards will be in Powerpipe format only.

An AWS tags checking tool that can be used to look for untagged resources, missing tags, resources with too many tags, and more.

Run checks in a dashboard:

image

Or in a terminal:

image

Documentation

Getting Started

Installation

Install Powerpipe (https://powerpipe.io/downloads), or use Brew:

brew install turbot/tap/powerpipe

This mod also requires Steampipe with the AWS plugin as the data source. Install Steampipe (https://steampipe.io/downloads), or use Brew:

brew install turbot/tap/steampipe
steampipe plugin install aws

Steampipe will automatically use your default AWS credentials. Optionally, you can setup multiple accounts or customize AWS credentials.

Finally, install the mod:

mkdir dashboards
cd dashboards
powerpipe mod init
powerpipe mod install github.com/turbot/steampipe-mod-aws-tags

Browsing Dashboards

Start Steampipe as the data source:

steampipe service start

Start the dashboard server:

powerpipe server

Browse and view your dashboards at http://localhost:9033.

Running Checks in Your Terminal

Instead of running benchmarks in a dashboard, you can also run them within your terminal with the powerpipe benchmark command:

List available benchmarks:

powerpipe benchmark list

Run a benchmark:

powerpipe benchmark run aws_tags.benchmark.untagged

Different output formats are also available, for more information please see Output Formats.

Configure Variables

Several benchmarks have input variables that can be configured to better match your environment and requirements. Each variable has a default defined in its source file, e.g., controls/limit.sp, but these can be overwritten in several ways:

It's easiest to setup your vars file, starting with the sample:

cp steampipe.spvars.example steampipe.spvars
vi steampipe.spvars

Alternatively you can pass variables on the command line:

powerpipe benchmark run aws_tags.benchmark.mandatory --var 'mandatory_tags=["Application", "Environment", "Department", "Owner"]'

Or through environment variables:

export PP_VAR_mandatory_tags='["Application", "Environment", "Department", "Owner"]'
powerpipe control run aws_tags.control.ec2_instance_mandatory

These are only some of the ways you can set variables. For a full list, please see Passing Input Variables.

Common and Tag Dimensions

The benchmark queries use common properties (like account_id, connection_name and region) and tags that are defined in the form of a default list of strings in the variables.sp file. These properties can be overwritten in several ways:

It's easiest to setup your vars file, starting with the sample:

cp steampipe.spvars.example steampipe.spvars
vi steampipe.spvars

Alternatively you can pass variables on the command line:

powerpipe benchmark run aws_tags.benchmark.limit --var 'tag_dimensions=["Environment", "Owner"]'

Or through environment variables:

export PP_VAR_common_dimensions='["account_id", "connection_name", "region"]'
export PP_VAR_tag_dimensions='["Environment", "Owner"]'
powerpipe benchmark run aws_tags.benchmark.limit

Remediation

Using the control output and the AWS CLI, you can remediate various tagging issues.

For instance, with the results of the ec2_instance_mandatory control, you can add missing tags with the AWS CLI:

#!/bin/bash

OLDIFS=$IFS
IFS='#'

INPUT=$(powerpipe control run aws_tags.control.ec2_instance_mandatory --var 'mandatory_tags=["Application"]' --output csv --header=false --separator '#' | grep 'alarm')
[ -z "$INPUT" ] && { echo "No instances in alarm, aborting"; exit 0; }

while read -r group_id title description control_id control_title control_description reason resource status account_id region
do
  aws resourcegroupstaggingapi tag-resources --region ${region} --resource-arn-list ${resource} --tags Application=MyApplication
done <<< "$INPUT"

IFS=$OLDIFS

To remove prohibited tags from EC2 instances:

#!/bin/bash

OLDIFS=$IFS
IFS='#'

INPUT=$(powerpipe control run aws_tags.control.ec2_instance_prohibited --var 'prohibited_tags=["Password"]' --output csv --header=false --separator '#' | grep 'alarm')
[ -z "$INPUT" ] && { echo "No instances in alarm, aborting"; exit 0; }

while read -r group_id title description control_id control_title control_description reason resource status account_id region
do
  aws resourcegroupstaggingapi untag-resources --region ${region} --resource-arn-list ${resource} --tag-keys Password
done <<< "$INPUT"

IFS=$OLDIFS

Open Source & Contributing

This repository is published under the Apache 2.0 license. Please see our code of conduct. We look forward to collaborating with you!

Steampipe and Powerpipe are products produced from this open source software, exclusively by Turbot HQ, Inc. They are distributed under our commercial terms. Others are allowed to make their own distribution of the software, but cannot use any of the Turbot trademarks, cloud services, etc. You can learn more in our Open Source FAQ.

Get Involved

Join #powerpipe on Slack →

Want to help but don't know where to start? Pick up one of the help wanted issues:

steampipe-mod-aws-tags's People

Contributors

brad-webb avatar cbruno10 avatar e-gineer avatar khushboo9024 avatar lalitlab avatar madhushreeray30 avatar misraved avatar priyanka-chatterjee-2000 avatar rinzool avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

steampipe-mod-aws-tags's Issues

Add a new benchmark to check tag values

Is your feature request related to a problem? Please describe.
❌ No

It is a feature that I need in my company, so I want to develop on this public repo 😄

Describe the solution you'd like
For some tags, we may want to restrict values.
For example, a tag Environment should only be Prod, Staging or Dev

In my company we use steampipe-mod-aws-tags to list resource with missing tags, but we also need to find resources with wrong tag (having consistent tags is useful to monitor costs!)

Update mod.sp to address deprecation warning

Describe the bug
The current version of the mod returns a warning due to using the deprecated option version instead of min_version for the aws plugin.

This issue can be trivially resolved by updating

  require {
    plugin "aws" {
      version = "0.81.0"
    }
  }

to

  require {
    plugin "aws" {
      min_version = "0.81.0"
    }
  }

Steampipe version (steampipe -v)
Example: v0.3.0

Plugin version (steampipe plugin list)
Example: v0.5.0

To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

AWS Organizations Tagging resources

Is your feature request related to a problem? Please describe.
Looking at the Steampipe docs I can’t see controls that allow for the querying of tags against Organizational resources, such as

  • AWS accounts
  • Organizational units
  • The organization’s root

These resources support tags are documented per AWS docs https://docs.aws.amazon.com/organizations/latest/userguide/orgs_tagging.html

Describe the solution you'd like
Once tables have been added with the tag column controls should be added to surface the additional tag data

Describe alternatives you've considered
None

Additional context
None

Add limit, mandatory, prohibited and untagged controls for supported AWS tables

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

*_expected_tag_values controls should evaluate resources without tags as 'skip'

Describe the bug
A clear and concise description of what the bug is.

Below is the example outcome, where S3 buckets are evaluated based on s3_bucket_expected_tag_values

steampipe check control.s3_bucket_expected_tag_values

The query evaluates non-tagged buckets as OK; by nature of this mod, we should validate whether buckets are tagged with proper values specified in the .spvar file or not; in case the tag is missing, the control should show as **skip**

image

Steampipe version (steampipe -v)
Example: v0.3.0

Plugin version (steampipe plugin list)
Example: v0.5.0

To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

Add default values to all variables to better support `mod get` functionality

When getting a mod with variables through mod get, if there are no defaults defined and no values defined locally, any steampipe command will then prompt the user for those variables. We should add defaults and provide instructions with a sample Steampipe variable file for a smoother mod get experience and local experience as well.

Add additional dimensions to queries

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

"Tagging resources" controls feel too heavy / redundant with other controls

For my mandatory tags run, I have 8,910 alarms of which 6.962 are from "Tagging resources". This feels wrong.

Should tagging resources be broken up into per-service information? Or is service information redundant relative to this control?

I believe the same problem exists for limit, mandatory, etc:

/src/steampipe-mod-aws-tags $ grep -r tagging_resource .
./CHANGELOG.md:- tagging_resource
./controls/mandatory.sp:    control.tagging_resource_mandatory,
./controls/mandatory.sp:control "tagging_resource_mandatory" {
./controls/mandatory.sp:  sql         = replace(local.mandatory_sql_region, "__TABLE_NAME__", "aws_tagging_resource")
./controls/untagged.sp:    control.tagging_resource_untagged,
./controls/untagged.sp:control "tagging_resource_untagged" {
./controls/untagged.sp:  sql         = replace(local.untagged_sql_region, "__TABLE_NAME__", "aws_tagging_resource")
./controls/prohibited.sp:    control.tagging_resource_prohibited,
./controls/prohibited.sp:control "tagging_resource_prohibited" {
./controls/prohibited.sp:  sql         = replace(local.prohibited_sql_region, "__TABLE_NAME__", "aws_tagging_resource")
./controls/limit.sp:    control.tagging_resource_tag_limit,
./controls/limit.sp:control "tagging_resource_tag_limit" {
./controls/limit.sp:  sql         = replace(local.limit_sql_region, "__TABLE_NAME__", "aws_tagging_resource")

cloud steampipe io_org_acme_workspace_awsmulti_dashboard_aws_tags benchmark mandatory

running control.s3_bucket_untagged against multiple aws accounts

I have two AWS accounts, call them jon-personal and jon-public. I can query them both by their names in my aws.spc. For the mod, though, it seems I need to reduce the aws.spc to a single uncommented entry in order to run control.s3_bucket_untagged, is that expected?

Null check insufficient to find all untagged AWS resources

Describe the bug
Resources that have no tags do not always pass the 'is null' test.

Steampipe version (steampipe -v)
Steampipe v0.20.9

Plugin version (steampipe plugin list)
hub.steampipe.io/plugins/turbot/aws@latest 0.113.0 aws

To reproduce

select arn from aws_dynamodb_table where tags is null
+-----+
| arn |
+-----+
+-----+
select arn from aws_dynamodb_table where (tags is null or not exists (select 1 from jsonb_object_keys(tags)))
+------------------------------------------------------------------+
| arn |
+------------------------------------------------------------------+
| arn:aws:dynamodb:ap-southeast-2:account-id:table/my-lock-table |
+------------------------------------------------------------------+
select arn, tags from aws_dynamodb_table where arn='arn:aws:dynamodb:ap-southeast-2:account-id:table/my-lock-table'
+------------------------------------------------------------------+------+
| arn | tags |
+------------------------------------------------------------------+------+
| arn:aws:dynamodb:ap-southeast-2:account-id:table/my-lock-table | {} |
+------------------------------------------------------------------+------+

Expected behavior
Empty tags objects that for whatever reason do not pass the 'is null' test, should still be counted as empty and the resource they are associated with marked as 'untagged'.

Additional context
I do not know why certain empty tag objects are failing the 'is null' check while others pass.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.