turbot / steampipe-plugin-turbot Goto Github PK
View Code? Open in Web Editor NEWUse SQL to instantly query the Turbot CMDB. Open source CLI. No DB required.
Home Page: https://hub.steampipe.io/plugins/turbot/turbot
License: Apache License 2.0
Use SQL to instantly query the Turbot CMDB. Open source CLI. No DB required.
Home Page: https://hub.steampipe.io/plugins/turbot/turbot
License: Apache License 2.0
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Describe the bug
The filter
column fails to populate with an unusual error.
Error: 2 connections failed:
connection 'taurus': rpc error: code = Unknown desc = failed to populate column 'filter': FromQual transform can only be called if there is a singe equals qual for the given column
connection 'canis': rpc error: code = Unknown desc = failed to populate column 'filter': FromQual transform can only be called if there is a singe equals qual for the given column (SQLSTATE HV000)
Steampipe version
❯ steampipe --version
steampipe version 0.11.0
Plugin version (steampipe plugin list
)
+--------------------------------------------------+---------+----------------------------------------------+
| Name | Version | Connections |
+--------------------------------------------------+---------+----------------------------------------------+
| hub.steampipe.io/plugins/turbot/aws@latest | 0.43.0 | aac,aaa,aws,aab,all_sandbox |
| hub.steampipe.io/plugins/turbot/azure@latest | 0.22.0 | azure |
| hub.steampipe.io/plugins/turbot/csv@latest | 0.1.0 | csv |
| hub.steampipe.io/plugins/turbot/gcp@latest | 0.19.0 | demo |
| hub.steampipe.io/plugins/turbot/github@latest | 0.10.0 | github |
| hub.steampipe.io/plugins/turbot/slack@latest | 0.3.0 | slack |
| hub.steampipe.io/plugins/turbot/steampipe@latest | 0.2.0 | steampipe |
| hub.steampipe.io/plugins/turbot/turbot@latest | 0.2.0 | corvus,taurus,astro,canis |
+--------------------------------------------------+---------+----------------------------------------------+
To reproduce
with workspace as (
select workspace
from turbot_resource
where filter = 'resourceId:"tmod:@turbot/turbot#/" level:self'
),
webhook_rotation_policy as (
select value,
workspace,
filter
from turbot_policy_setting
where filter = 'policyTypeId:"tmod:@turbot/turbot#/policy/types/webhookSecretRotation"'
)
select w.workspace as resource,
e.filter,
case
when e.value is null then 'Skip'
else e.value
end as webhook_rotation_setting,
case
when e.value like 'Enforce: Rotate webhook secret' then 'ok'
else 'alarm'
end as status,
case
when e.value like 'Enforce: Rotate webhook secret'
then 'Webhook Secrets in ' || w.workspace || ' are set to rotate.'
else 'Webhook Secrets in ' || w.workspace || ' do not rotate.'
end as status
from workspace w
left join webhook_rotation_policy e using (workspace)
webhook_rotation_policy
query alone and it works just fine. The workspace
query returns properly too. My hunch is that filter error comes from joining the two queries.Expected behavior
I get a list of workspaces with a column indicating whether the Webhook rotation policy has been set to Enforce
or not.
Additional context
Working on a Turbot Workspace health mod. This query is a part of that effort.
Describe the bug
When querying {resource{data}}
, Turbot will always return the most recent resource data state. When making queries in turbot_resource
, this is appropriate behavior. However, for notifications, we want the previous resource state. Using {resource{object}}
is the correct query to make.
Steampipe version (steampipe -v
)
❯ steampipe -v
steampipe version 0.14.6
Plugin version (steampipe plugin list
)
| hub.steampipe.io/plugins/turbot/turbot@latest | 0.5.0 |
To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).
resource_created
and resource_updated
notifications259791392905645
in the below query with the resource ID from step 1.select id, process_id, notification_type, create_timestamp, resource_title, resource_new_version_id, resource_old_version_id, resource_type_id, resource_type_uri, resource_data
from turbot_notification
where filter = 'notificationType:resource resourceId:"259791392905645"'
Expected behavior
The results from the query in step 2, should reflect the resource's change over time.
Additional context
The query needs to change on line 164, line 309 and line 66
Description
Sometimes, we want to get a count of a set of controls in Turbot without having to pull over every control to do the count Steampipe-side. Turbot's resources GraphQL API queries have a metadata.stats.total
section that includes count
data. This ticket requests the creation of a turbot_control_count table that exclusively returns the metadata.stats.total
.
controls(filter: $filter, paging: $paging) {
metadata {
stats {
total
}
}
References
Describe the bug
I'm trying to build a report of permission grants in Turbot. I'm having a hard time identifying the difference between AWS/Owner
and Turbot/Owner
. I see we have a level_title
column which provies [Owner, Admin, Metadata, etc]
but no column that says [AWS, Azure, GCP, Turbot]
.
Steampipe version (steampipe -v
)
0.12.2
Plugin version (steampipe plugin list
)
0.4.0
To reproduce
select *
from turbot_active_grant
where identity_profile_id like '%bob%'
Run this against the Turbot Demo environment.
Expected behavior
A column that indicates whether this is grant is a AWS/*
, Azure/*
, GCP/*
or Turbot/*
. An additional column that shows the grant as it shows in the Turbot console would be nice (AWS/Owner
), though not strictly necessary.
Additional context
Add any other context about the problem here.
Is your feature request related to a problem? Please describe.
Verifying that Installed Mods are up-to-date is an important part of verifying workspace health. A regular turbot_resource
query will get the installed mods. Discovery of the current Recommended
version isn't currently possible through Steampipe.
Describe the solution you'd like
A new table for available mod version, perhaps called turbot_mod_versions
. Should be able to search by modName, OrgName, free text search and status.
Describe alternatives you've considered
There is no alternative, other than to go look in the Turbot console for this information.
Additional context
The table should represent the same descriptive power as this query (pulled from the Turbot console).
query modVersionSearchByName {
modVersionSearches(search: "", modName: "aws", orgName: "turbot", status: AVAILABLE) {
items {
identityName
name
versions {
status
version
}
}
}
}
References
Add any related links that will help us understand the resource, including vendor documentation, related GitHub issues, and Go SDK documentation.
I would like to audit who has active grant permissions in my turbot workspace.
Is your feature request related to a problem? Please describe.
If we need a count of controls or notifications, without the metadata column, we have to ship over a large number of rows to Steampipe in order to get a count. Doing a count of rows makes much more sense in the Turbot DB.
Describe the solution you'd like
Introduction of the metadata
column to the turbot_control
and turbot_notification
tables.
Describe alternatives you've considered
None. I'm doing what Cody asked me to do 😁
Additional context
Useful when aggregating data across multiple workspaces.
Recompipe the plugin with steampipe-plugin-sdk v1.8.2 and Go Version 1.17
Describe the bug
queries to the table turbot_mod_version
fail with AccessDenied exception from GovCloud
Steampipe version (steampipe -v
)
Example: v0.16.0
Plugin version (steampipe plugin list
)
Example: v0.8.0
To reproduce
configure a connection to a workspace running in AWS Gov Cloud
select * from turbot_mod_version;
Expected behavior
Query returns list of installed mod versions for this workspace
Use Case
I would like to write a Turbot Workspace Health mod. Determining the number of policy values in tbd
, invalid
, and error
, and their age is very important to overall workspace health. Please include a metadata
and workspace
columns in this table.
Description
Sometimes, we want to get a count of a set of resources in Turbot without having to pull over every resource to do the count steampipe-side. Turbot's resources GraphQL API queries have a metadata.stats.total
section that includes count
data. However, a column for metadata
counts in the normal turbot_resource table has been deemed inapproriate (and I agree). This ticket requests the creation of a turbot_resource_count table that exclusively returns the metadata.stats.total
.
resources(filter: $filter, paging: $paging) {
metadata {
stats {
total
}
}
References
Resource Counts in the turbot_notification table
If this implementation is successful, additional tables for turbot_control_count
, and turbot_notification_count
will be requested.
Is your feature request related to a problem? Please describe.
The existing turbot_*
tables are insufficient to cover the breadth of the Turbot API. Further, queries like resource_control_policy.graphql are easy to do in raw GraphQL but painful to do in Steampipe.
There is also considerable difficulty for users new to Turbot and to Steampipe with grokking where the filtering happens, whether Turbot-side or Steampipe-side. This difficulty can lead to long running queries when a user specifies Steampipe-side filtering then unintentionally pulls over hundreds of thousands of rows from Turbot. I believe/hope that if the user specifies the GraphQL themselves, it will be clear to them where the filtering/joining is happening.
Describe the solution you'd like
Specify a path to a file or directory on the local file system that contains GraphQL files. On launch, Steampipe will parse these query files then autogenerate tables and columns to match these queries. The overal operation would be similar to how dynamic tables are created for the CSV
, Terraform
and GoogleSheets
plugins.
Describe alternatives you've considered
There aren't any really, other than to write the GraphQL queries into a general purpose programming language.
Additional context
The Turbot GraphQL API is incredibly rich. The current approach to statically defined tables removes some of that richness and imposes additional development load on Turbot plugin developers to implement each new table.
Describe the bug
The column name has a typo.
It is poliy_type_trunk_title
It should be policy_type_trunk_title
Plugin version (steampipe plugin list
)
latest
https://hub.steampipe.io/plugins/turbot/turbot/tables/turbot_policy_value#inspect
References
Add any related links that will help us understand the resource, including vendor documentation, related GitHub issues, and Go SDK documentation.
Is your feature request related to a problem? Please describe.
It is difficult to troubleshoot or understand what exactly Steampipe is requesting from Turbot. A user must translate in their head from SQL to GraphQL. For new users this can be very difficult. Dumping the query to logs would give new users a way to link their steampipe query to Turbot GraphQL. (Basically, this would be like the "Developers" tab in the Turbot console.)
Describe the solution you'd like
When STEAMPIPE_LOG=TRACE
, output the Turbot GraphQL query and variables to the Steampipe logs.
Describe alternatives you've considered
Dig through the API logs in Turbot Master.
Additional context
This is a further enhancement to make it easier for users to tell the difference between Turbot-side and Steampipe-side filtering.
Describe the bug
A clear and concise description of what the bug is.
Steampipe version (steampipe -v
)
Example: v0.3.0
Plugin version (steampipe plugin list
)
Example: v0.5.0
To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).
Expected behavior
A clear and concise description of what you expected to happen.
Additional context
Add any other context about the problem here.
Is your feature request related to a problem? Please describe.
When querying turbot notifications across multiple workspaces, it's difficult to keep track of where the notifications came from without the workspace
column. Related to: #3
Describe the solution you'd like
Inclusion of workspace
column in the turbot_notification table.
Describe alternatives you've considered
There aren't any really.
Additional context
None.
Describe the bug
no values are returned for filter column if it is not passed as a qual
select
id, filter
from
turbot_demo.turbot_resource
where
resource_type_uri = 'tmod:@turbot/aws-iam#/resource/types/accessKey'
and
filter = '$.turbot.custom.createTimestamp:<=T-30d'
returns
+-----------------+-----------------------------------------+
| id | filter |
+-----------------+-----------------------------------------+
| 209793964907564 | $.turbot.custom.createTimestamp:<=T-30d |
| 216255846230771 | $.turbot.custom.createTimestamp:<=T-30d |
| 245280856563960 | $.turbot.custom.createTimestamp:<=T-30d |
| 215084373625757 | $.turbot.custom.createTimestamp:<=T-30d |
| 203192415818086 | $.turbot.custom.createTimestamp:<=T-30d |
| 241433740146509 | $.turbot.custom.createTimestamp:<=T-30d |
| 241434343667200 | $.turbot.custom.createTimestamp:<=T-30d |
| 224758800125111 | $.turbot.custom.createTimestamp:<=T-30d |
| 241434545526718 | $.turbot.custom.createTimestamp:<=T-30d |
| 245281067298667 | $.turbot.custom.createTimestamp:<=T-30d |
+-----------------+-----------------------------------------+
but
select
id, filter
from
turbot_demo.turbot_resource
where
resource_type_uri = 'tmod:@turbot/aws-iam#/resource/types/accessKey'
returns
+-----------------+--------+
| id | filter |
+-----------------+--------+
| 209793964907564 | <null> |
| 224758800125111 | <null> |
| 216255846230771 | <null> |
| 241434343667200 | <null> |
| 215084373625757 | <null> |
| 245280856563960 | <null> |
| 203192415818086 | <null> |
| 241434545526718 | <null> |
| 241433740146509 | <null> |
| 245281067298667 | <null> |
+-----------------+--------+
Steampipe version (steampipe -v
)
v0.13.0
Plugin version (steampipe plugin list
)
v0.4.0
To reproduce
run queries abovve
Expected behavior
filter column should. be populated
Additional context
n/a
Describe the bug
In the Turbot Plugin, turbot_tag table. To confirm my understanding of it:
If all is true, how does the turbot_tags table return null / '[]' for the resource_id column? Wouldnt every tag have at least 1 resource id associated?
Steampipe version (steampipe -v
)
v0.19.5
Plugin version (steampipe plugin list
)
Turbot v0.10.0
To reproduce
Example, if you run this query in Steampipe:
select
*
from
turbot_tag
order by
resource_ids,
key,
value;
You will see examples of resource Ids column with []
. However I would expect at least one resource ID. Right?
When you look up any of the Tags with []
in Turbot specifically, you see there are active resources associated. A tag like "Bucket Name: bob-demo-4-12-2023" is active, in the CMDB, and in AWS with that tag. But the result for it is '[]'.
In Turbot a search like: tags:'Bucket Name'='bob-demo-4-12-2023'
returns one bucket. But in Steampipe its []
Expected behavior
All tags have at least 1 resourceId associated.
References
Turbot docs on Notifications: https://turbot.com/v5/docs/concepts/notifications
API Refernce for Notifications: https://turbot.com/v5/docs/reference/graphql/query/notifications
Notification Definition: https://turbot.com/v5/docs/reference/graphql/object/Notification
To get Activity information out of Turbot, you need access to the Notifications table.
Describe the bug
A clear and concise description of what the bug is.
I am trying to query turbot_active_grant with limit of 10 records in my turbot v5 environment which has 20+k records . i am getting out of memory error . I have 4GB Mem. I tried using latest Steampipe v0.16.0-rc.8 version . but no luck.
Steampipe version (steampipe -v
) v0.16.0-rc.8
Example: v0.3.0
Plugin version (steampipe plugin list
) 0.5.0
Example: v0.5.0
To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).
run below query in amazon linux 2
select grant_id,
resource_id,
identity_profile_id,
identity_display_name,
identity_email,
identity_status,
level_uri,
resource_type_uri,
workspace,
create_timestamp
from nonprod.turbot_active_grant
where resource_trunk_title = 'turbot'
and level_trunk_title = 'superuser'
limit 10;
Expected behavior
A clear and concise description of what you expected to happen.
retrun limited rows in few seconds
Additional context
Add any other context about the problem here.
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
References
We have tables for Turbot Controls and Turbot Notifications. Steampipe's values as diagnostic tool would be greatly enhanced with the addition of a table to grab process logs also. This may also require enhancements on the turbot_control
table to include "last_process_id" information to look up process logs.
Use Case
A customer reports a problem with a given control. We can hand them a steampipe query that will take a control_id. The query then returns any resource updates along with the control changes and the debug logs for each control change. This way we get a single diagnostic package instead of piecemeal "send me this, send me something else, send me another thing....etc."
The intent is to build a query to dump the information related to https://{workspace}/apollo/controls/{control_id}/control
as well as https://{workspace}/apollo/processes/{process_id}/detail
and https://{workspace}/apollo/processes/{process_id}/logs?filter=logLevel%3A%3E%3Ddebug
and https://{workspace}/apollo/processes/{process_id}/notifications (We already have the turbot_notification
table so this is covered)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.