brainly / terraform-provider-redshift Goto Github PK

Any plans on adding support for configuring a socks proxy via the provider similar to this MySQL provider?
https://registry.terraform.io/providers/petoju/mysql/latest/docs#proxy
That way you can use the proxy just for this provider and not others in the same TF project.

Grant "all" is a supported method of granting all access to a user or group for tables and schemas. But the current implementation of grant does not support that.

Can this is be a supported option?

SVL_USER_INFO is not available inside Redshift Serverless. Could we use pg_user for this purpose?

Error reading User: pq: permission denied for relation svl_user_info

I get the following error when try to create redshift grant for external schema:

│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to redshift_grant.user, provider "provider[\"registry.terraform.io/brainly/redshift\"]" produced an unexpected new value: Root resource was present, but
│ now absent.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Terraform plan seems like following:

  # redshift_grant.service_accounts["id"] will be created
  + resource "redshift_grant" "service_accounts" {
      + id          = (known after apply)
      + object_type = "schema"
      + privileges  = [
          + "usage",
        ]
      + schema      = "<schema_name>"
      + user        = "<user_name>"
    }

At the same time grant for regular schema apply works properly.
Also if I run following SQL query, grant applies without any error:

GRANT USAGE ON SCHEMA <external_schema_name> TO <user>;

For users who do not want to manage their passwords via the terraform module, but DO want to manage super users, the check I (dumbly) requested for super user passwords prevents that -- superusers only need to have their password set at the time that they are created on redshift, not when they are imported.

Currently I get the following when trying to upgrade from 0.5.0 to 0.6.0:

│ Error: Users that are superusers must define a password.
│ 
│   with module.main.redshift_user.users["foo_super"],
│   on ../modules/redshift/main.tf line 40, in resource "redshift_user" "users":
│  40: resource "redshift_user" "users" {
│ 
│ Error: Users that are superusers must define a password.
│ 
│   with module.main.redshift_user.users["bar_super"],
│   on ../modules/redshift/main.tf line 40, in resource "redshift_user" "users":
│  40: resource "redshift_user" "users" {
│ 

The check to make sure a super user resource has a password shoudl only occur if the user doesn't exist in redshift. Anyway, I think my original feature request was misguided and pointless, apologies!

Hello

Using Redshift Spectrum, we'd like all users to be able to parse json so we need to enable the following session settings to all users:

ALTER USER XXXX SET json_serialization_enable TO true;
ALTER USER XXXX SET json_serialization_parse_nested_strings TO true;
ALTER USER XXXX SET enable_case_sensitive_identifier TO true;

Right now we do it manually but we would like to handle it with your module but it seems like it cannot be handle with the resource redshift_user.
Is there any workaround ? If not, is anyone able to develop this feature please ? 🙏

Thanks !
Vincent

provider "redshift" {
host = var.redshift_host
username = var.redshift_user
password = var.redshift_password
}

The above logic works only when the redshift host/endpoint is publicly reachable, which will never (mostly) be the case in prod deployments.
When the TF cloud is trying to make a call, it is not able to reach the Redshift host it is a private VPC. How we are planning to address this?

Thanks
gurmukh

Hi.
Actually we are trying to move all the queries that we have in Redshift related to internal configuration of the database (users, groups, schemas and permissions).
Until now we have got no problem with users, groups and schema resources but now as we are trying to move the permissions we tried to use imports as the same way we made with the other resources.

The module id is different as the others ones as it is not using an internal id from redshift but the provider generates it and we tried to generate the id for existing grant permissions as the same way. When we do it we faced an error that the provider does not support import for grant resources.

Error: resource redshift_grant doesn't support import

Could be possible to add the import function for this resources?

Attempting to create a user as superuser without setting a password gives this error from redshift at apply-time.

error creating user mysuperuser: pq: CREATEUSER and PASSWORD DISABLE are incompatible

It would be nicer to have an error on the plan phase.

It seems to be that it doesn't support role?

Hello

We need to give rights ("select" or more) on multiple schemas.
For now it looks like a lot like

resource "redshift_grant" "readonly_schema1" {
  group       = redshift_group.readonlygroup.name
  schema      = "schema1"
  object_type = "table"
  privileges  = ["select"]
}

resource "redshift_grant" "readonly_schema2" {
  group       = redshift_group.readonlygroup.name
  schema      = "schema2"
  object_type = "table"
  privileges  = ["select"]
}

It a real mess between since we need A LOT of redshift_grant due to our huge amount of groups x schema.
Would it be possible to declare a list of schema instead of a single schema ?
It would look like as follow.

resource "redshift_grant" "readonly" {
  group       = redshift_group.readonlygroup.name
  schemas      = ["schema1", "schema2"]
  object_type = "table"
  privileges  = ["select"]
}

Don't you have yourself the same problem ?

Vincent

the language object is also in an object the redshift_grant resource?

i would like to grant the usage of language to a group like below

grant usage on language sql to group udf_devs;

https://docs.aws.amazon.com/redshift/latest/dg/udf-security-and-privileges.html

Creating an external_from_mysql to Aurora Serverless v2 throws an error "Unsupported source database type unknown" however the schema is created successfully (but terraform breaks)

Hi!

AWS redshift now implements a clause WITH PERMISSIONS that allows to manage datashare object at object level. Is it possible to add that feature in the provider please ?

Regards

The case of a username enclosed in double quotation marks is always preserved regardless of the setting of the enable_case_sensitive_identifier configuration option.

https://docs.aws.amazon.com/redshift/latest/dg/r_enable_case_sensitive_identifier.html

The provider always encloses usernames but saves the value to the state as lowercase.
This must be some recent change since tests that have been in the provider from the beginning are now failing

I'm not sure if I'm trying to do something that it's not supported by the module.

In our organization, we manage multiple AWS accounts. For these tests, we have the following scenario:

• Terraform is run with the user's credentials from account A (id: 58xxxxxx), while the redshift cluster was created in account B (82xxxxxx).
• The user account has permission to assume any role, and we have one role on the (redshift) account that provides admin access to every AWS resource. This method works with no problems for creating the Redshift cluster or any other AWS resource.

When I try to use this module, I'm getting the following error when doing terraform plan:

Error: operation error Redshift: GetClusterCredentials, https response error StatusCode: 403, RequestID: c4d960a3-47d2-4f8d-bda4-995cc1055d5f, api error AccessDenied: User: arn:aws:iam::58xxxxxx:user/terraform-user is not authorized to perform: redshift:GetClusterCredentials on resource: arn:aws:redshift:us-east-1:58xxxxxx:dbuser:redshift-test-9597/redshift_test_admin_6112

I guess the user is trying to access the GetClusterCredentials method without assuming the necessary role, as there's no such option in the module. It works if the Redshift cluster is located in the user's project.

The redshift_grant resource (https://registry.terraform.io/providers/brainly/redshift/latest/docs/resources/grant) currently only supports grants to groups, even though the equivalent native Redshift GRANT statement (https://docs.aws.amazon.com/redshift/latest/dg/r_GRANT.html) supports both groups and users.

Hello!

I would like to use Redshift provider to grant privileges for in-database machine learning.

GRANT CREATE MODEL
    TO { username [ WITH GRANT OPTION ] | ROLE role_name | GROUP group_name | PUBLIC } [, ...]

GRANT { EXECUTE | ALL [ PRIVILEGES ] }
    ON MODEL model_name [, ...]

    TO { username [ WITH GRANT OPTION ] | ROLE role_name | GROUP group_name | PUBLIC } [, ...]

If a datashare already exists and one changes this property, the error below is seen. It seems setting this specific option needs to run outside a transaction block

ALTER DATASHARE SET PUBLICACCESSIBLE cannot run inside a transaction block

I am trying to remove some users from a group but when run the apply I am getting an error from the provider that cannot find the user info.
I tried with different groups and users getting the same result. Not sure if there is a problem with the query in checkIfUserExists

module.aws-redshift.redshift_group.this["group_name"]: Modifying... [id=44633053]
2022-09-16T13:34:16.670+0200 [ERROR] vertex "module.aws-redshift.redshift_group.this[\"group_name\"]" error: error reading info about user: %!s(<nil>)
╷
│ Error: error reading info about user: %!s(<nil>)
│
│   with module.aws-redshift-data-redshift.redshift_group.this["group_name"],
│   on modules/aws-redshift/main.tf line 85, in resource "redshift_group" "this":
│   85: resource "redshift_group" "this" {
│
╵
´´´

Configuration:

# External schema using AWS Glue Data Catalog
resource "redshift_schema" "data_catalog" {
  name  = "db_schema"
  owner = redshift_user.owne.name
  external_schema {
    database_name = "db" 
    data_catalog_source {
      iam_role_arns = [
        "arn:aws:iam::....:role/....",
      ]
      create_external_database_if_not_exists = true 
    }
  }
}

After turn on debug mode:

redshift_schema.data_catalog: Creating...
2021-10-04T16:20:57.592+0200 [INFO]  Starting apply for redshift_schema.data_catalog
2021-10-04T16:20:57.593+0200 [DEBUG] redshift_schema.data_catalog: applying the planned Create change
2021-10-04T16:20:58.684+0200 [INFO]  provider.terraform-provider-redshift_v0.4.1: 2021/10/04 16:20:58 [DEBUG] creating external schema: CREATE EXTERNAL SCHEMA "db_schema" FROM DATA CATALOG DATABASE 'db' IAM_ROLE 'arn:aws:iam::....:role/....': timestamp=2021-10-04T16:20:58.684+0200
2021-10-04T16:20:58.823+0200 [INFO]  provider.terraform-provider-redshift_v0.4.1: 2021/10/04 16:20:58 [DEBUG] setting schema owner: ALTER SCHEMA "db_schema" OWNER TO "owner": timestamp=2021-10-04T16:20:58.823+0200
2021-10-04T16:21:02.523+0200 [INFO]  provider.terraform-provider-redshift_v0.4.1: 2021/10/04 16:21:02 [DEBUG]: sql: transaction has already been committed or rolled back: timestamp=2021-10-04T16:21:02.523+0200

Bug: There is missing create external database if not exists

Environment:

Terraform v1.0.8
on darwin_amd64
+ provider registry.terraform.io/brainly/redshift v0.4.1
+ provider registry.terraform.io/hashicorp/aws v3.61.0
+ provider registry.terraform.io/hashicorp/random v3.1.0

redshift_group is creating newgroup.

Is there anyway to add users to an existing group?

Like this below code I got from some page but not working.

resource "aws_redshift_group_membership" "example" {
  group_name = "my-redshift-group"
  users      = ["user1", "user2"]
}

When applying redshift_grant on schema level, this is translated to GRANT ... ON ALL TABLES. This does cover all tables, views and materialised views.

When reading table grants provider is checking grants only on tables. When view is recreated using CREATE OR REPLACE VIEW construct, grants are reset for the view and provider is not detecting drift on plan.

I'm trying to grant a redshift user access to use a shared database, created from a data share.

It works when doing it directly in the database with below command, but with the terraform provider it doesn't seem to be possible.

In database:
grant usage on database myDatabase to myUser;

Error message when using the provider:
Error: Invalid privileges list [usage] for object of type database

I've also tried to grant usage to the specific schema in the database but that gives me below error:
Error: could not start transaction: pq: Cannot connect to shared database "myDatabase". Connect to the databases in your cluster instead and use cross-database query notation <shareddatabase>.<schema>.<object> to query the data in shared database.

Hi,

I found out from AWS support that all access to ACLs will be revoked from Redshift, as they consider this to be internal information and don't want users to access it directly. This will break the provider, as it relies heavily on ACLs for grants and default privileges.

I asked them for alternatives, and they exist:

• svv_relation_privileges, svv_schema_privileges, svv_database_privileges and so on for table/schema/database/etc privileges
• svv_default_privileges for default privileges

They didn't tell me exactly when access will be revoked.

I'm not familiar with Go at all so I can't rewrite the whole thing, but I checked the readDatabaseGrants function to see what changes are required, and it seems doable with something like this:

func readDatabaseGrants(db *DBConnection, d *schema.ResourceData) error {
	var identityType, identityName, query string
	var databaseCreate, databaseTemp bool

	_, isUser := d.GetOk(grantUserAttr)

	if isUser {
		identityType = "user"
		identityName = d.Get(grantUserAttr).(string)
	} else {
		identityType = "group"
		identityName = d.Get(grantGroupAttr).(string)
	}

	query = `
SELECT privilege_type
FROM svv_database_privileges
WHERE
    database_name=$1
    AND identity_type=$2
    AND identity_name=$3
`

	queryArgs := []interface{}{db.client.databaseName, identityType, identityName}

...
}

Everything below that line would have to be updated, since the query now returns a list of privileges for that database and identity, and that's the part I can't do.

The Terraform aws provider allows setting AWS profile like this:

provider "aws" {
  profile = "customprofile"
}

Currently this Redshift provider does not allow that. The underlying aws-sdk-go-v2 implicitly uses the environment variable AWS_PROFILE if set.

This means that if you are using AWS profiles, you need to remember to run Terraform with AWS_PROFILE=customprofile terraform plan, or the GetClusterCredentials call will fail. This is pretty cumbersome.

I think it would be a good idea to add a top level configuration option to this Redshift provider, like this:

provider "redshift" {
  profile = "customprofile"
}

Implementing it should be pretty straight-forward unless I'm mistaken.

The aws-sdk-go-v2 allows specifying a profile explicitly:

cfg, err := config.LoadDefaultConfig(context.TODO(), 
	config.WithSharedConfigProfile("test-account"))

The provider schema would need to be expanded with the profile option:

The redshiftSdkClient function would need to pass the profile name to LoadDefaultConfig, if set, or else fall back to calling it as before:

Creating a redshift_schema resource with an rds_mysql_source block successfully creates the schema, but then fails with the error "Unsupported source database type unknown".

I just started using the provider and ran into an issue. I tried to grant permissions on a new schema to public, but this caused an error and broke my state file. Setting the user to be "public" results in an error like this during plan:

Error: failed to get user ID: sql: no rows in result set

I think the issue is when trying to check the state, the code runs a query filtering for the username, but public is special, so it isn't listed.

Would it be possible to add functionality to handle public?

Hello,

it seems that when deleting an user there is a query revoking privileges using cascade, which redshift seems to don't accept.
this is the query:

which retries until terraform exits successfully without actually droping the users.

removing cascade worked for me.
Id be happy to submit a pull request if thats ok with you.

GRANT ALL ON TABLE xyz leaves the table with a permission string of =arwdRxtD/ whereas the currently allowed set of GRANT permissions ("SELECT", "UPDATE", "INSERT", "DELETE", "DROP", "REFERENCES") can only set the table permission string to =arwdxD/.

Maybe possible solutions are to add an ALL possibility, or to add the R and t permissions?

The RULE and TRIGGER permissions are undocumented, but they are needed for redshift's materialized views implementation. A user who does not have those permissions to a materialized view can not refresh the view. https://docs.aws.amazon.com/redshift/latest/dg/materialized-view-refresh-sql-command.html

First of all, great plugin (:

I looked at the code, and I saw that when creating a new Datashare, it adds all the tables under the schema.
Is it possible to add a feature to create a new Dateshare with specific tables in a schema?

Thanks

This issue also affects redshift_permissions but we have migrated to redshift_grants so haven't created a test case for that.

The code used to check grants consists of SQL functions like these:

decode(charindex('r',split_part(split_part(array_to_string(relacl, '|'),gr.groname,2 ) ,'/',1)), 0,0,1) as select,

This is imperfect and gets confused in a number of cases. It's meant to parse ACLs like these:

{"group my_reader_group=r/my_user",my_user=arwdRxtD/myuser}

These have the format of grantee=perms/grantor and grantee may be an user in the form of username or group in the form of group groupname and matches against each table in a target schema

If a group exists with the same name as an user, the code above can break the following ways:

1. The code may match the grantor portion of an ACL entry and fail to pick up any permissions
2. It may pick up grants to an user rather than the group, depending on the order in which the ACL entries occur.
3. If a schema has 10 tables and the user has SELECT granted to only one of these, that may be interpreted by the code as having SELECT granted on all tables in the schema (I haven't verified this)
4. Related to the the above, the code may pick up grants created by redshift_default_privileges if a table was added to the schema, even if no redshift_grants resources were applied before.
5. I don't understand how schema grants are verified (if at all) when there are no tables in a schema and the above query returns zero rows.

Depending on the nature of the error this may result in an one-off diff or even a permanent diff, usually when tables are created outside of Terraform, especially if the creating user had some default permissions that would apply grants to a newly created table.

Hello

Would it be possible to enhance this framework to handle table creation/update as it is possible in others managed warehouses like BigQuery and Snowflake ?

We have multiple environment and we would like to make sure they have the same tables definition.

If not, do you have any recommandation to handle this use case of maintaining multiple redshift environment up to date ?

Thanks
Vincent

Full log

Terraform will perform the following actions:

  # redshift_privilege.contributor_schema_external will be created
  + resource "redshift_privilege" "contributor_schema_external" {
      + group       = "group_contributors"
      + id          = (known after apply)
      + object_type = "schema"
      + privileges  = [
          + "usage",
        ]
      + schema      = "my_external_schema"
    }

Plan: 1 to add, 0 to change, 0 to destroy.
redshift_privilege.contributor_schema_external: Creating...
redshift_privilege.contributor_schema_external: Still creating... [10s elapsed]
redshift_privilege.contributor_schema_external: Still creating... [20s elapsed]
redshift_privilege.contributor_schema_external: Still creating... [30s elapsed]
redshift_privilege.contributor_schema_external: Still creating... [40s elapsed]
redshift_privilege.contributor_schema_external: Still creating... [50s elapsed]
redshift_privilege.contributor_schema_external: Still creating... [1m0s elapsed]
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to redshift_privilege.contributor_schema_external,
│ provider "provider[\"registry.terraform.io/brainly/redshift\"]" produced an
│ unexpected new value: Root resource was present, but now absent.
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
Releasing state lock. This may take a few moments...
ERRO[0104] Module [..]/redshift-videoamp-database has finished with an error: 1 error occurred:
        * exit status 1
  prefix=[..]/redshift-videoamp-database] 
ERRO[0104] 1 error occurred:
        * exit status 1

Hello,

It would be great if we could use redshift_grant resource to grant EXECUTE on functions.

Redshift specifications for granting and revoking this permission are:

• GRANT:

GRANT { EXECUTE | ALL [ PRIVILEGES ] }
    ON { FUNCTION function_name ( [ [ argname ] argtype [, ...] ] ) [, ...] | ALL FUNCTIONS IN SCHEMA schema_name [, ...] }
    TO { username [ WITH GRANT OPTION ] | GROUP group_name | PUBLIC } [, ...]

• REVOKE:

REVOKE [ GRANT OPTION FOR ]
EXECUTE 
    ON FUNCTION function_name ( [ [ argname ] argtype [, ...] ] ) [, ...]
    FROM { username | GROUP group_name | PUBLIC } [, ...]
[ CASCADE | RESTRICT ]

We could do the same for procedures, which is very close.

Hello

I usully use grant assume to assign a role to a group like this :

GRANT ASSUMEROLE ON 'arn:aws:iam::xxxxxxxxxxx:role/xxxxxxxxxxxx-01' TO GROUP xxx_GRP_RW FOR COPY, UNLOAD;

Unless I'm mistaken in your module it's not possible to specify a iam role right ?

Can this is be a supported features?

Many Thanks!
OvO

Goal

When I try to implement grant privilege including truncate I have the following error

Error: Invalid privileges list [select truncate update insert delete] for object of type table

  on redshift_queries.tf line 82, in resource "redshift_grant" "data_group_tables":
  82: resource "redshift_grant" "data_group_tables" {

If I comment the truncate privilege, everything goes fine.

Part of my code where I have the issue

# create data group
resource "redshift_group" "data_group" {
  depends_on = [aws_redshift_cluster.redshift_cluster]
  name       = "DATA_USERS"
  users = [
    redshift_user.data_user.name
  ]
}

# Init tables if needed
resource "aws_redshiftdata_statement" "init_tables" {
  depends_on         = [redshift_schema.schema]
  for_each           = local.sql_table_list
  cluster_identifier = aws_redshift_cluster.redshift_cluster.cluster_identifier
  database           = aws_redshift_cluster.redshift_cluster.database_name
  db_user            = aws_redshift_cluster.redshift_cluster.master_username
  statement_name     = replace(each.value.file_name, ".sql", "")
  sql                = file("templates/${each.value.file_name}")
}

# add grant on tables
resource "redshift_grant" "data_group_tables" {
  for_each    = aws_redshiftdata_statement.init_tables
  group       = redshift_group.data_group.name
  schema      = redshift_schema.schema.name
  object_type = "table"
  objects     = [each.key]
  privileges = [
    "select",
    "insert",
    "delete",
    "truncate",
    "update"
  ]
}

Part of the solution (I guess)

After searching a bit in the code, I found that the error comes from the validatePrivileges function from the helpers.go file (L169 for the case "TABLE")

There are probably other stuff to do but I don't have really the time to investigate for now.

AWS doc

Part of the doc for the truncate privilege

Thank you for your feedback

Hello, I am trying to create a redshift data share and I am getting below error. The cluster is not accessible externally. Does this plugin requires it be public ?

operation error Redshift: GetClusterCredentials, https response error StatusCode: 404, RequestID: 54f7c64e-ecba-4910-a982-0134da944dcf, ClusterNotFoundFault: Cluster redshift-cluste not found.

This used to work, but has been broken by always hashing the password string before sending it to CREATE USER.

There are now two PRs to fix this - I did the second one because I forgot there was already one:

#90

#96

On a fresh terraform state, and the following tf config (notice the error in the schema owner - is user but should be xuser)

output saved to TF_LOG_FILE: log.broke.txt
:

terraform {
  required_providers {
    redshift = {
      source  = "brainly/redshift"
      version = "0.5.1"
    }
  }
}
variable "redshift_host" { type = string }
variable "redshift_username" { type = string }
variable "redshift_password" {
  type      = string
  sensitive = true
}
variable "redshift_database" { type = string }
provider "redshift" {
  host            = var.redshift_host
  username        = var.redshift_username
  password        = var.redshift_password
  database        = var.redshift_database
  sslmode         = "require"
  max_connections = 0
}
resource "redshift_user" "user" {
  name      = "xuser"
}
resource "redshift_group" "group" {
  name  = "xgroup"
  users = ["xuser"]
}
resource "redshift_schema" "schema" {
  name  = "xschema"
  owner = "user"
}
resource "redshift_grant" "grants" {
  group       = "xgroup"
  schema      = "xschema"
  object_type = "table"
  privileges  = ["SELECT", "INSERT", "UPDATE", "DELETE", "DROP", "REFERENCES"]
}

got the following error:

% terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # redshift_grant.grants will be created
  + resource "redshift_grant" "grants" {
      + group       = "xgroup"
      + id          = (known after apply)
      + object_type = "table"
      + privileges  = [
          + "delete",
          + "drop",
          + "insert",
          + "references",
          + "select",
          + "update",
        ]
      + schema      = "xschema"
    }

  # redshift_group.group will be created
  + resource "redshift_group" "group" {
      + id    = (known after apply)
      + name  = "xgroup"
      + users = [
          + "xuser",
        ]
    }

  # redshift_schema.schema will be created
  + resource "redshift_schema" "schema" {
      + id    = (known after apply)
      + name  = "xschema"
      + owner = "user"
      + quota = 0
    }

  # redshift_user.user will be created
  + resource "redshift_user" "user" {
      + connection_limit = -1
      + create_database  = false
      + id               = (known after apply)
      + name             = "xuser"
      + superuser        = false
      + valid_until      = "infinity"
    }

Plan: 4 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

redshift_user.user: Creating...
redshift_group.group: Creating...
redshift_grant.grants: Creating...
redshift_schema.schema: Creating...
redshift_user.user: Creation complete after 2s [id=166]
redshift_grant.grants: Still creating... [10s elapsed]
redshift_grant.grants: Still creating... [20s elapsed]
redshift_grant.grants: Still creating... [30s elapsed]
redshift_grant.grants: Still creating... [40s elapsed]
redshift_grant.grants: Still creating... [50s elapsed]
redshift_grant.grants: Still creating... [1m0s elapsed]
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to redshift_grant.grants, provider "provider[\"registry.terraform.io/brainly/redshift\"]" produced an unexpected new value: Root resource
│ was present, but now absent.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Could not create redshift group: pq: user "xuser" does not exist
│ 
│   with redshift_group.group,
│   on main.tf line 41, in resource "redshift_group" "group":
│   41: resource "redshift_group" "group" {
│ 
╵
╷
│ Error: pq: user "user" does not exist
│ 
│   with redshift_schema.schema,
│   on main.tf line 46, in resource "redshift_schema" "schema":
│   46: resource "redshift_schema" "schema" {
│ 

state after only has one resource: redshift_user.user

if you create a resource like so:

resource "redshift_schema" "external" {
    name = var.external_schema_name
    owner = "owner"
    external_schema {
        database_name = var.catalog_database_name
        data_catalog_source {
            iam_role_arns = [
                var.spectrum_iam_role_arn
            ]
            create_external_database_if_not_exists = true
        }
    }
}

resource "redshift_grant" "schema_group_engineering" {
    object_type     = "schema"

    schema          = var.external_schema_name
    privileges      = ["create, "usage"]   # effective equivalent of ALL
    group           = "engineering"
}

This gets into an infinite loop where each cycle to plan/apply is expecting to add the 'create' permission to the resource.

When I run this series under a debug log, I see the following lines that are relevant:

2022-06-22T23:41:08.831+0700 [INFO]  provider.terraform-provider-redshift_v1.0.0: 2022/06/22 23:41:08 [DEBUG] Created REVOKE query: REVOKE ALL PRIVILEGES ON SCHEMA "warehouse_external" FROM GROUP "engineering": timestamp=2022-06-22T23:41:08.830+0700
2022-06-22T23:41:08.878+0700 [INFO]  provider.terraform-provider-redshift_v1.0.0: 2022/06/22 23:41:08 [DEBUG] Created GRANT query: GRANT create,usage ON SCHEMA "warehouse_external" TO GROUP "engineering": timestamp=2022-06-22T23:41:08.878+0700
2022-06-22T23:41:10.174+0700 [INFO]  provider.terraform-provider-redshift_v1.0.0: 2022/06/22 23:41:10 [DEBUG] Collected schema 'warehouse_external' privileges for  engineering: [usage]: timestamp=2022-06-22T23:41:10.174+0700
2022-06-22T23:41:10.174+0700 [INFO]  provider.terraform-provider-redshift_v1.0.0: 2022/06/22 23:41:10 [DEBUG]: sql: transaction has already been committed or rolled back: timestamp=2022-06-22T23:41:10.174+0700

As you can see, the provider built a command: GRANT create,usage ON SCHEMA "warehouse_external" TO GROUP "engineering"

and then shortly after, when the provider read the permissions back out, only 'usage' is seen Collected schema 'warehouse_external' privileges for engineering: [usage]

Thus, we can see that the command is only setting a subset of the permissions.

If you try to run this GRANT command manually against the system, what you can see Is that Redshift is actually reporting an error (or could be a warning) that the provider is not catching and bubbling up to the user. The error from Redshift looks like this:

CREATE privilege on external schema can only be granted to IAM Roles using GRANT on EXTERNAL SCHEMA.  For users or groups, only USAGE privilege can be granted or ownership of external schema can be transfered.

I would ask that you catch this type of error from Redshift and expose it to the engineer running the apply plan. The fix on the resource is pretty simple in this case (remove the 'create' permission). But the terraform user may spend quite a bit of time trying to understand what is going wrong.

The redshift_privilege resource currently creates grants but also runs ALTER DEFAULT PRIVILEGES. This appears to be due to a misunderstanding of how default privileges work in Redshift.

Per https://docs.aws.amazon.com/redshift/latest/dg/r_ALTER_DEFAULT_PRIVILEGES.html:

Defines the default set of access privileges to be applied to objects that are created in the future by the specified user.
...
FOR USER target_user
Optional. The name of the user for which default privileges are defined... The default value is the current user.

Default privileges are essentially grants that are automatically applied when a particular user creates an object. In other words, if you do:

resource "redshift_privilege" "readonly_tables" {
  group       = "test_group"
  schema      = "public"
  object_type = "table"
  privileges  = ["SELECT"]
}

What actually runs is this:

ALTER DEFAULT PRIVILEGES
IN SCHEMA "public"
GRANT SELECT ON TABLES
TO GROUP "test_group"

which is equivalent to this:

ALTER DEFAULT PRIVILEGES
FOR USER CURRENT_USER
IN SCHEMA "public"
GRANT SELECT ON TABLES
TO GROUP "test_group"

which actually means the select permissions will only be given on new tables created by whoever ran terraform.

If what we're really trying to achieve here is for test_group to have select permissions on newly created tables in public then what we actually have to do is run something akin to:

-- redshift doesn't actually support a for loop like this,
-- but you get the idea.
FOR db_user IN (SELECT usename FROM pg_user) DO
  ALTER DEFAULT PRIVILEGES
  FOR USER db_user
  IN SCHEMA "public"
  GRANT SELECT ON TABLES
  TO GROUP "test_group"
END FOR

hi,
I used your provider script is handy, but I have more than 200 users for each schema and I have more than 200 schemas it works fine for 30 schemas but when I am using it for more than 40schemas then terraform plans get stuck for more than 1 hr and falling. can you please help me to run more schemas I run terraform in my local and GitHub Actions and terraform.io seeing the same issue.

I've been trying to destroy a redshift DB and getting this error, not too sure if this is a bug or if we're running into something else here.
The only way we can get round it is to either delete it manually.

Terraform will perform the following actions:

  # redshift_database.db will be destroyed
  - resource "redshift_database" "db" {
      - connection_limit = -1 -> null
      - id               = "27376203" -> null
      - name             = "database-prd" -> null
      - owner            = (sensitive) -> null
    }

Plan: 0 to add, 0 to change, 1 to destroy.
redshift_database.db: Destroying... [id=27376203]
╷
│ Error: pq: syntax error at or near "-"
│ 
│ 
╵
Problem applying destroying terraform environment

Hello.
I am trying to give an user syslog_access="UNRESTRICTED" while this user is not a superuser, since I know this is not a problem in Redshift but I am finding this error o the plan phase.

│ Error: Conflicting configuration arguments
│
│   with module....
│   on modules/.../main.tf line 4, in resource "redshift_user" "this":
│    4:   superuser       = var.superuser
│
│ "superuser": conflicts with syslog_access

There is a way to solve it? Thanks

When I look at the log, I see a large number of warnings about non-computed attribute like

2021-12-21T14:22:02.999-0100 [WARN]  Provider "registry.terraform.io/brainly/redshift" produced an in
valid plan for redshift_user.xyz, but we are tolerating it because it is using the legacy plugin
 SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .superuser: planned value cty.False for a non-computed attribute
      - .syslog_access: planned value cty.StringVal("RESTRICTED") for a non-computed attribute
      - .valid_until: planned value cty.StringVal("infinity") for a non-computed attribute
      - .connection_limit: planned value cty.NumberIntVal(-1) for a non-computed attribute
      - .create_database: planned value cty.False for a non-computed attribute

I dont know if these are actual problems, but they make it less nice to read the logs :-)

Whenever a user name or group name uses one of the valid ACSII characters between 33–126, the name is recorded with double quotations for ACLs.
For such cases reading grants is defective and returns nothing.
It results in perpetual reapplying grants despite their presence in the database.

When we have two databases are configured in a cluster and with different schema's are created on the databases, How can we grant privileges for different schema's on different databases. In the below example, if the raw schema is in one database and staging schema is on another database and our terraform point the 1st database, does the below second resource can identify the staging schema in 2nd database and assigns the necessary privileges?

resource "redshift_default_privileges" "de_raw_transform" {
group = "transform"
owner = "dataeng"
schema = "raw"
object_type = "table"
privileges = ["select"]
}

resource "redshift_default_privileges" "de_staging_transform" {
group = "transform"
owner = "dataengg"
schema = "staging"
object_type = "table"
privileges = ["select"]
}

We are trying to grant "execute" for a number of groups on procedure/functions in public schema and terraform constantly wants to re-apply grants.

# redshift_grant.grant_group_public_schema_procedure_grants["xapi"] will be updated in-place
  ~ resource "redshift_grant" "grant_group_public_schema_procedure_grants" {
        id          = "gn:xapi_ot:procedure_public"
      ~ privileges  = [
          + "execute",
        ]
        # (3 unchanged attributes hidden)
    }

DEBUG logs snippet:

2022/12/06 12:19:10 [WARN] Provider "registry.terraform.io/brainly/redshift" produced an unexpected new value for redshift_grant.grant_group_public_schema_procedure_grants["xapi"] during refresh.
      - .privileges: planned set element cty.StringVal("execute") does not correlate with any element in actual
2022/12/06 12:19:10 [TRACE] EvalWriteState: using RefreshState for redshift_grant.grant_group_public_schema_procedure_grants["xapi"]
2022/12/06 12:19:10 [TRACE] EvalWriteState: recording 51 dependencies for redshift_grant.grant_group_public_schema_procedure_grants["xapi"]

Redshift provider version: 1.0.3
Terraform version: 0.14.9

Symptoms look similar to #94 .

The following produces a state with two redshift_grant resources having identical id. I think makes it impossible to ever have an import for the redshift_grant resource?

Maybe it could use . or - to join the parts? (invalid name characters https://docs.aws.amazon.com/redshift/latest/dg/r_names.html)

(yeah, it's a little contrived, but imagine it were object_type = "table" instead of schema eventually someone is going to have something like that, even if it does seem like poor naming choices)

Working import for would be really helpful for the process of terraforming an existing database - to validate that the grant resource matches the existing db grants. ie terraform import redshift_grant.mygrant y.table.x

terraform {
  required_providers {
    redshift = {
      source  = "brainly/redshift"
      version = "0.5.1"
    }
  }
}

variable "redshift_host" { type = string }
variable "redshift_username" { type = string }
variable "redshift_password" {
  type      = string
  sensitive = true
}
variable "redshift_database" { type = string }

provider "redshift" {
  host            = var.redshift_host
  username        = var.redshift_username
  password        = var.redshift_password
  database        = var.redshift_database
  sslmode         = "require"
  max_connections = 0
}

resource "redshift_user" "user" {
  name      = "xuser"
}

resource "redshift_group" "y_schema" {
  name  = "y_schema"
  users = [redshift_user.user.name]
}

resource "redshift_group" "y" {
  name  = "y"
  users = [redshift_user.user.name]
}

resource "redshift_schema" "x" {
  name  = "x"
  owner = redshift_user.user.name
}

resource "redshift_schema" "schema_x" {
  name  = "schema_x"
  owner = redshift_user.user.name
}

resource "redshift_grant" "grants" {
  group       = redshift_group.y_schema.name
  schema      = redshift_schema.x.name
  object_type = "schema"
  privileges  = ["USAGE"]
  #privileges = ["SELECT", "INSERT", "UPDATE", "DELETE", "DROP", "REFERENCES"]
}

resource "redshift_grant" "grants2" {
  group       = redshift_group.y.name
  schema      = redshift_schema.schema_x.name
  object_type = "schema"
  privileges  = ["USAGE"]
  #privileges = ["SELECT", "INSERT", "UPDATE", "DELETE", "DROP", "REFERENCES"]
}