googlecloudplatform / terraformer Goto Github PK

View Code? Open in Web Editor NEW

11.8K 141.0 1.6K 67.61 MB

CLI tool to generate terraform files from existing infrastructure (reverse Terraform). Infrastructure to Code

License: Apache License 2.0

Go 99.20% HCL 0.80%

cloud terraform terraform-configurations gcp google-cloud hcl golang infrastructure-as-code aws kubernetes

terraformer's Issues

Do not create "empty" folders when no resources were found

when I do

terraformer import google --resources=addresses,dns

but if any addresses resource were not found in the project, I got an addresses folder with providers.tf and "empty" tf state file.

Can we avoid this by adding CLI parameter like --empty-resource-folders=false?

GCP: Add support Google Cloud Build

Add ability to import GCP Project

It would be cool if you will add support for google_project resource and use google_project.projectname.id and google_project.projectname.name for child resources as a reference.

Infinity loop when trying to import not existant Stackdriver

When I'm trying to import projects resource monitoring but never actually use it I receive tons of repeating errors:

$ terraformer import google --projects projectname --resources monitoring
2019/05/10 08:53:38 google importing project projectname
2019/05/10 08:53:38 google importing... monitoring
2019/05/10 08:53:40 error with alert: rpc error: code = InvalidArgument desc = 'projects/projectname' is not a Stackdriver workspace.
2019/05/10 08:53:40 error with alert: rpc error: code = InvalidArgument desc = 'projects/projectname' is not a Stackdriver workspace.
2019/05/10 08:53:40 error with alert: rpc error: code = InvalidArgument desc = 'projects/projectname' is not a Stackdriver workspace.

And it repeats until I stop it with C-c.

Plugin error during a try of importing project

I use the binary release, and the binary after having cloned the repository. I've got same error :

$ ~/bin/terraformer-linux-amd64 import google --resources=gcs,firewalls,forwardingRules,networks --connect=true --projects=mygcpproject
2019/05/06 09:17:33 google importing project mygcpproject
2019/05/06 09:17:33 google importing... gcs
2019/05/06 09:17:34 plugin error: fork/exec : no such file or directory
2019/05/06 09:17:34 fork/exec : no such file or directory

Add support XenOrchestra Provider

https://github.com/terra-farm/terraform-provider-xenorchestra

Add support Github provider

https://www.terraform.io/docs/providers/github/

GCP: Add support Google Dataproc Resources

GCP: Add support google_cloud_scheduler_job

GCP: Add support Google App Engine Resources

Add support Kubernetes Provider

Kubernetes provider can be supported by terraformer.
Use kubernetes go client for get IDs.
Use kubectl config default file.

docs: https://www.terraform.io/docs/providers/kubernetes/index.html

Can it support a custom endpoint (similar to TF options)

https://www.terraform.io/docs/providers/aws/guides/custom-service-endpoints.html

AWS Route53 alias record will be imported with TTL

AWS Route53 alias record will be imported to route53_record.tf like below.

resource "aws_route53_record" "AAAAAAAAA_example-net-_A" {
  alias {
    evaluate_target_health = false
    name                   = "example.net"
    zone_id                = "AAAAAAAAA"
  }

  name    = "example.net"
  ttl     = "0"
  type    = "A"
  zone_id = "${aws_route53_zone.example-net.zone_id}"
}

However, ttl is not required for alias record.
cf: AWS: aws_route53_record - Terraform by HashiCorp

Add support Cloudflare Provider

https://www.terraform.io/docs/providers/cloudflare/index.html

Hi
This is not an issue
First congratulation for the greate tool
I want to know if you have best practices or recommendation for the better use with GCP
For example I must run the import with all the resources at same time or you recommend separate for example networks from instances??

GCP: Add support google_dataflow_job

must `import google subnetworks` set ZONE?

Hi
When I forgot ZONE(eg:like -a,-b), error has occured, I think this it unxpected behavior.

$ terraformer import google --resources=subnetworks --connect=true --zone=asia-northeast1 --projects=prj
2019/06/13 12:17:29 google importing project prj
2019/06/13 12:17:29 google importing... subnetworks
2019/06/13 12:17:30 googleapi: Error 400: Invalid value for field 'region': 'asia'. Unknown region., invalid

When i added ZONE, it works well.

$ terraformer import google --resources=subnetworks --connect=true --zone=asia-northeast1-a --projects=prj
2019/06/13 12:17:22 google importing project prj
2019/06/13 12:17:22 google importing... subnetworks
2019/06/13 12:17:26 google Connecting.... 
2019/06/13 12:17:26 google save subnetworks
2019/06/13 12:17:26 [DEBUG] New state was assigned lineage "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
2019/06/13 12:17:26 google save tfstate for subnetworks

subnetworks, addresses occures same error too.

Access denied for aws import

Hey there,

I am trying to test this cool tool to import some aws resources into our tf project.
Problem is, we are using cross account assumed roles, with 2FA enabled. For example, my ~/.aws/credentials file looks like this:

[default]
aws_access_key_id = ...
aws_secret_access_key = ...

[profile1]
role_arn = arn:aws:iam::<account-id-1>:role/<role-name-1>
source_profile = default
mfa_serial = arn:aws:iam::<central-account-id>:mfa/<my-username>

[profile2]
role_arn = arn:aws:iam::<account-id-2>:role/<role-name-2>
source_profile = default
mfa_serial = arn:aws:iam::<central-account-id>:mfa/<my-username>

so when trying to import s3 buckets, with this commands: terraformer import aws --resources=s3 --connect=true --regions=eu-west-1
I am getting:

2019/05/08 13:13:47 aws importing region eu-west-1
2019/05/08 13:13:47 aws importing... s3
2019/05/08 13:13:47 AccessDenied: Access Denied
	status code: 403, request id: 7164....5B8F351, host id: 7q8nhD6LnVeniWww9arkIcBQX.................NdzDgMCNBhC610MZM=

Ismy cross-account setup currently supported by terraformer?

GCP: Add support Google BigQuery Resources

Add support Azure

GCP: Add support google_filestore_instance

GCP: Add support google_redis_instance

Filtering route53 resources documentation

Hello,

I struggled to understand how to use filters with aws route53 resources. It may be useful to write somewhere how to fetch hosted zone IDs (with aws-cli but whithout the /hostedzone/ part) and that records ID are made of <hosted zone id>_<fqdn>_<record type>.

I can make a PR if you tell me where this documentation belongs.

GCP: Add support KMS Resources

Add support Grafana Provider

Grafana best open source tool for visualization metrics.
Grafana Provider support only 4 resources

grafana_alert_notification
grafana_dashboard
grafana_data_source
grafana_organization

Grafana has api for get ID for each resources.
API docs: https://grafana.com/docs/http_api/
maybe API Token need be pass in flags
It's can be usable for users with a lot of number alerts and dashboards.

Comparison w/ terraforming project

I see that you link to it as a similar project, but it would be helpful to understand what the pros/cons of terraformer vs. terraforming would be for a user.

import google monitoring returns empty tf files.

Hi, I'm using project owner account to import. but import google monitoring returns empty .tf files below.

terraformer import google --resources=monitoring --connect=true --zone=asia-northeast1 --projects=my-gcp-prj
2019/06/11 15:56:57 google importing project my-gcp-prj
2019/06/11 15:56:57 google importing... monitoring
2019/06/11 15:57:00 project: required field is not set
2019/06/11 15:57:00 project: required field is not set
2019/06/11 15:57:00 project: required field is not set
2019/06/11 15:57:00 project: required field is not set
2019/06/11 15:57:00 project: required field is not set
2019/06/11 15:57:00 project: required field is not set
2019/06/11 15:57:00 project: required field is not set
2019/06/11 15:57:00 project: required field is not set
2019/06/11 15:57:00 project: required field is not set
2019/06/11 15:57:00 google Connecting.... 
2019/06/11 15:57:00 google save monitoring
2019/06/11 15:57:00 [DEBUG] New state was assigned lineage  "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
2019/06/11 15:57:00 google save tfstate for monitoring

monitoring_alert_policy.tf

resource "google_monitoring_alert_policy" "projects--my-gcp-prj--alertPolicies--12270733397295926724" {}

resource "google_monitoring_alert_policy" "projects--my-gcp-prj--alertPolicies--18416277687926950534" {}

resource "google_monitoring_alert_policy" "projects--my-gcp-prj--alertPolicies--2641331155153181887" {}

resource "google_monitoring_alert_policy" "projects--my-gcp-prj--alertPolicies--6165830597848982763" {}

monitoring_notification_channel.tf

resource "google_monitoring_notification_channel" "projects--my-gcp-prj--notificationChannels--14664454090038840730" {}

resource "google_monitoring_notification_channel" "projects--my-gcp-prj--notificationChannels--7956856499735477724" {}

resource "google_monitoring_notification_channel" "projects--my-gcp-prj--notificationChannels--9929777985012302021" {}

monitoring_uptime_check_config.tf

resource "google_monitoring_uptime_check_config" "projects--my-gcp-prj--uptimeCheckConfigs--uptime-check-for-api-1559734933" {}

resource "google_monitoring_uptime_check_config" "projects--my-gcp-prj--uptimeCheckConfigs--uptime-check-for-web-1559734602" {}

Multiple AWS Route53 zones with same domain name causes "duplicated resource" error

If you have multiple zones configured on AWS Route 53 with same domain name, Terraformer raises an error stating that resources are duplicated.

duplicate resource found: aws_route53_zone.example--com

GCP: Add support Google PubSub Resources

GCP: Add support Google Spanner Resources

Any plans vSphere/NSX-T provider?

https://github.com/terraform-providers/terraform-provider-nsxt
https://github.com/terraform-providers/terraform-provider-vsphere

Honestly, my personal use-case is weak and novel; I'd only use this for a homelab vSphere environment. Just curious if there were any plans to interface w/ these providers in the future?

Thanks, feel free to close this whenever.

import google `dns` resource is always unsuccess. (2019/5/5)

I tried to get google dns resource, but failed.
The console output is like below:

cd /path/to/working_directory 

❯ ./terraformer import google --resources=dns --connect=true --projects=MY_PROJECT, --state=bucket
2019/05/05 13:35:52 google importing project MY_PROJECT
2019/05/05 13:35:52 google importing... dns
2019/05/05 13:35:55 project: required field is not set
2019/05/05 13:35:55 project: required field is not set
2019/05/05 13:35:55 project: required field is not set
2019/05/05 13:35:55 project: required field is not set
2019/05/05 13:35:55 project: required field is not set
2019/05/05 13:35:55 project: required field is not set
2019/05/05 13:35:55 project: required field is not set
2019/05/05 13:35:55 project: required field is not set
2019/05/05 13:35:55 project: required field is not set
2019/05/05 13:35:55 project: required field is not set
panic: interface conversion: interface {} is nil, not string

goroutine 1 [running]:
github.com/GoogleCloudPlatform/terraformer/gcp_terraforming.(*CloudDNSGenerator).PostConvertHook(0xc000168400, 0x2ff0980, 0xc000168400)
	/path/to/working_directory/gcp_terraforming/clouddns.go:125 +0x6a7
github.com/GoogleCloudPlatform/terraformer/cmd.Import(0x2ff0b60, 0xc0000b9e40, 0xc0004ae650, 0x1, 0x1, 0xc0000c6a50, 0x30, 0x2c20d86, 0x9, 0x7ffeefbff8c2, ...)
	/path/to/working_directory/cmd/import.go:88 +0x1ad
github.com/GoogleCloudPlatform/terraformer/cmd.newCmdGoogleImporter.func1(0xc0004c0780, 0xc000331a90, 0x0, 0x5, 0x0, 0x0)
	/path/to/working_directory/cmd/google.go:37 +0x34a
github.com/spf13/cobra.(*Command).execute(0xc0004c0780, 0xc000331a40, 0x5, 0x5, 0xc0004c0780, 0xc000331a40)
	/Users/ME/go/pkg/mod/github.com/spf13/[email protected]/command.go:762 +0x465
github.com/spf13/cobra.(*Command).ExecuteC(0xc0004c0280, 0xc0000b9440, 0xc0000b9400, 0xc00047bf88)
	/Users/ME/go/pkg/mod/github.com/spf13/[email protected]/command.go:852 +0x2ec
github.com/spf13/cobra.(*Command).Execute(...)
	/Users/ME/go/pkg/mod/github.com/spf13/[email protected]/command.go:800
github.com/GoogleCloudPlatform/terraformer/cmd.Execute(0x0, 0x0)
	/path/to/working_directory/cmd/root.go:34 +0x28
main.main()
	/path/to/working_directory/main.go:25 +0x22

I've upgraded terraform provider google 1.20 -> 2.0, but import google dns has failed as same.

GCP: Add support GKE(few bugs)

GKE today is supported, but with few bugs.
Need tested and fix bugs.

Delete optinal numeric zero values

When an optinal numeric value is missing, terraform puts a 0 in it's state and it won't be removed by current delete empty value code block which is just removing the empty strings.

As there are some optinal nemeric values that has a must-be-positive/non-zero constraint, having zero for the missing optinal numeric values is causing the resource to be rejected.

plugin error: readdirent: invalid argument

Hello!
First things first: I love this project! This is a tool/feature I'm missing in terraform.

I ran into problems right away though.

I'm using the binary https://github.com/GoogleCloudPlatform/terraformer/releases/download/0.7/terraformer-darwin-amd64

with this gcp provider's binary for darwin
https://releases.hashicorp.com/terraform-provider-google/2.5.1/
(I tried the 2.5.0 too with the same result)

When running this or any other command on my project:

terraformer import google --resources=gcs,firewalls  --zone=europe-west1-b --projects=my-secret-project

I'm getting

2019/05/03 14:38:36 plugin error: readdirent: invalid argument
2019/05/03 14:38:36 readdirent: invalid argument

Any ideas?

Error 403: Required 'compute.instances.list' permission for 'projects/project', forbidden

Hi Guys
Im having this issue running terraformer against another project different that the own the VM with terraformer.

When I run terraformer against resources of the same project is OK
But When I run terraformer against resources of another project I get the follow error

Error 403: Required 'compute.instances.list' permission for 'projects/project', forbidden

The target project has the compute.googleapis.com and the my user has Project Owner permissions

Thanks for your help

Not able to install terraformer

Hi, I am newbie here.
OS: Windows 10
Following the Readme.md, I am able to clone the git repo. But after that I hit complete road block.

Can you please add documentation for Windows as well.
I have installed go lang as well.
but not sure how to execute this command ?

Run GO111MODULE=on go mod vendor
Run go build -v
Copy your Terraform provider's plugin(s) to folder ~/.terraform.d/plugins/{darwin,linux}_amd64/, as appropriate.

go build -v fails

Running $ go build -v
Getting these errors. help will be much appreciated.

# github.com/terraform-providers/terraform-provider-openstack/openstack
../../../go/pkg/mod/github.com/terraform-providers/[email protected]/openstack/data_source_openstack_images_image_v2.go:162:3: unknown field 'Tag' in struct literal of type "github.com/gophercloud/gophercloud/openstack/imageservice/v2/images".ListOpts
../../../go/pkg/mod/github.com/terraform-providers/[email protected]/openstack/resource_openstack_blockstorage_volume_v1.go:212:3: cannot use d.Get("name").(string) (type string) as type *string in field value
../../../go/pkg/mod/github.com/terraform-providers/[email protected]/openstack/resource_openstack_blockstorage_volume_v1.go:213:3: cannot use d.Get("description").(string) (type string) as type *string in field value
../../../go/pkg/mod/github.com/terraform-providers/[email protected]/openstack/resource_openstack_blockstorage_volume_v2.go:223:3: cannot use d.Get("name").(string) (type string) as type *string in field value
../../../go/pkg/mod/github.com/terraform-providers/[email protected]/openstack/resource_openstack_blockstorage_volume_v2.go:224:3: cannot use d.Get("description").(string) (type string) as type *string in field value
../../../go/pkg/mod/github.com/terraform-providers/[email protected]/openstack/resource_openstack_blockstorage_volume_v2.go:286:27: not enough arguments in call to "github.com/gophercloud/gophercloud/openstack/blockstorage/v2/volumes".Delete
	have (*gophercloud.ServiceClient, string)
	want (*gophercloud.ServiceClient, string, "github.com/gophercloud/gophercloud/openstack/blockstorage/v2/volumes".DeleteOptsBuilder)
../../../go/pkg/mod/github.com/terraform-providers/[email protected]/openstack/resource_openstack_compute_instance_v2.go:567:3: undefined: availabilityzones.ServerExt
../../../go/pkg/mod/github.com/terraform-providers/[email protected]/openstack/resource_openstack_compute_secgroup_v2.go:175:3: cannot use d.Get("description").(string) (type string) as type *string in field value
../../../go/pkg/mod/github.com/terraform-providers/[email protected]/openstack/resource_openstack_dns_recordset_v2.go:185:26: cannot use d.Get("description").(string) (type string) as type *string in assignment
../../../go/pkg/mod/github.com/terraform-providers/[email protected]/openstack/resource_openstack_dns_zone_v2.go:191:26: cannot use d.Get("description").(string) (type string) as type *string in assignment
../../../go/pkg/mod/github.com/terraform-providers/[email protected]/openstack/resource_openstack_dns_zone_v2.go:191:26: too many errors

Possible wrong formatting or mistype in import google --help

Import current State to terraform configuration from google cloud

Usage:
   import google [flags]
   import google [command]

Available Commands:
  list        Import current State to terraform configuration

Flags:
  -b, --bucket string        gs://terraform-state
  -c, --connect               (default true)
  -f, --filter strings       google_compute_firewall=id1:id2:id4
  -h, --help                 help for google
  -o, --path-output string    (default "generated")
  -p, --path-patter string   {output}/{provider}/custom/{service}/ (default "{output}/{provider}/{service}/")
      --projects strings     
  -r, --resources strings    firewalls,networks
  -s, --state string         local or bucket (default "local")
  -z, --zone string

Use " import google [command] --help" for more information about a command.

--path-patter looks incorrect for me.
I suspect output format issue or just a simple mistype in the option name.

Importing wildcard DNS records from AWS Route53 fail

Hello,

Terraformer's output is empty when importing wildcard DNS records from Route53.

terraformer import aws --resources=route53 --filter=aws_route53_record='myzoneid_*.example.com._CNAME' --regions=eu-west-3

There is no problem importing this record from terraform import.

go.sum checksum mismatch (2019/5/4)

I got the error below:

❯ go build -v
go: downloading github.com/ryanuber/columnize v2.1.0+incompatible
go: downloading github.com/mitchellh/colorstring v0.0.0-20150917214807-8631ce90f286
go: downloading github.com/mitchellh/panicwrap v0.0.0-20170106182340-fce601fe5557
go: extracting github.com/ryanuber/columnize v2.1.0+incompatible
go: extracting github.com/mitchellh/colorstring v0.0.0-20150917214807-8631ce90f286
go: extracting github.com/mitchellh/panicwrap v0.0.0-20170106182340-fce601fe5557
go: downloading github.com/hashicorp/go-getter v0.0.0-20181119194526-bd1edc22f8ea
verifying github.com/hashicorp/[email protected]: checksum mismatch
	downloaded: h1:o+/o9FSMvgMmv5Ss+EUutVEwZfcFR1nJKzOqlVJGFPY=
	go.sum:     h1:B2Aqv5hbW6F55OFgXAcSqGhqKi4Jb6IRLkKoA5ohgzU=

please check and correct go.sum.

(workaround : rename go.sum to go.sum_ , then run go build -v again.)

add travis-ci support

add .travis-ci.yaml
go build -v
go vet
gofmt

Start use schema.Schema instead terraform.ResourceProvider

Today terraformer use terraform.ResourceProvider interface for get ProviderSchema with Attributes.
In other interface we can get more Attributes from providers(like deprecation options)
Need use schema.Schema from github.com/hashicorp/terraform/helper/schema

GCP: Add support Google Cloud Functions Resources

AWS tags with dot in name generate invalid TF

Tag named foo.bar with value baz generates

  tags {
    foo {
      bar = "baz"
    }
  }

googlecloudplatform / terraformer Goto Github PK

terraformer's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs