nbering / terraform-provider-ansible

"Logical" provider for integrating with an Ansible Dynamic Inventory script.

Ansible Terraform Provider

A Terraform provider serving as an interop layer for an Ansible dynamic inventory script.

Read the introductory blog post for an explanation of the design motivations behind this provider.

Installation

Installation can be accomplished in two different ways:

1. For Terraform 0.13+ Use terraform required_providers block
2. Installing a pre-compiled release (recommended)
3. Compiling from source

For Terraform 0.13+ Use terraform required_providers block

providers.tf

terraform {
  required_providers {
    ansible = {
      source = "nbering/ansible"
      version = "1.0.4"
    }
  }
}
provider "ansible" {}

then perform another init to download the provider.

$ terraform init
Initializing modules...
Initializing the backend...
Initializing provider plugins...
- Finding nbering/ansible versions matching "1.0.4"
...
- Installing nbering/ansible v1.0.4...
- Installed nbering/ansible v1.0.4 (self-signed, key ID xxxxxxxxxxxxxx)
...
Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

Installing a Pre-Compiled Release

Downloading and installing a pre-compiled terraform-provider-ansible release is the recommended method of installation since it requires no additional tools or libraries to be installed on your workstation.

1. Visit the releases page and download the latest release for your target architecture.

2. Unzip the downloaded file and copy the terraform-provider-ansible binary to a designated directory as described in Terraform's plugin installation instructions.

Compiling From Source

Note: Terraform requires Go 1.11 or later to successfully compile.

Note: Dependencies are no longer included in this repository. You may need the bazaar version control utility to download some of Terraform's Go-lang module dependencies.

If you'd like to take advantage of features not yet available in a pre-compiled release, you can compile terraform-provider-ansible from source.

In order to compile, you will need to have Go installed on your workstation. Official instructions on how to install Go can be found here.

Alternatively, you can use gimme as a quick and easy way to install Go:

$ sudo wget -O /usr/local/bin/gimme https://raw.githubusercontent.com/travis-ci/gimme/master/gimme
$ sudo chmod +x /usr/local/bin/gimme
$ gimme 1.10
# copy the output to your `.bashrc` and source `.bashrc`.

Once you have a working Go installation, you can compile terraform-provider-ansible by doing the following:

$ go get github.com/nbering/terraform-provider-ansible
$ cd $GOPATH/src/github.com/nbering/terraform-provider-ansible
$ make

You should now have a terraform-provider-ansible binary located at $GOPATH/bin/terraform-provider-ansible. Copy this binary to a designated directory as described in Terraform's plugin installation instructions

Terraform Configuration Example

resource "ansible_host" "example" {
    inventory_hostname = "example.com"
    groups = ["web"]
    vars = {
        ansible_user = "admin"
    }
}

resource "ansible_group" "web" {
  inventory_group_name = "web"
  children = ["foo", "bar", "baz"]
  vars = {
    foo = "bar"
    bar = 2
  }
}

Compatibility

Version 1.0.0 of this project is compatible with Terraform version 0.12-beta2. You will also need 2.0.0+ of the terraform-inventory script, as the internal structure of Terraform state files has changed.

If you need a version compatible with an earlier version of Terraform, use version 0.0.4.

When upgrading to Terraform 0.12.x, you may need to change your configuration files to reflect changes to the new version of the Hashicorp Configuration Lanaguage (HCL). The only known incompatibility is that vars attributes now require an equals operator (=).

0.11.x

resource "ansible_host" "example" {
    inventory_hostname = "example.com"
    vars {
        ansible_user = "admin"
    }
}

0.12.x

resource "ansible_host" "example" {
    inventory_hostname = "example.com"
    vars = {
        ansible_user = "admin"
    }
}

Alternatives and Similar Projects

jonmorehouse/terraform-provisioner-ansible

A Terraform Provisioner that runs Ansible-Local on a target machine at creation-time.

adammck/terraform-inventory

A very similar solution to this one, without the Logical provider. Depends on specific Terraform resource types, and relies heavily on cloud-providers' tag implementations.

Ansible Collection: Terraform

An Ansible collection that runs Terraform plans and deployments.

License

Contributions specific to this project are made available under the Mozilla Public License.

Code under the vendor/ directory is copyright of the various package owners, and made available under their own license considerations.

terraform-provider-ansible's Issues

Should this still be working?

After copying terraform-provider-ansible_v1.0.3 to .terraform/plugins/drawin_amd64, Terraform is still complaining about not being able to find the provider:

Provider "ansible" not available for installation.

A provider named "ansible" could not be found in the Terraform Registry.

This may result from mistyping the provider name, or the given provider may
be a third-party provider that cannot be installed automatically.

In the latter case, the plugin must be installed manually by locating and
downloading a suitable distribution package and placing the plugin's executable
file in the following directory:
    terraform.d/plugins/darwin_amd64

Terraform detects necessary plugins by inspecting the configuration and state.
To view the provider versions requested by each module, run
"terraform providers".


Error: no provider exists with the given name

Use Dynamic Terraform State During Terraform Execution

Is it possible to use this provider and the dynamic terraform.py inventory script during the execution of a 'terraform apply'? What I'd like to be able to do is some early provisioning of new instances during the execution of Terraform. I can do this now if I hard code a bootstrap inventory file with some matching host patterns. However, I'd like to make my Terraform more dynamic. I was trying to use this provisioner, but it looks like the Terraform state that it pulls does not include any changes made during the current execution. Is this just an inherent limit of Terraform?

For example, here's an Ansible host resource:

resource "ansible_host" "rke-cluster" {
  depends_on = [
    module.rke-cluster
  ]

  count = 1

  inventory_hostname = module.rke-cluster.tags[count.index]["Name"]

  groups = [
    "rke"
  ]

  vars = {
    ansible_host = module.rke-cluster.private_dns[count.index]
    ansible_ssh_common_args = "-o ProxyCommand=\"ssh -W %h:%p centos@${aws_eip.jumpbox-eip.public_ip}\""
  }
}

And later on, here is a local provisioner that attempts to get the dynamic inventory:

resource "null_resource" "bootstrap-rke-cluster" {
  depends_on = [
    ansible_host.rke-cluster
  ]

  count = 01

  provisioner "local-exec" {
    command = <<EOC
  ansible/terraform.py;
EOC
  }
}

But the output doesn't show the instance that was just added to the inventory (it should show a second host, but only shows a single host from a previous execution):

{
  "_meta": {
    "hostvars": {
      "jumpbox-00": {}
    }
  },
  "all": {
    "children": [],
    "hosts": [
      "jumpbox-00"
    ],
    "vars": {}
  },
  "jumpbox": {
    "children": [],
    "hosts": [
      "jumpbox-00"
    ],
    "vars": {}
  }
}

Terraform v0.12 compatibility

Terraform v0.12 cannot load terraform-provider-ansible and prints following message:

Error: failed to load provider "ansible": Incompatible API version with plugin. Plugin version: 4, Client versions: [5]

See-Also: https://www.terraform.io/upgrade-guides/0-12.html

Publish provider for use in Terraform 0.13

👋 Hi, I'm on the Terraform Providers team at HashiCorp. With the release of the Terraform 0.13 beta, users can now download and install community providers from the registry. We are inviting provider authors (especially those for popular community providers) to publish their providers in a closed beta.

To get invited to the closed beta, please email [email protected]. We need:

• A list of GitHub usernames to add to the beta (you and any team members who will publish)
• List of provider repositories you plan to publish
• A GPG public key in ASCII-armor format, which you will be using to sign your provider releases

You can use one key for all of your providers, or separate keys if you prefer. If you are publishing from an organization, this key or keys will be associated with that namespace. Once in the beta, you can manage personal keys in the UI as well.

Samples for Ansible

Hi,

I am looking for some samples where using terraform we will be calling Ansible playbooks and these playbooks use static as wells as dynamic inventory.

Please let me know if there is any such samples are available?

Publish macos arm64 build

Don't know if the author is still here, but in case - is it possible to publish macos arm64 build to terraform providers repository?
The more and more M1 mac are in use, and for me, it's the case.
No code changes are needed and I was able to build it without any customization of scripts or configs.
Currently, I'm using dev_overrides to make terraform use my local build, but that prevents me from using terraform init for affected configurations, as init still tries to download provider and fails, so it's all manual steps - like removing provider, init, readding provider, and it's not good for teamwork too.

Installation instructions are unclear and broken

The installation instructions are unclear,
for the sake of simplicity, please let people know that it suffices to copy the binary (in releases) to
the .terraform/plugins/*/ where * is the system where terraform is installed.

Moreover, the name of the release doesn't match the terraform naming standards for plugins.
instead of terraform-provider-ansible-v0.0.1
must be terraform-provider-ansible_v0.0.1
terraform recognize the name of the module after the "terraform-provider-" and delimits it with "_".
Without it, the module cannot be found and terraform command fails.

I'm starting with Terraform and spent quite some time figuring it out.
Thanks for the great work.

Problems upgrading to registry-sourced plugin

[Edited]
This summarizes the steps that resulted in a successful upgrade for me.
But I had missed this important step: If tf init complains that it can't find hashicorp/ansible, that means one of your modules is using the ansible plugin (say, by declaring an ansible_group resource) but does not have a required_provider statement! Could be your root module or any included module.

TLDR: This error means you need to add a required_providers block to all the modules which list hashicorp/ansible as a provider in the output of terraform providers.

[moved from #32]

When upgrading to Terraform 0.13.x, I'm having problems related to moving from the manually-installed hashicorp/ansible plugin to a registry-installed version:

First, I added the necessary required_providers block to my root module:

terraform {
  [...]
  required_providers {
    ansible = {
      source = "nbering/ansible"
      version = "1.0.4"
    }
    aws = {
      source = "hashicorp/aws"
    }
  }
}

Now when I run tf init --upgrade it still fails looking for the older name:

 $ tf init --upgrade
Upgrading modules...
- security_group in ../modules/app-tier/security-group
- swarm_master in ../modules/app-tier/instance
- vpc in ../modules/app-tier/vpc-remote

Initializing the backend...

Initializing provider plugins...
- Finding latest version of -/aws...
- Finding latest version of nbering/ansible...
- Finding latest version of hashicorp/aws...
- Finding latest version of hashicorp/ansible...
- Installing -/aws v3.5.0...
- Installed -/aws v3.5.0 (signed by HashiCorp)
- Installing nbering/ansible v1.0.4...
- Installed nbering/ansible v1.0.4 (self-signed, key ID 1A9FBF0FFD10BE33)
- Installing hashicorp/aws v3.5.0...
- Installed hashicorp/aws v3.5.0 (signed by HashiCorp)

Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://www.terraform.io/docs/plugins/signing.html

Error: Failed to install provider

Error while installing hashicorp/ansible: provider registry
registry.terraform.io does not have a provider named
registry.terraform.io/hashicorp/ansible

Even after replacing the provider in my project state:

 $ tf state replace-provider registry.terraform.io/-/ansible registry.terraform.io/nbering/ansible
Terraform will perform the following actions:

  ~ Updating provider:
    - registry.terraform.io/-/ansible
    + registry.terraform.io/nbering/ansible

Changing 3 resources:

  ansible_host.monitor
  module.swarm_master.ansible_group.swarm_nodes
  module.swarm_master.ansible_host.swarm_master

Do you want to make these changes?
Only 'yes' will be accepted to continue.

Enter a value: yes

Successfully replaced provider for 3 resources.

It feels like the state still contains the reference to hashicorp/ansible even though no resources have been moved to the new provider.
[EDIT] it's not the state that's holding the stale reference--it's strictly the code. Use terraform providers to identify the module with the dangling reference, and add a required_providers block to that module.

Submit this provider to Terraform Registry

Now that 0.13.0 is out Terrform supports Explicit Provider Source Locations, which means it can install providers available in the Terraform Registry.

It would make it simpler to use this provider if it was available there.

Run ansible as part of Terraform rollout

This is probably out of scope for this project but I wanted to explore this idea:

Could we add this feature to the provider:

resource "ansible_host" "example" {
    inventory_hostname = "example.com"
    groups = ["web"]
    vars = {
        ansible_user = "admin"
    }
}

resource "ansible_provisioning" "web" {
  hosts = [ansible_host.example]
  galaxy_roles = file("${path.module}/requirements.txt")
  playbook = file("${path.module}/web.yaml")
}

If one uses a reference to the cloud resource as input for ansible_host the ansible provisioning could be deferred until the resource is up.

Feel free to close if this is way out of scope :)

provider.ansible: Incompatible API version with plugin. Plugin version: 1, Ours: 4

On MacOS:

1. terraform init => fine
2. terraform apply

Error: Error asking for user input: 1 error(s) occurred:

• provider.ansible: Incompatible API version with plugin. Plugin version: 1, Ours: 4

3. terraform --version
   Terraform v0.11.2

• provider.ansible (unversioned)
• provider.aws v1.7.0

Feature: Ansible groups and group vars.

I have plans to add another entity to cover group vars, similar to the ansible_host resource. I haven't a need for it yet, but someone might.

ansible_host vars dictionary can't handle arrays

This one's an interesting edge case.

I wanted to pass an array of hostnames to a variable...

resource "ansible_host" "example" {
  inventory_hostname = "example.com"
  vars {
    certificate_domains = [ 
      "${dnsimple_record.domain1.hostname}",
      "${dnsimple_record.domain2.hostname}"
    ]
  }
}

This received an error like this:

Error: ansible_host.example: vars (certificate_domains): '' expected type 'string', got unconvertible type '[]interface {}'

This might be one of those cases where the intersection of the Go and Terraform type systems are not going to do what I want.

I'm thinking about adding a resource along the lines of what I proposed in #4, but specifically to handle the array case. Maybe something like:

# This is not a resource that exists now, just an example of what could be added.
resource "ansible_host_array_var" "example" {
    inventory_hostname = "example.com"
    key                = "certificate_domains"
    value              = ["${dnsimple_record.domain1.hostname}", "${dnsimple_record.domain2.hostname}"]
}

Build a release for Terraform 0.12.0

🤦‍♂

I did the release build for v1.0.2 against Terraform 0.12.0-rc1 because I didn't realize the official release was published this past week. I must have crawled under a rock for most of the week or something. 😄

Handling the "count" parameter in Terraform

Hi,
This might be a question more for an example, than an actual issue. Let's say I'm setting up 5 load-balanced servers:

resource "aws_instance" "web" {
  ami           = "ami-408c7f28"
  instance_type = "t1.micro"
  count = "5"
}

How would I pass the 5 servers to Ansible using this? Is it even possible?

resource "ansible_host" "example" {
    inventory_hostname = "example.com"
    groups = ["web"]
    vars {
        ansible_user = "admin"
    }
}

Thanks in advance,
Andy

Document ansible_host_var and ansible_group_var resources.

Update vendor dependencies.

The vendor/ dependencies are getting a little stale at 3-months old and counting. The specific versions were originally taken from one of the official Terraform Providers, and installed with govendor. These should be updated, and a process should be documented for updating these files.

Publish docs via registry

Reformat the documentation to play nice with the Terraform registry.

https://www.terraform.io/docs/registry/providers/docs.html

Update documentation to reflect Terraform 0.13 registry-based install

provider.ansible: Unrecognized remote plugin message: 2|unix|/tmp/tf-plugin164979924

Hi,

I'm facing the following error message:

eimert@EIM terraform-gce $ terraform plan

Error: Error asking for user input: 1 error(s) occurred:

* provider.ansible: Unrecognized remote plugin message: 2|unix|/tmp/tf-plugin164979924

This usually means that the plugin is either invalid or simply
needs to be recompiled to support the latest protocol.

provider is installed:

eimert@EIM terraform-gce $ terraform providers
.
├── provider.ansible
├── provider.google
├── module.google-dns-managed-zone
│   └── provider.google (inherited)
└── module.powerzone
    └── provider.google (inherited)

Using terrafom ansible plugin release v0.0.4:

-rwxr-xr-x 1 eimert eimert 26528047 apr 20  2018 /home/eimert/terraform.d/plugins/terraform-provider-ansible_v0.0.4

And the latest terraform (atm):

eimert@EIM terraform-gce $ terraform --version
Terraform v0.11.9
+ provider.ansible (unversioned)
+ provider.google v1.19.1

Then I compiled the master branch, still the same error message.

-rwxr-xr-x 1 eimert eimert 24407556 okt 21 11:18 /home/eimert/terraform.d/plugins/terraform-provider-ansible

/home/eimert/terraform.d/plugins/
└── terraform-provider-ansible

The main.tf:

provider "google" {
  credentials = "${file("king-of-my-google-cloud-castle.json")}"
  project     = "smashing-dash-1992"
}

module "google-dns-managed-zone" {
  # source          = "github.com/Eimert/terraform-google-dns-managed-zone"
  source          = "./modules/terraform-google-dns-managed-zone"

  dns_name        = "clouder-zone"
  dns_zone        = "clouder.eimertvink.nl."
}

module "powerzone" {
  # source          = "git::https://github.com/Eimert/terraform-google-compute-engine-instance?ref=ansible-provisioner"
  source          = "./modules/terraform-google-compute-engine-instance"

  amount          = 1
  region          = "europe-west4"
  zone            = "europe-west4-c"
  name_prefix     = "wild-gixxer"
  machine_type    = "custom-2-4096"
  disk_type       = "pd-ssd"
  disk_size       = "15"
  disk_image       = "centos-cloud/centos-7"

  dns_name        = "${module.google-dns-managed-zone.dns_name}"
  dns_zone        = "${module.google-dns-managed-zone.dns_zone}"
  dns_record_name = "pww-dev"

  user_data       = "firestone-lab"
  username        = "eimert"
  public_key_path = "~/.ssh/id_rsa.pub"
}

resource "ansible_host" "example" {
  inventory_hostname = "${module.powerzone.addresses}"
  playbook = "play.yml"
  vars {
    ansible_user = "${module.powerzone.username}"
  }
}

AFAIK the /tmp file is not readable:

eimert@EIM terraform-gce $ less /tmp/tf-plugin394044065
/tmp/tf-plugin394044065 is not a regular file (use -f to see it)
eimert@EIM terraform-gce $ less -f /tmp/tf-plugin394044065
/tmp/tf-plugin394044065: Apparaat of adres bestaat niet
eimert@EIM terraform-gce $ stat /tmp/tf-plugin394044065
  Bestand: '/tmp/tf-plugin394044065'
  Grootte: 0            Blokken: 0            IO-blok: 4096   socket
Apparaat: fc01h/64513d   Inode: 18492238     Koppelingen: 1
Toegang: (0755/srwxr-xr-x)   UID: ( 1000/  eimert)   GID: ( 1000/  eimert)
Toegang:   2018-10-21 11:23:43.958402135 +0200
Gewijzigd: 2018-10-21 11:23:43.958402135 +0200
Veranderd: 2018-10-21 11:23:43.958402135 +0200
Ontstaan:  -

Text is partly in Dutch, apologies!

Any thoughts on getting the ansible provisioner to work? I can provide more details if necessary.

Release script creates binary by invalid name.

The plugin filename convention was inaccurately documented by Terraform, so the "helpful" file name I used for the release binary filenames is incorrect. The dash(-) between the plugin name and the version number should be an underscore(_).

hashicorp/terraform#15705

Compile as static binary?

Hi

would it be possible to make the binary available as a static binary? This will make it possible to run the provider in alpine based docker images (i.e. https://hub.docker.com/r/hashicorp/terraform).

Thanks in advance,

Rudy

Add an install script.

I experimented a while ago with making a little shell script that could be used to download, unpack, and install the release binaries.

I was looking at something like NodeJS's one or two-liner installation process, which looks like this:

curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
sudo apt-get install -y nodejs

I experimented with this a while ago, and found the most difficult part was detecting the right OS and architecture. Every POSIX system is a little bit different about how it presents these from the various system information commands.

Here's what I came up with in that experiment (running on Ubuntu):

#!/usr/bin/env bash

TEMPFILE=`mktemp`
echo "TEMP: $TEMPFILE"

wget https://github.com/nbering/terraform-provider-ansible/releases/download/v0.0.2/terraform-provider-ansible-linux_amd64.zip -O $TEMPFILE
mkdir -p ~/.terraform.d/plugins
unzip -d ~/.terraform.d/plugins $TEMPFILE
rm $TEMPFILE

Creating the host list from modules

I have a bunch of EC2 instances created via a module. I can get the list of internal IP addresses of these instances. How do I declare this in the ansible_host resource?

Feature: Separate host and group variables resources.

Would solve problems with:

• setting variables by dynamic maps or lists
• setting variables where count= may be required
• setting a variable in the root module, to attach to a host declared in a submodule

# This is not a resource that exists now, just an example of what could be added.
resource "ansible_host_var" "example" {
    inventory_hostname = "db.example.com"
    key                = "db_password"
    value              = "testing1234"
}

Would require changes to the dynamic inventory script as well, because it would have to understand the resource type, and somehow merge the ansible_host.vars and ansible_host_var values.