ned1313 / deep-dive-terraform

Exercise files for my Pluralsight course.

Deep-Dive-Terraform

Welcome to Terraform - Deep Dive version 3. These exercise files are meant to accompany my course on Pluralsight. The course was developed using version 1.5.x of Terraform.

If you're looking for the older versions of the course, that is still available on the v1 and v2 branches. I am no longer maintaining them, but I thought I would keep them around for posterity or if you're already working through those versions of the course.

AWS Account

You are going to need an account where you have FullAdmin permissions. You are going to be creating policies, roles, profiles, VPCs, etc. If you don't have enough permissions in your current environment, then I recommend creating a temporary account to mess around in. In fact, probably do that regardless. You don't want to accidentally mess something up at work because you were trying to learn about Terraform.

You may exceed your EIP address quota when deploying multiple enviornments. You can request an increase through the AWS console in the Services Quotas area, under the Amazon Elastic Compute Cloud category. I recommend setting it to 15 just to be safe. It should be approved almost immediately, but may take 30 minutes to apply. So if you do it now, it should be ready long before you get to that portion of the course.

Using the files

Each folder represents a module from the course and they all build off of each other. You will be creating two working directories in your local copy of the exercise files: network_config and application_config. Both of these folders are part of the .gitignore so they will not be committed back to the repository.

In the module folders, I have included Terraform configurations to help set up the necessary prerequisites. I've also included the commands you should run if you're following along. You don't have to run them verbatim, and I encourage you to mess around and try out different flags and commands. If you run into an issue, please submit it as such and I will do my best to remediate it.

Line Endings

An issue I have discovered from time to time is that Terraform doesn't much like the Windows style of ending a line with both a Carriage Return (CR) and a Line Feed (LF), commonly referred to as CRLF. If you are experiencing strange parsing issues, change the line ending to be Line Feed (LF) only. In VS Code this can be down by clicking on the CRLF in the lower right corner and changing it to LF.

MONEY!!!

A gentle reminder about cost. The course will have you creating resources in AWS. Some of the resources are not going to be 100% free. In most cases I have tried to use the Free-tier when possible, but AWS is about to start charging for IPv4 addresses even when they're attached to a resource. I'm not sure how this will impact the free tier, so just be cognizant of what you're creating.

When you complete an exercise in the course, you can always run terraform destroy and approve the destruction to remove all resources from AWS. Once you get to the application deployment, you will need to leave the networking in place for a successful application deployment. If you wish to tear things down, destroy the application first and then the network.

Terraform Cloud and GitHub Actions

Previous versions of this course used Docker, Consul, and Jenkins to provide remote state and CI/CD operations. This ended up being a sticking point for several people, and they ended up abandoning the course. To simplify things, I have replaced those technologies with Terraform Cloud and GitHub Actions. This is not an overt endorsement of either technology, there are plenty of other great options when it comes to remote state storage and CI/CD for IaC. I chose GitHub Actions because we are already using GitHub any way, and I chose Terraform Cloud because it has a solid free tier and it is part of the Terraform Associate certification. Speaking of which...

Certification

HashiCorp has released the Terraform Certified Associate certification. You might be wondering if this course fully prepares you for the cert. It does not. Taking this course along with the Terraform - Getting Started course on Pluralsight will meet all of the learning objectives for the certification, but there is no substitute for running the software on your own and hacking away.

I have coauthored a certification guide with Adin Ermie which you can find on Leanpub. This is an unofficial guide, but I believe in concert with the Pluralsight courses you will be in a good position to sit the exam.

Conclusion

I hope you enjoy taking this course as much as I did creating it. I'd love to hear feedback and suggestions for revisions.

Thanks and happy automating!

Ned

deep-dive-terraform's Issues

Unsupported attribute at m7 tfe_outputs commit plan

https://github.com/timakamystery/globo-webapp/blob/tfe-outputs/resources.tf

│ Error: Unsupported attribute │ │ on security_groups.tf line 23, in resource "aws_security_group" "webapp_http_inbound_sg": │ 23: vpc_id = data.tfe_outputs.networking.nonsensitive_values.vpc_id │ ├──────────────── │ │ data.tfe_outputs.networking.nonsensitive_values is object with no attributes │ │ This object does not have an attribute named "vpc_id".

tfe = { source = "hashicorp/tfe" version = "= 0.47.0" }

`diff --git a/datasources.tf b/datasources.tf
index bafe7b0..888c009 100644
--- a/datasources.tf
+++ b/datasources.tf
@@ -4,4 +4,9 @@

data "aws_ssm_parameter" "amzn2_linux" {
name = "/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2"
+}
+
+data "tfe_outputs" "networking" {

• organization = var.tfe_organization
• workspace = var.tfe_workspace_name
  }
  \ No newline at end of file
  diff --git a/resources.tf b/resources.tf
  index f3ab76d..5272bf0 100644
  --- a/resources.tf
  +++ b/resources.tf
  @@ -26,10 +26,10 @@ locals {
  ##################################################################################

resource "aws_instance" "main" {

• count = length(var.public_subnets)

• count = length(data.tfe_outputs.networking.nonsensitive_values.public_subnets)
  ami = nonsensitive(data.aws_ssm_parameter.amzn2_linux.value)
  instance_type = var.instance_type

• subnet_id = var.public_subnets[count.index]

• subnet_id = data.tfe_outputs.networking.nonsensitive_values.public_subnets[count.index]
  vpc_security_group_ids = [
  aws_security_group.webapp_http_inbound_sg.id,
  aws_security_group.webapp_ssh_inbound_sg.id,
  @@ -97,7 +97,7 @@ resource "aws_lb" "main" {
  internal = false
  load_balancer_type = "application"
  security_groups = [aws_security_group.webapp_http_inbound_sg.id]

• subnets = var.public_subnets

• subnets = data.tfe_outputs.networking.nonsensitive_values.public_subnets

  enable_deletion_protection = false

@@ -120,7 +120,7 @@ resource "aws_lb_target_group" "main" {
port = 80
target_type = "instance"
protocol = "HTTP"

• vpc_id = var.vpc_id

• vpc_id = data.tfe_outputs.networking.nonsensitive_values.vpc_id
  }

resource "aws_alb_target_group_attachment" "main" {
diff --git a/security_groups.tf b/security_groups.tf
index 9419606..9459c41 100644
--- a/security_groups.tf
+++ b/security_groups.tf
@@ -20,7 +20,7 @@ resource "aws_security_group" "webapp_http_inbound_sg" {
cidr_blocks = ["0.0.0.0/0"]
}

• vpc_id = var.vpc_id

• vpc_id = data.tfe_outputs.networking.nonsensitive_values.vpc_id

  tags = local.common_tags
  }
  @@ -36,7 +36,7 @@ resource "aws_security_group" "webapp_ssh_inbound_sg" {
  cidr_blocks = [var.ip_range]
  }

• vpc_id = var.vpc_id

• vpc_id = data.tfe_outputs.networking.nonsensitive_values.vpc_id

  tags = local.common_tags
  }
  @@ -52,7 +52,7 @@ resource "aws_security_group" "webapp_outbound_sg" {
  cidr_blocks = ["0.0.0.0/0"]
  }

• vpc_id = var.vpc_id

• vpc_id = data.tfe_outputs.networking.nonsensitive_values.vpc_id

  tags = local.common_tags
  }
  \ No newline at end of file
  diff --git a/terraform.tf b/terraform.tf
  index 0b2278d..ac83b91 100644
  --- a/terraform.tf
  +++ b/terraform.tf
  @@ -4,5 +4,9 @@ terraform {
  source = "hashicorp/aws"
  version = "~>5.0"
  }

• tfe = {

•  source  = "hashicorp/tfe"
  

•  version = "~>0.49.2"
  

• }
  }
  }
  \ No newline at end of file
  diff --git a/variables.tf b/variables.tf
  index b1ef070..1f042b4 100644
  --- a/variables.tf
  +++ b/variables.tf
  @@ -37,14 +37,12 @@ variable "api_key" {
  description = "(Required) API key for web app to talk to SaaS platform."
  }

-variable "public_subnets" {

• type = list(string)
• description = "(Required) List of subnet IDs for EC2 instance deployments."
  +variable "tfe_organization" {

• type = string
• description = "(Required) TFE Organization to use for remote state."
  }

-variable "vpc_id" {
+variable "tfe_workspace_name" {
type = string

• description = "(Required) VPC ID of VPC for application deployment."

• description = "(Required) TFE Workspace Name to use for remote state."
  }`

when doing terraform plan data.tfe_outputs.networking.nonsensitive_values.vpc_id basically is not found. comparing w/the complete code in m9 - it has no diff. also tried. init, then plan from there, - same error.
tried both latest 0.49.2 and 0.47.0 of tfe provider, no diff

M3 Error deleting VPC

Hi Ned,

Im going through model 3 and I have countered 2 issues:

1. for running this line ./junior_admin.sh says permission denied and then continued without it, now wanted to destroy but it doesn't destroy it all
2. Error deleting VPC: DependencyViolation: The vpc 'vpc-04454c66fe0afd6c2' has dependencies and cannot be deleted.
   status code: 400, request id: 944c3d7a-520d-4b32-97fc-2103712a1444

Copy command in m7_commands.txt need a -r flag

putting in a PR

m6 jsondecode issue

Error: Error in function call

on resources.tf line 56, in locals:
56: subnet_count = jsondecode(data.consul_keys.networking.var.networking)["subnet_count"]
|----------------
| data.consul_keys.networking.var.networking is ""

Call to function "jsondecode" failed: EOF.

I have exactly the same code you have, not sure why this is happening

~/git/terraform-deep-dive-by-Ned-Bellavance/m6/networking
❯ terraform workspace list
default

• development

tf.workspace
terraforom.workspace
Error: Reference to undeclared resource

on line 1:
(source code not available)

Deep-Dive-Terraform:Readme update

According to Pluralsight, the following course has been retired:
https://app.pluralsight.com/library/courses/terraform-getting-started

Your readme doc refers to this course being needed for the preparation of the Terraform certificate.

Terraform plan on m3 exercise gives lot of errors

When running terraform plan on m3 exercise I get lot of errors:

Error: Cannot import to non-existent resource address
│
│ Importing to resource address module.main.aws_subnet.public[0] is not possible, because that address does not exist in configuration. Please ensure that the resource key is correct, or remove this import block.
╵
╷
│ Error: Cannot import to non-existent resource address
│
│ Importing to resource address module.main.aws_subnet.public[1] is not possible, because that address does not exist in configuration. Please ensure that the resource key is correct, or remove this import block.
╵
╷
│ Error: Cannot import to non-existent resource address
│
│ Importing to resource address module.main.aws_internet_gateway.this[0] is not possible, because that address does not exist in configuration. Please ensure that the resource key is correct, or remove this import block.
╵
╷
│ Error: Cannot import to non-existent resource address
│
│ Importing to resource address module.main.aws_route.public_internet_gateway[0] is not possible, because that address does not exist in configuration. Please ensure that the resource key is correct, or remove this import block.
╵
╷
│ Error: Cannot import to non-existent resource address
│
│ Importing to resource address module.main.aws_route_table.public[0] is not possible, because that address does not exist in configuration. Please ensure that the resource key is correct, or remove this import block.
╵
╷
│ Error: Cannot import to non-existent resource address
│
│ Importing to resource address module.main.aws_route_table_association.public[0] is not possible, because that address does not exist in configuration. Please ensure that the resource key is correct, or remove this import block.
╵
╷
│ Error: Cannot import to non-existent resource address
│
│ Importing to resource address module.main.aws_route_table_association.public[1] is not possible, because that address does not exist in configuration. Please ensure that the resource key is correct, or remove this import block.
╵

AWS Account ID

FYI - The AWS account ID can be enumerated automatically, provided that the user/role employed by terraform has read permissions for sts:GetCallerIdentity.

data "aws_caller_identity" "current" {}

output "account_id" {
  value = "${data.aws_caller_identity.current.account_id}"
}

terraform state show on a child module isn't working

The terraform state show ADDR isn't working on a child module:
Example:

terraform state show module.main.aws_vpc.this[0]

Gives the bellow result:

no matches found: module.main.aws_vpc.this[0]

Is there a way to expose all VPC terraform instance information from root module ?

Module 6 - common_tags merge order incorrect?

In Module 6 "Workspaces and Collaboration", the locals block in resources.tf for common_tags merges the tags in Consul with the terraform.workspace value.

common_tags = merge({
        Environment = terraform.workspace
      },
      jsondecode(data.consul_keys.networking.var.common_tags)
    )

However, Terraform's merge function favors values later in the argument sequence if maps contain the same key. Since the Consul data is listed last here, the "Environment" tag always contains the Consul value of "Production". Should resources.tf be changed so that terraform.workspace is the last argument to the merge function?

Module 3 Issues

Spotted a few of issues when running through module 3;

1. backend.tf file included in folder for m3, but not shown in the Pluralsight video. Running terraform init with this file asks for a path to be provided for Consul. Presume this file has crept in from a later module, so I propose it is deleted.
2. jq needs to be installed to run junior_admin.sh. Suggest this pre-requisite is called out.
3. ImportCommands.txt in this repository includes id values, whereas these are not included in the course video. As these values will always be unique, should they be removed from the commands file?
4. When setting up the import commands, the video states the Private Route Table Association ID is not included in the output of junior_admin.sh, however this was shown in the output for me. Is this a discrepancy between the junior_admin.sh and the ps1 version? Or an updated version of the script which doesn't quite match the video?
5. Import commands fail on Mac using ZSH, as the ADDR needs to be encapsulated in "", otherwise zsh identifies the [] in the address as a file path (ref: ohmyzsh/ohmyzsh#7894). e.g. "module.vpc.aws_subnet.public[2]".

jq would be fail in m1-lecture

I got parsing error in jq from my terminal.
error msg : "parse error: Invalid numeric literal at line 1, column 2"

Find the root case and create pr - #22

module 2

Hello,

can you please explain the line ' terraform init --var-file="..\..\terraform.tfvars" ' ?
You do this in module 2 without explaining it (unless I missed it). It sounds like we need to provide this, but why at init? At the plan stage a little later, you mention it is included in the module, but it's not. I'm confused.

missing "test" dir in M7 "Creating a Terraform Panic"

The "Creating a Terraform Panic" video in "M7 - Troubleshooting Terraform" refers to the Deep-Dive-Terraform/m7/panic/test directory. The directory is missing from the repo, though.
A minor glitch, as the video shows the debug process extensively anyway, but I thought it was worth reporting!

template provider has been archived

On at least Mac with M1 processor, starting in m5 (and then in m6 and further) the use of the template provider in the templates.tf file causes an init error:

➜  networking git:(v2) ✗ terraform init -backend-config="path=networking/state/globo-primary"
Initializing modules...
Downloading registry.terraform.io/terraform-aws-modules/vpc/aws 2.78.0 for vpc...
- vpc in .terraform/modules/vpc

Initializing the backend...

Successfully configured the backend "consul"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Finding hashicorp/consul versions matching "~> 2.0"...
- Finding hashicorp/aws versions matching ">= 2.70.0, ~> 3.0"...
- Finding latest version of hashicorp/template...
- Installing hashicorp/aws v3.75.2...
- Installed hashicorp/aws v3.75.2 (signed by HashiCorp)
- Installing hashicorp/consul v2.17.0...
- Installed hashicorp/consul v2.17.0 (signed by HashiCorp)
╷
│ Error: Incompatible provider version
│ 
│ Provider registry.terraform.io/hashicorp/template v2.2.0 does not have a package available for your
│ current platform, darwin_arm64.
│ 
│ Provider releases are separate from Terraform CLI releases, so not all providers are available for all
│ platforms. Other versions of this provider may have different platforms supported.

I'm using a Macbook Pro:

➜  Deep-Dive-Terraform git:(v2) ✗ terraform version
Terraform v1.3.6
on darwin_arm64

The docs say to use templatefile, but not sure how to do that in the current module setup. I'm doing a workaround for now by renaming the templates.tf file to xxxtemplates.tfxxx (taking it out of the config, basically) and adding two lines to locals in resources.tf and changing two lines in the vpc module to use the two local vars:

locals {
  cidr_block   = jsondecode(data.consul_keys.networking.var.networking)["cidr_block"]
  subnet_count = jsondecode(data.consul_keys.networking.var.networking)["subnet_count"]
  public_cidrsubnets = [for subnet in range(local.subnet_count) : cidrsubnet(local.cidr_block, 8, subnet)]
  private_cidrsubnets = [for subnet in range(local.subnet_count) : cidrsubnet(local.cidr_block, 8, subnet +10)]

  common_tags = merge(jsondecode(data.consul_keys.networking.var.common_tags),
    {
      Environment = terraform.workspace
    }
  )
}

##################################################################################
# RESOURCES
##################################################################################

# NETWORKING #
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~>2.0"

  name = "globo-primary-${terraform.workspace}"

  cidr            = local.cidr_block
  azs             = slice(data.aws_availability_zones.available.names, 0, local.subnet_count)
  private_subnets = local.private_cidrsubnets
  public_subnets  = local.public_cidrsubnets

  enable_nat_gateway = true

  create_database_subnet_group = false

  tags = local.common_tags
}

m5/networking/resources.tf vpc nat bug

The m5/networking/resources.tf vpc resource has enable_nat_gateway = true causing new NAT objects to be created when you run the terraform plan -out config.tfplan command.

Deep-Dive-Terraform/m5/networking/resources.tf

Line 72 in cbb63f5

enable_nat_gateway = true

The same issue is in m5/networking2/resources.tf

Deep-Dive-Terraform/m5/networking2/resources.tf

Line 76 in cbb63f5

enable_nat_gateway = true