sangeetprasad / aws_datalake

Big data analysis using a data lake.

AWS_DataLake

Big data lake analysis using AWS infrastructure.

Cloud - ☁️

IaaC

AWS Data lake Diagram

Purpose

Shows how to use AWS with Terraform to accomplish the following tasks:

• How to build an AWS data lake with Terraform

Prerequisites

• You must have an AWS account, and have your default credentials and AWS Region configured
• You must have Terraform installed

Cautions

• As an AWS best practice, grant this code least privilege, or only the permissions required to perform a task. For more information, see Grant Least Privilege in the AWS Identity and Access Management User Guide.
• This code has been tested in us-west-1 AWS Regions only. However it should work in any other region.
• Running this code it will result in charges to your AWS account.

• Only for testing and not production tls_private_key: Important Security Notice

The private key generated by this resource will be stored unencrypted in your Terraform state file. Use of this resource for production deployments is not recommended. Instead, generate a private key file outside of Terraform and distribute it securely to the system where Terraform will be run.

How to run this code

• How to run this code
  • Quick Start
  • Quick Destroy
  • Notes

Quick Start

It will deploy to the account setup for the aws cli called 'default'

cd AWS_DataLake
terraform init
terraform validate
terraform apply --auto-approve

Quick Destroy

terraform destroy --auto-approve

Notes

• clone this Git repo
• cd to the the directory AWS_DataLake
• use terraform init command to initialize the Terraform working directory and to download plugins for a provider (aws)
• terraform validate command will validate the code before actual deployment
• terraform plan command will show the execution code before making the actual deployment
• terraform apply
• Alternate command : terraform apply -auto-approve
• terraform destroy
• Alternate command : terraform destroy -auto-approve
• terraform fmt -recursive # A way to format the terraform code