yugabyte / aws-cloudformation

YugabyteDB AWS Cloud Formation Template

AWS CloudFormation

This repo contains an AWS CloudFormation template to deploy YugabyteDB cluster on AWS. It does the following:

• Creates a VPC with three public subnets
• Creates an instance in each subnet
  • Note that the instances that get created use Amazon Linux 1 as the OS.
• Deploys a YugabyteDB cluster across these three nodes

Pre-Flight Checks

• Make sure that the selected region has three or more AZs.
  • This allows this template creates 3 public subnets in three different availability zone.
• Make sure your AZ supports the instance type specified.
• As of now, the following regions are supported by this template:
  • EU (Ireland)
  • EU (London)
  • EU (Paris)
  • EU (Frankfurt)
  • Asia Pacific (Tokyo)
  • Asia Pacific (Sydney)
  • US East (N. Virginia)
  • Asia Pacific (Singapore)
  • Asia Pacific (Mumbai)
  • US West (Oregon)
  • US East (Ohio)

Usage

Deploying From AWS Command Line

• Clone this repo.

  $ git clone https://github.com/yugabyte/aws-cloudformation.git 
  

• Change current directory to cloned git repo directory
• Use aws cli to create cloudformation template
  

  aws cloudformation create-stack                                             \
          --stack-name <your-stack-name>                                      \
          --template-body file://yugabyte_cloudformation.yaml                 \
          --parameters ParameterKey=DBVersion,ParameterValue=2024.1.0.0-b175    \
                       ParameterKey=KeyName,ParameterValue=<you-ssh-key-name>
  

• Wait until the creation of all resources is complete.
• Once the cloudformation stack creation is complete, you can describe it as shown below.

  $ aws cloudformation describe-stacks --stack-name <your-stack-name>
  

  In output you will get the VPC id and YugabyteDB admin URL.

Deploying From AWS console

• Clone this repo.

  $ git clone https://github.com/yugabyte/aws-cloudformation.git 
  

• Login to aws console and navigate to CloudFormation service dashboard.
• Click on create stack button.
• Select Template is ready in prepare template section.
• Select Upload a template file in specify template section.
• Click choose file button in specify template section and upload the yugabyte_cloudforamtion.yaml file. Click on the next button.
• Specify your stack name and parameters for the stack. Click on next.
• Add a tag to your stack and choose IAM role if required. Click on next.
• Review the CloudFormation stack. If everything looks good, click on create stack button.
• Once stack creation gets compleated, you can access the YugabyteDB admin from URL you get in the stack output section.

aws-cloudformation's Issues

aws cloudformation command is broken

Syntax of the aws cloudformation is incorrect. When copied using the button, it includes a $ symbol, doesn't escape the new-lines, and has a syntactically-invalid comma separating the DB and SSHkey parameters.

Update ulimits for root user processes with CloudFormation template

Current install_software.sh sets ulimits for ec2-user and needs to be updated for root and persisted for reboots.

Need way to parameterize number of nodes created

We can parameterize the replication factor but we can't parameterize the number of nodes because in CloudFormation template we can not specify the number of EC2 instances dynamically.

Version is out of date

The release version of yugabyte-db is out of date and results in an undeployed cluster.

Missing referenced file in current cloudformation config

I followed these instructions in your docs: https://docs.yugabyte.com/latest/deploy/public-clouds/aws/ but my deploy kept failing and rolling back.

I noticed that in the cloudformation yaml https://raw.githubusercontent.com/yugabyte/aws-cloudformation/master/yugabyte_cloudformation.yaml there is a line: https://raw.githubusercontent.com/YugaByte/aws-cloudformation/master/scripts/init_ysql.sh which no longer exists in the repo. I tried removing that line and the instances do boot up, but there is no admin UI at port 7000 (might be unrelated)

CI/CD for YB CloudFormation

A CI/CD pipeline which deploys the YB stack using CloudFormation in AWS and verifies with some basic tests

Cloudformation Template Fails to Deploy YugbayteDB

Default version the template is incorrect (unpublished) 2.15.2.0-b175. The latest available version is 2.15.2.0-b87. This causes the download failure in install_software.sh.

Migrate to use common utilities repo instead of self contained scripts

Migrate aws cloudformation to use https://github.com/YugaByte/utilities instead of using self contained scripts.