influxdata / amazon-cloud-formation-influxdb-enterprise

AWS Cloud Formation for InfluxDB Enterprise.

[NOTICE] This repo is archived.

The cooresponding AWS Marketplace offer that used these templates has been deprecated. This repo is being archived to reflect the lack of active development and support for these templates.

InfluxDB Enterprise AWS Cloud Formation Templates

This repository contains templates to build AMIs for InfluxDB Enterprise as well as CloudFormation templates to deploy and configure an InfluxDB Enterprise cluster. These are also the source AMI and CloudFormation templates used in InfluxData's InfluxDB Enterprise product listed on the AWS Marketplace.

Getting Started

The assets in this repo can be used on their own or through the AWS Marketplace. The benefit of using AWS Marketplace is that the InfluxDB Enterprise license is billed through AWS. The templates in this repo can also be used directly outside the AWS Marketplace. However, these templates and images are not required to run InfluxDB Enterprise on AWS Marketplace. See the Installation guide to learn how to install InfluxDB Enterprise without using the assests in this repo.

AWS Marketplace

To deploy InfluxDB Enterprise through AWS Marketplace, subscribe to the InfluxDB Enterprise offer on AWS Marketplace](https://aws.amazon.com/marketplace/pp/InfluxData-Inc-InfluxDB-Enterprise/prodview-m6excs2fnnq5c). At the end of the subscription flow, you will be provided access to several AMIs for InfluxDB Enterprise and the option to deploy them as an InfluxDB Enterprise cluster using AWS CloudFormation. When using this method, an InfluxDB Enterprise license key is not needed and the license costs for usage are automatically billed through AWS Marketplace. Please reach out to [email protected] for more details.

Using the templates in this repo

The templates in this repo can be used to deploy an InfluxDB Enterprise cluster without using the AWS Marketplace. Note that this may not be the most flexible deployment pattern for upgrades.

Prerequisites

• Install Hashicorp's Packer image build tool.
• Install AWS' CLI tool.
  • Configure the AWS CLI with your account credentials, e.g. aws configure.

Build the AMI images

The first step is to build AMI images to use in the Cloud Formation templates.

Note: InfluxData does not maintain AMIs outside the AWS Marketplace at this time. Unless you feel comfortable building your own images, we recommend subscribing to the InfluxDB Enterprise offer on AWS Marketplace or following the InfluxDB Enterprise installation guide.

Install Hashicorp's Packer image build tool. Configure Packer with AWS credentials. Then, run the following commands to build the images:

cd packer
packer build influxdb.json

Record the AMI IDs created by Packer.

Note: The InfluxDB Enterprise images are based on the Amazon Linux 2 AMI.

Deploy the Cloud Formation template

First, install and configure the AWS CLI with your account credentials.

Replace AMI IDs in the Cloud Formation template in this repo with the AMI IDs generated above or subscribe to the InfluxDB Enterprise offer on AWS Marketplace.

Next, run either the deploy-multi-az.sh and deploy-single-az.sh script to create a cluster. When using self-built AMIs, license key must be passed in as an environment variable.

export LICENSE_KEY=a1b2c3d4-xxxx-xxxx-xxxx-xxxxxxxxxxxx
./deploy-multi-az.sh

amazon-cloud-formation-influxdb-enterprise's Issues

Set up CI for AWS assets

Set up continuous integration via CircleCI, Jenkins, or AWS CloudDeploy.

On each commit to master, CI should:

• Validate Packer templates.
• Validate CF templates.

For each new tag, CI should:

• Build new images for release to the AWS Marketplace.
• Deploy a new cluster via CloudFormation with the new images.
• Run a few smoke tests.
• Delete the cluster and clean up artifacts.

Programmatically generate templates

As we add additional options and multiple data node configurations (2, 4, 8 data nodes), we will need to start generating the JSON for the CloudFormation templates.

There are several projects out there which all programmatic generation of CloudFormation templates. This Golang template builder (published by AWS) or Troposphere python lib (the largest community project) look like promising tools to script the templates.

Add support for cfn-hup

To support online configuration updates through the CF templates requires adding a cfn-hup listener to execute cfn-init tasks.

Add option to specify certificate ARN

Add optional Lambda task to take volume snapshots

An AWS Lambda function that takes snapshots of the data and meta node volumes would be a nice option to add to the templates.

It could even be a separate stack that consumes the output of the main template.

Typo in README description

See text below:

a license key much be passed in as an environment variable.

Page links returning 404 error

This following page links are returning 404 error(s):

• InfluxData's InfluxDB Enterprise product [https://github.com/influxdata/amazon-cloud-formation-influxdb-enterprise/blob/master]
• documentation [ https://github.com/influxdata/amazon-cloud-formation-influxdb-enterprise/blob/master]

Review templates follow all security recommendations

InfluxDB Enterprise security recommendations. We should make sure all of these recommendations are incorporated into the templates or possible to enable as options.