gruntwork-io / terraform-aws-couchbase

Reusable infrastructure modules for running Couchbase on AWS

⚠️ As of February, 2022, we are putting this repo on hold and will not be investing in further improvements to it. ⚠️

This is because:

• Maintaining the code for a complicated, distributed, stateful system like Couchbase is very time consuming.
• We’ve had very little customer usage / interest in a self-managed Couchbase offering, so it does not make sense to continue to make a large investment in it.

Therefore, for the time being, if you need to use Couchbase, we strongly recommend using one of the following alternatives:

• Couchbase in AWS marketplace
• Couchbase Capella

If you have questions or concerns, please contact us at [email protected].

Couchbase AWS Module

This repo contains a set of modules for deploying Couchbase on AWS using Terraform and Packer. Couchbase is a distributed NoSQL document database. This module supports running Couchbase as a single cluster:

Or as multiple clusters for the various Couchbase services (data, management, search, index, query) and Sync Gateway:

Features

• Deploy Couchbase and Sync Gateway.
• Automatic bootstrapping.
• Cross-region replication
• Multi-dimensional scaling, allowing you to separately scale data, management, search, index, query, and Sync Gateway nodes.
• Auto healing.
• Web console UI.

Learn

This repo is maintained by Gruntwork, and follows the same patterns as the Gruntwork Infrastructure as Code Library, a collection of reusable, battle-tested, production ready infrastructure code. You can read How to use the Gruntwork Infrastructure as Code Library for an overview of how to use modules maintained by Gruntwork!

Core concepts

• Couchbase documentation: The core documentation for Couchbase, inculding guides for administrators, developers, SQL developers, and mobile developers.
• Couchbase tutorials: hands-on guides for getting started with Couchbase.
• Couchbase Security: overview of how to secure your Couchbase clusters.

Repo organization

Check out How to use this repo: for an overview.

• modules: the main implementation code for this repo, broken down into multiple standalone, orthogonal submodules.
• examples: This folder contains working examples of how to use the submodules.
• test: Automated tests for the modules and examples.
• root: The root folder is an example of how to use the submodules to deploy a Couchbase cluster. The Terraform Registry requires the root of every repo to contain Terraform code, so we've put one of the examples there. This example is great for learning and experimenting, but for production use, please use the underlying modules in the modules folder directly.

Deploy

Non-production deployment (quick start for learning)

If you just want to try this repo out for experimenting and learning, check out the following resources:

• examples folder: The examples folder contains sample code optimized for learning, experimenting, and testing (but not production usage).
• quick start: A quick start guide for this repo.

Production deployment

If you want to deploy this repo in production, check out the following resources:

• Couchbase deployment guidelines: A guide on how to configure Couchbase for production. All of these settings are exposed by the modules in the modules folder.
• Security options: The security options you can configure using these modules.
• Credentials: How to pass credentials securely to your Couchbase server.
• Memory settings: How to configure memory settings in your Couchbase server.

Manage

Day-to-day operations

• How to connect to Sync Gateway
• How to connect to the Couchbase Web Console
• How to connect to the Couchbase Server via SDK

Major changes

• How to upgrade a Couchbase cluster

Support

If you need help with this repo or anything else related to infrastructure or DevOps, Gruntwork offers Commercial Support via Slack, email, and phone/video. If you're already a Gruntwork customer, hop on Slack and ask away! If not, subscribe now. If you're not sure, feel free to email us at [email protected].

Contributions

Contributions to this repo are very welcome and appreciated! If you find a bug or want to add a new feature or even contribute an entirely new module, we are very happy to accept pull requests, provide feedback, and run your changes through our automated test suite.

Please see CONTRIBUTING.md for instructions.

License

Please see LICENSE for details on how the code in this repo is licensed.

Copyright © 2019 Gruntwork, Inc.

terraform-aws-couchbase's Issues

Rename vars.tf to variables.tf

README.md specifies "vars.tf". File in the repo is variables.tf small issue, but should be changed.

Unsupported argument

I downloaded the main.tf from erraform-aws-couchbase/modules/load-balancer-target-group/
And then I copied the target group module changing the source to the above folder that has the main.tf
ran terraform init and terraform apply
crashed with error Unsupported argument for all argument
Terraform v0.12.24

• provider.aws v2.58.0

EBS volumes fail to format on nitro instances

While trying to setup a production cluster using the data node instance type m5a.4xlarge I'm getting the following error from the couchbase-commons/mount-volume.sh script.

cat /opt/couchbase/var/lib/couchbase/logs/mock-user-data.log

Mounting EBS Volume for the data directory
2020-12-28 17:56:44 [INFO] [part-001] Creating ext4 file system on /dev/xvdh...
mke2fs 1.44.1 (24-Mar-2018)
The file /dev/xvdh does not exist and no size was specified.

There is no file in /dev for xvdh but it does indeed show that device name in the aws console as attached. When I run lsblk on one of the instances I only see the following:

NAME        TYPE  SIZE FSTYPE   MOUNTPOINT                  LABEL
nvme1n1     disk  200G
nvme0n1     disk   50G
└─nvme0n1p1 part   50G ext4     /                           cloudimg-rootfs

I'm able to manually format the nvme device using the mount_volume function but the ASG fails to create instances when I change the data_volume_device_name to /dev/nvme1n1

Launching a new EC2 instance. Status Reason: Invalid device name /dev/nvme1n1. Launching EC2 instance failed.

[update] It appears that aws nitro based instances and EBS device naming don't symlink the xvdX names any longer and the script doesn't account for this. 🤨

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/nvme-ebs-volumes.html#identify-nvme-ebs-device

Security issue: User Data of instance contains secrets

After terraform creates couchbase server instances, credentials and other secrets included in user data bash script

Do not allow auto scaling group to terminate unhealthy instances

I'm not exactly sure I understand the use case for killing unhealthy Couchbase EC2 instances. This seems like an overly destructive operation, particularly because you don't have an opportunity to troubleshoot or correct/failover the node gracefully and re-balance the cluster.

I encountered a scenario in my test environment where a bad query caused excessive CPU usage on multiple nodes, which then resulted in the health check of the nodes to not return healthy. The instances were terminated by the ASG and I lost data/indexes.

For now, I have added the following attribute to the aws_autoscaling_group: suspended_processes = ["Terminate", "ReplaceUnhealthy"] to prevent terminations. Once the cluster is spun up, I don't really want/need the ASG to terminate nodes.

Actually, now that I think about, I don't really understand why you would even want the nodes to be in an auto-scaling group with the way the current modules work. I could see maybe adding index/query nodes based on demand, but the modules don't currently do that, as far as I can tell.

Some options might be: make the suspended_processes variable so that they can be set, and default to not terminating instances. You could also remove the nodes from the ASG entirely and just set the desired number of nodes you want to be launched.

If neither of these are desirable, I'm curious to know why an ASG is being used as opposed to just launching a specified number of instances.

Change comment to read couchbase-iam-policies instead of security-group-rules

terraform-aws-couchbase/examples/couchbase-cluster-simple-dns-tls/main.tf

Line 239 in 5d54d8f

# source = "git::[email protected]:gruntwork-io/terraform-aws-couchbase.git//modules/couchbase-server-security-group-rules?ref=v0.0.1"

When a new node is added or deleted to/from the cluster, sync gateway configuration is not updated on the running nodes

When a new node is added or deleted to/from the cluster, sync gateway configuration is not updated on the running nodes which causes rebalancing failures

Possibility of passing in tags for security group in couchbase-cluster module

Hello,

My team has pretty strict tagging guidelines, and I was wondering if it'd be possible to add another argument to the couchbase-cluster module, say, security_group_tags that would be passed to the aws_security_group.lc_security_group resource tags argument to allow tagging.

On a separate but related note, would it be possible to have the option to use a name rather than name_prefix in the aws_security_group.lc_security_group resource?

Thanks,
Brian

N.B. I'd be happy to submit the PR, if desired

Unable to build ubuntu-18-docker in ami config

The instructions for building the local docker image work for ubuntu-docker which is a Ubuntu 16 image. The config file has an entry for ubuntu-18-docker but it fails.

https://github.com/gruntwork-io/terraform-aws-couchbase/tree/master/examples/local-mocks

==> ubuntu-18-docker: Starting docker container...
    ubuntu-18-docker: Run command: docker run --privileged -v /sys/fs/cgroup:/sys/fs/cgroup -v /Users/jakemallory/.packer.d/tmp054385387:/packer-files -d -i -t gruntwork/ubuntu-test:18.04 /sbin/init
==> ubuntu-18-docker: Error running container: Docker exited with a non-zero exit status.
==> ubuntu-18-docker: Stderr: docker: Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: exec: "/sbin/init": stat /sbin/init: no such file or directory: unknown.
==> ubuntu-18-docker:
Build 'ubuntu-18-docker' errored after 2 seconds 513 milliseconds: Error running container: Docker exited with a non-zero exit status.
Stderr: docker: Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: exec: "/sbin/init": stat /sbin/init: no such file or directory: unknown.

The run command is defined the exact same as the v16 based image but doesn't like the "/sbin/init" run command.

Swapping the run command to /bin/bash gets it past that error but then fails with following message:

sudo: systemctl: command not found

Unsupported argument at load-balancer-target-group

Getting unsupported argument error on "terraform plan"

Error trace:
Error: Unsupported argument

on .terraform/modules/couchbase_target_group/modules/load-balancer-target-group/main.tf line 59, in resource "aws_alb_listener_rule" "http_path":
59: field = condition.value["field"]

An argument named "field" is not expected here.

Error: Unsupported argument

on .terraform/modules/couchbase_target_group/modules/load-balancer-target-group/main.tf line 60, in resource "aws_alb_listener_rule" "http_path":
60: values = condition.value["values"]

An argument named "values" is not expected here.

Error: Unsupported argument

on .terraform/modules/sync_gateway_target_group/modules/load-balancer-target-group/main.tf line 59, in resource "aws_alb_listener_rule" "http_path":
59: field = condition.value["field"]

An argument named "field" is not expected here.

Error: Unsupported argument

on .terraform/modules/sync_gateway_target_group/modules/load-balancer-target-group/main.tf line 60, in resource "aws_alb_listener_rule" "http_path":
60: values = condition.value["values"]

An argument named "values" is not expected here.

Terraform v0.12.29

• provider.aws v3.0.0
• provider.template v2.1.2

Ubuntu 20.04

Focus support on EE

It looks like ownership changed on the four AMIs recently?

terraform-aws-couchbase/main.tf

Line 233 in ca71f30

owners = ["738755648600"] # Gruntwork

I used the main.tf one week ago to create a cluster using the default AMI and it worked.
But today I was getting an error until I looked up the owners on the AMIs and I saw they were different than what is in the source code.

data "aws_ami" "coubase_ubuntu_example" {
most_recent = true
#owners = ["738755648600"] # Gruntwork
owners = ["562637147889"] # Gruntwork - Seems to be the new AMI owner id

missing ami -- typo? coubase_ubuntu_example

note it looks like a typo below? coubase should be couchbase?

$ terraform apply
data.template_file.user_data_server: Refreshing state...
data.aws_iam_policy_document.instance_role: Refreshing state...
data.aws_vpc.default: Refreshing state...
data.aws_iam_policy_document.auto_discover_cluster: Refreshing state...
data.aws_ami.coubase_ubuntu_example: Refreshing state...
data.aws_subnet_ids.default: Refreshing state...

Error: Error refreshing state: 1 error(s) occurred:

* module.couchbase.data.aws_ami.coubase_ubuntu_example: 1 error(s) occurred:

* module.couchbase.data.aws_ami.coubase_ubuntu_example: data.aws_ami.coubase_ubuntu_example: Your query returned no results. Please change your search criteria and try again.

Roll out upgrade

Any plans to add this ?

clean_ami_name function is no longer supported by packer

Hi, it seems that deprecated clean_ami_name function used in couchbase-ami example is removed in packer 1.5.0 (packer changelog)

Probably clean_resource_name should be used instead https://www.packer.io/docs/templates/engine.html#specific-to-amazon-builders-

Couchbase Community 5.0.1 not working with Amazon Linux AMI

Hello,

I used Packer to build a Couchbase AMI with the couchbase-ami script. I'm trying to install Couchbase Community version 5.0.1; however, this is not working with the Amazon Linux 2 builder. When I run the couchbase-cluster and run-couchbase-server modules, the ec2 instance built from the AMI fails on the run-couchbase-server start_couchbase function. I've tried to run the Couchbase server manually after SSH'ing in. After running sudo systemctl start couchbase-server I get an error: opt/couchbase/lib/erlang/erts-5.10.4.0.0.1/bin/beam.smp: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory

The libtinfo.so.5 file does not exist on the Amazon Linux 2 build. Granted, this could be an incompatibility with Amazon Linux and Couchbase and not necessarily a problem with your code. But I thought it should be brought to your attention. I was able to use a workaround by sym-linking libtinfo.so.5 to libtinfo.so.6 in the user-data.sh. This gets server up and running, but is not a great solution.

Thanks,
Brian

P.S Using the ubuntu builder with the exact same configuration works perfectly.

How to upgrade a Couchbase cluster

The current link directs to empty page. Do you have the procedure how to upgrade the cluster ?

calculate_memory_settings_automatically always gives all memory to data service

It looks like memory is always being allocated based on all the services regardless if you pass in the --node-services argument.

terraform-aws-couchbase/modules/run-couchbase-server/run-couchbase-server

Line 754 in 027eb19

read data_ramsize index_ramsize fts_ramsize < <(calculate_memory_settings_automatically "$cluster_services")

Is there a reason this is ignoring the $node_services argument passed in and always using local cluster_services="$DEFAULT_SERVICES"?

Default to AWS Marketplace AMIs

error in run-sync-gateway/wait_for_couchbase_clusters when config file has backtics

terraform-aws-couchbase/modules/run-sync-gateway/run-sync-gateway

Line 185 in 027eb19

databases=($(cat "$config" | jq -c '.databases | .[]?'))

This line of code is throwing an error when the config file has backticks in the import_filter or sync json properties as mentioned in the docs

The error:

parse error: Invalid numeric literal at line 42, column 0

Rebalance failed. Auto scale trigger everytime

HI,
I was testing couchbase-cluster-simple-dns-tls from example
In the main.tf
min_size = 1
max_size = 2
everything worked fine.
After changing/applying
min_size = 2
max_size = 2
2 node started and seems everything working fine , but after reverting back to
min_size = 1
max_size = 2
something started doing wrong. My master node is terminated and Rebalance started failing. Whenever i visit admin dashboard auto scale group starts trigger and creates new node and destroy its after sometime.

I'm new in this area , but i think this issue can be handle by making primary node terminate proof.

Not able to update load_balancer.https_listener_ports_and_certs

Hi,

I am using terraform 0.12.12 and I have a load_balancer resource pointing to "gruntwork-io/couchbase/aws//modules/load-balancer" version "0.2.2".

When I try to modify the segment https_listener_ports_and_certs by removing one listener and I do a terraform plan, terraform complains with the following message:

Error: Error in function call

  on .terraform/modules/load_balancer/gruntwork-io-terraform-aws-couchbase-b2c210b/modules/load-balancer/outputs.tf line 27, in output "https_listener_arns":
  27:   value = zipmap(
  28:
  29:
  30:
    |----------------
    | aws_alb_listener.https is tuple with 4 elements
    | data.template_file.https_listener_ports is tuple with 3 elements

Call to function "zipmap" failed: number of keys (3) does not match number of
values (4).

This seems to be a terraform issue. Modifying the file gruntwork-io-terraform-aws-couchbase-b2c210b/modules/load-balancer/outputs.tf by explicitly adding a dependency on var.https_listener_ports_and_certs solves the problem:

data "template_file" "https_listener_ports" {
  count    = length(var.https_listener_ports_and_certs)
  template = var.https_listener_ports_and_certs[count.index]["port"]

  depends_on = [
    var.https_listener_ports_and_certs
  ]
}

THP is not disabled

The script to disable THP is copied to the machine but no command is ever run to execute it.

The CB docs mention that this command must be run on Ubuntu for example

sudo update-rc.d disable-thp defaults

This should be added to the install script

404 on readme link - How to upgrade a Couchbase cluster

Unsupported argument in resource "aws_alb_listener_rule" "http_path"

Hi,
I'm using the module and during "terraform plan" I have an issue with unsupported argument:

`Error: Unsupported argument

on ../../modules/load-balancer-target-group/main.tf line 60, in resource "aws_alb_listener_rule" "http_path":
60: values = condition.value["values"]

An argument named "values" is not expected here.`

It looks like that the issue is because of "condition block" inside resource "aws_alb_listener_rule" as it is deprecated.

Could you please check and advise.

Thanks

Community Couchbase 6.0.0 and sync-gateway 2.5.0 are available. Are there any plans to update the installation scripts to use these new versions?

Community Couchbase 6.0.0 and sync-gateway 2.5.0 are available. Are there any plans to update the installation scripts to use these new versions?

Volumes resizing and re-attachment

Hi, I've got two questions:

1. Assuming that disks are almost full, how can I increase their size?
2. I'm running simple cluster example and I've observed that when one node is terminated, volumes are not re-attached. As I understand It shouldn't be an issue if only one node is broken, because when ASG will start new an instance, data will be replicated to the new disk. But what when more nodes (or all) go down? Based on your experience, do you recommend to handle volumes re-attachment somehow for production system? If so, how can it be done?

Fix typos in install-couchbase-server script

swapiness should be swappiness in the script.

Main problem is in update_swapiness but I would recommend that this change be made throughout the file and in the README.md file.

change this line:
file_replace_or_append_text "^vm.swappiness.=.$" "vm.swapiness = $swapiness" "$SWAPINESS_CONFIG_FILE"

To:
file_replace_or_append_text "^vm.swappiness.=.$" "vm.swappiness = $swappiness" "$SWAPPINESS_CONFIG_FILE"
}

Without this change, the swappiness value is not changed on the EC2 to 0.