Currently, in order to achieve blue/green deployment with worker groups (i.e. updating to a new AMI), I have to add a new worker group with the updated AMI, let them spin up, drain the old nodes so pods transition, then scale down the old worker group (set min/max/desired to 0).
This is not a terrible way of doing it but the problem is that the old ASG (and related resources) sticks around forever and there doesn't seem to be a way to clean up the old stuff without major surgery. If I change the AMI 3 times, I now have 3 worker groups - 2 inactive and scaled to 0 and one active.
Is there a better way of doing this with this module? There's a distinct possibility I'm missing some fundamental terraform concepts but this seems like a complex issue to me. My code ends up looking like this after a new worker group is fully deployed and the old is scaled down (you can see how even semi-frequent deployments would make this list long and leave a lot of trailing garbage):
map(
"name", "k8s-worker-179fc16f",
"ami_id", "ami-179fc16f",
"asg_desired_capacity", "0",
"asg_max_size", "0",
"asg_min_size", "0",
),
map(
"name", "k8s-worker-67a0841f",
"ami_id", "ami-67a0841f",
"asg_desired_capacity", "5",
"asg_max_size", "8",
"asg_min_size", "5",
"instance_type","${lookup(var.worker_sizes, "${terraform.workspace}")}",
"key_name", "${aws_key_pair.infra-deployer.key_name}",
"root_volume_size", "48"
)
I have seen other code around the internet that does blue/green ASGs but those are for much simpler use-cases IMO - a create_before_destroy
and letting it rip would bring a K8s cluster down. I have no qualms with multiple apply steps - its the cleanup part that I'm after.