sieunhantanbao / sd2411_azure_infrastructure

Provisioning Azure Resources by Terraform, and handling the Continue Deployment (CD) by ArgoCD

SD2411 Infrastructure and ArgoCD

Table Of Content

• Reference Repositories
• Overview
• Key Features
• Prerequisite Tools
• Infrastructure set up
  • Provision Azure Kubernetes Service (AKS)
  • Provision Azure Container Registry (ACR)
  • Provision Virtual Machine (VMs)
• Software/Tools setup
  • Install ArgoCD (with helm support enable)
  • Install ArgoCD Image Updater
  • Install Argo Rollouts and Kubernetes Plugin (kubectl argo rollout)
  • Setup Prometheus and Grafana (monitoring)
  • Install Istio
  • Setup Jenkins on Virtual Machine
• Deploy application with ArgoCD and Demonstration
  • Deploy application
• Manage the application on the ArgoCD UI
  • Demonstration screenshot
• Monitoring with Prometheus and Grafana
• Cleanup Azure resources
  • Cleanup Azure Kubernetes Service (AKS)
  • Cleanup Azure Virtual Machine (VMs)
  • Cleanup Azure Container Registry (ACR)

Reference Repositories

Repository	Description
sd2411_msa	Application source code. This contain a backend, frontend and use the mongo as database
sd2411_devops_ci	This contains the Jenkins Groovy files (Jenkins Shared Library). When the sd2411_msa has changed the source code, it will call the Jenkins files in this repo to build the source code (CI process)
sd2411_helm_charts	This contains the helm charts definition and helm chart packages (manifest) to deploy the apps from the sd2411_msa
sd2411_azure_infrastructure	Ops source code. This contains the infrastructure as code (iac) to provision the Azure resources with terraform. This also handles the Continue Deployment (CD) with ArgoCD

Overview

Key Features

• Provision the Azure resources by Terraform
  • Azure Kubernetes Service (AKS)
  • Azure Container Registry (ACR)
  • Virtual Machine (VMs)
• Install/setup software/tools
  • Jenkins on VM to build and scan the app code
  • ArgoCD with helm support
  • ArgoCD Image Updater
  • Argo Rollouts
  • Istio
  • Prometheus and Grafana
• Handle Continues Deployment (CD) process with ArgoCD.

Prerequisite Tools

• Azure Command-line Interface (CLI)
• Kubectl CLI
• Terraform CLI

Infrastructure set up

Provision Azure Kubernetes Service (AKS)

This will provision an AKS cluster in High Availability (Use Multi-AZs)

• Change directory (cd) to iac/terraform/aks/ha: cd iac/terraform/aks/ha
• Modify the variables in the variables.tf file to match your requirements.
• Run the below commands
  • terraform init
  • terraform plan --out tfplan.out
  • terraform apply tfplan.out

Provision Azure Container Registry (ACR)

• Change directory (cd) to iac/terraform/aks/acr: cd iac/terraform/acr
• Modify the variables in the variables.tf file to match your requirements.
• Run below command
  • terraform init
  • terraform plan --out tfplan.out
  • terraform apply tfplan.out

Provision Virtual Machine (VMs)

This will provision an Ubuntu VM with Docker, Jenkins, Trivy, and Kubectl installed

• Change directory (cd) to iac/terraform/aks/vm: cd iac/terraform/vm
• Modify the variables in the variables.tf file to match your requirements.
• Run below command
  • terraform init
  • terraform plan --out tfplan.out
  • terraform apply tfplan.out

Note: The script to install Docker, Jenkins, Trivy, and Kubectl can be found in iac/terraform/vm/azure-user-data.sh

Software/Tools setup

Install ArgoCD (with helm support enable)

• Get AKS credential: az aks get-credentials --resource-group <your_resource_group_name> --name <your_aks_cluster_name>
• Create argocd namespace: kubectl create namespace argocd
• Navigate to cd tools/argocd, run this command kubectl apply -n argocd -f install-argocd.yaml
• Edit argocd-server to change (ClusterIP to LoadBalancer): kubectl edit svc argocd-server -n argocd
• Get ArgoCD password (username: admin): kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath={.data.password} | base64 -d
• Login to ArgoCD by the External IP (URL) from the argocd-server service: kubectl get svc argocd-server -n argocd

Note: By default when installing the ArgoCD from the source (i.e. kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml), it does not enable the helm chart support. So, we have updated the tools/argocd/install-argocd.yaml to enable it

Install ArgoCD Image Updater

• Navigate to cd tools/argocd-image-updater and run this command: kubectl apply -f 0-install-argocd-image-updater.yaml
• Create docker registry secret azure-container-secret in agrocd namespace. This is to allow the Argo CD Image Updater to pull the docker images from the ACR. Please refer here to create a new Service Principal for the ACR: Pull images from an Azure container registry to a Kubernetes cluster using a pull secret. This will allow the ArgoCD Image Updater to fetch/list the images tag/version from the ACR: kubectl create secret docker-registry azure-container-secret --namespace agrocd --docker-server=<container registry name>.azurecr.io --docker-username=<Service principal ID> --docker-password=<Service principal password>

Install Argo Rollouts and Kubernetes Plugin (kubectl argo rollout)

• This tool is to support the Blue/Green deployment strategy in K8S.
• Please refer to this link Argo Rollouts for the detail of the installation.

Setup Prometheus and Grafana (monitoring)

• Navigate to cd tools/monitoring and run this command sh install-monitoring-tools.sh. For more details, please visit HERE.

Install Istio

• Follow this instruction to enable the Istio on the AKS cluster.
• If you do not use the AKS then follow this GUIDE to install Istio with Istioctl tool to the Kubernetes cluster.

Setup Jenkins on Virtual Machine

The step Provision Virtual Machine has already installed a Jenkins. Please refer to How To Install Jenkins on Ubuntu 22.04 (starts from step #6: Set up Jenkins). While setup the Jenkins, please make sure the plugins below get installed

• Jenkins suggested plugins
• Docker PipelineVersion
• Pipeline Utility Steps
• HTML Publisher

For further details of the setup, please visit sd2411_devops_ci

Deploy application with ArgoCD and Demonstration

Deploy application

• Change directory (cd) to argocd/helm/{environment_name} (i.e. cd argocd/helm/qa) and run the below commands
  • Deploy the Azure Container Registry (ACR) secret. Please refer here to create a new Service Principal for the ACR: Pull images from an Azure container registry to a Kubernetes cluster using a pull secret. This will allow the helm to pull the images from the ACR.
    • kubectl create ns qa
    • kubectl create secret docker-registry qa-acr-secret \ --namespace qa \ --docker-server=<container registry name>.azurecr.io \ --docker-username=<Service principal ID> \ --docker-password=<Service principal password>
  • Deploy database: kubectl apply -f 1-mongo.yml
  • Deploy backend: kubectl apply -f 2-backend.yml
  • Deploy frontend: kubectl apply -f 3-frontend.yml

Manage the application on the ArgoCD UI

Demonstration screenshot

• ArgoCD UI
  
• Frontend-qa application (kubectl port-forward service/frontend 80:3000 -n qa)
  
• ArgoCD UI (multiple environments)
  

Monitoring with Prometheus and Grafana

• Prometheus
• Grafana

Cleanup Azure resources

Cleanup Azure Kubernetes Service (AKS)

• Change directory (cd) to iac/terraform/aks/ha: cd iac/terraform/aks/ha
• Run the below command
  • terraform destroy
• Type: yes to confirm the cleanup.

Cleanup Azure Virtual Machine (VMs)

• Change directory (cd) to iac/terraform/vm: cd iac/terraform/vm
• Run the below command
  • terraform destroy
• Type: yes to confirm the cleanup.

Cleanup Azure Container Registry (ACR)

• Change directory (cd) to iac/terraform/acr: cd iac/terraform/acr
• Run the below command
  • terraform destroy
• Type: yes to confirm the cleanup.