bennu / terraform-helm-kong

Allow to use Helm with Kong Chart as Kong Ingress Controller or as just only an API Gateway on kubernetes

KONG MODULE

This repo allow to use Helm with Kong Chart as Kong Ingress Controller or as just only an API Gateway on kubernetes.

Api Kong Gateway Content

Kong Gateway is the world’s most popular open source API gateway, built for multi-cloud and hybrid, and optimized for microservices and distributed architectures

Stable Version

Name	Version
Kong Module	0.1.6

Requirements

Name	Version
kubernetes	>= 1.16
postgreSQL	>= 9.5
terraform	>= 0.13

• Another Ingress Controler (optional)

Components

Name	Version	URL
Kong Chart	1.15.0	https://github.com/Kong/charts/blob/main/charts/kong/CHANGELOG.md#1150
Kong docker image	2.2.1	https://github.com/Kong/docker-kong/releases/tag/2.2.1
Image IngressController	1.1	https://github.com/Kong/kubernetes-ingress-controller/blob/main/CHANGELOG.md#110---20201209
Kong for Kubernetes	0.10	https://konghq.com/blog/kong-for-kubernetes-0-10-released-with-ingress-v1-resource-improved-ingress-class-handling-and-more/

Examples main.tf

Kong as API Gateway

module "kong_apigateway" {
  # Using our module your can set a versions to deploy specific features
  source  = "bennu/kong/helm"
  version = "0.1.6"

  db_host   = var.db_host
  db_name   = var.db_name
  db_pass   = var.db_pass
  db_user   = var.db_user
  namespace = "kong"

  enable_proxy_ingress      = true
  proxy_ingress_host        = "prod.api.xyz.com"
  proxy_ingress_annotations = {
    kubernetes.io/ingress.class: "nginx"
  }
}

Kong as Ingress Controller

module "kong_ingresscontroller" {
  source  = "bennu/kong/helm"

  db_host = var.db_host
  db_name = var.db_name
  db_pass = var.db_pass
  db_user = var.db_user

  create_ingress_controller = true

  # It is possible to set a definition about the resources quotas of pods,
  # so you only need to declare the request and / or the limits as you need.
  resources = {
    requests = {
      cpu    = "250m"
      memory = "275Mi"
    }
    limits = {
      cpu    = "750m"
      memory = "550Mi"
    }
  }
}

Custom kong.conf

module "kong" {
  source  = "bennu/kong/helm"
  ...

  # we can configure customs values for kong.conf (https://github.com/Kong/kong/blob/master/kong.conf.default)
  # only need to pass a list of names and values using variable "extra_env_configs" as below.
  extra_env_configs = [
    {
      "name"  = "nginx_http_client_header_buffer_size",
      "value" = "16k"
    },
    {
      "name"  = "nginx_http_large_client_header_buffers",
      "value" = "8 64k"
    },
    {
      "name"  = "mem_cache_size",
      "value" = "200m"
    }
  ]
  ...
}

Module Variables

Some details about variables for this Kong module.

Inputs

Name	Description	Type	Default	Required
db_host	PostgreSQL database hostname	string	n/a	yes
db_name	PostgreSQL database name	string	n/a	yes
db_pass	PostgreSQL database password	string	n/a	yes
db_user	PostgreSQL database user	string	n/a	yes
admin_annotations	Annotations for the Kong admin service	map(any)	{}	no
admin_ingress_annotations	Annotations for Kong admin ingress	map(any)	{}	no
admin_ingress_hostname	Kong admin hostname	string	"admin.local"	no
admin_ingress_path	Kong admin path on Ingress	string	"/"	no
admin_service_type	Kong admin service type on Kubernetes	string	"ClusterIP"	no
autoscaling_cpu_average_usage	Cpu average usage for autoscaling	number	70	no
autoscaling_max_replicas	Number of maximum replicas of pods	string	2	no
autoscaling_mem_average_usage	Memory average usage for autoscaling	number	75	no
autoscaling_min_replicas	Number of minimum replicas of pods	string	1	no
bash_image	Bash docker image name for jobs	string	"bash"	no
bash_image_tag	Bash docker image tag for jobs	number	5	no
chart_extra_set_configs	Using a list of maps as [{"name"="foo", "value"="bar"},] to create dynamics blocks of 'set' to merge with values	list(any)	[]	no
chart_name	Helm chart name for Kong	string	"kong"	no
chart_repository	Helm chart repository for Kong	string	"https://charts.konghq.com"	no
chart_version	Helm chart version for Kong	string	"1.15.0"	no
create_ingress_controller	Create an Kong Ingress Controller	bool	false	no
database_engine	Database engine for Kong	string	"postgres"	no
db_port	PostgreSQL database port	string	"5432"	no
enable_admin_ingress	Admin exposure using another Ingress Controller	bool	false	no
enable_admin_service	Enable Kong admin service	bool	true	no
enable_autoscaling	Define if autoscale option is enable for Kong's pods	bool	false	no
enable_proxy_https	Enable TLS on Kong proxy service	bool	false	no
enable_proxy_ingress	Proxy exposure using another Ingress Controller	bool	false	no
enable_proxy_service	Enable Kong proxy service	bool	true	no
extra_env_configs	Define a list of maps as [{"name"="foo", "value"="bar"},] to configure customs values for kong.conf	list(any)	[]	no
ingress_controller_install_crds	Install CRDS for Kong ingress controller, ONLY if using HELM 2.	bool	false	no
ingress_image	Define repository for image ingress controller	string	kong/kubernetes-ingress-controller	no
ingress_image_tag	Define tag for image ingress controller	string	1.1	no
kong_image	Kong docker image name	string	"kong"	no
kong_tag	Kong docker image tag	string	"2.2.1-alpine"	no
migrations_post_upgrade	Able to activate post upgrade containers	bool	true	no
migrations_pre_upgrade	Able to activate pre upgrade containers	bool	true	no
migrations_resources	Define the limits and/or requests for migrations containers	map(any)	{}	no
name	Value for kong name in pods	string	""	no
namespace	Namespace where resources are deployed	string	"default"	no
priority_class_name	Priority indicates the importance of a Pod relative to other Pods. Kubernetes already ships with two PriorityClasses: system-cluster-critical and system-node-critical.	string	""	no
proxy_annotations	Annotations for the Kong proxy service	map(any)	{}	no
proxy_ingress_annotations	Annotations for proxy on another Ingress Controller	map(any)	{}	no
proxy_ingress_host	Proxy Host on another Ingress Controller	string	"api.local"	no
proxy_ingress_path	Proxy path on another Ingress Controller	string	"/"	no
proxy_service_type	Kong proxy service type on Kubernetes	string	"ClusterIP"	no
reg_cred	Registry secret credential	list(any)	[]	no
registry	Custom registry host for be used in all the containers	string	""	no
replica_count	Number of Kong pod replicas if autoscaling is not enable	string	1	no
resources	Define the limits and/or requests on pod resources	map(any)	{}	no

Outputs

Name	Description
ingressclass	Kong ingress class name
name	Name of helm release for kong
uri_admin_service	URI for internal kong admin service

terraform-helm-kong's Issues

Insufficient quota to match these scopes: [{PriorityClass In [system-node-critical system-cluster-critical]}]

Hello guys.

I am facing problems when deploying provider kong on terraform, as evidenced below:

ERROR

Error in pod kong: insufficient quota to match these scopes: [{PriorityClass In [system-node-critical system-cluster-critical]}]
Events error kubernets: Scaled up replica set kong-qbjyjt9i-kong-77c654764b to 1 | ScalingReplicaSet

VERSIONS

terraform {
  required_version = "0.13.7"
  #backend "gcs" { bucket = "pd-terraform-state" }
  required_providers {
    google = {
      version = "3.64.0"
      source  = "hashicorp/google"
    }
    google-beta = {
      source  = "hashicorp/google-beta"
      version = "~> 3.66.1"
    }
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "~> 2.1.0"
    }
    helm = {
      source  = "hashicorp/helm"
      version = "~> 2.1.2"
    }
  }
}

main.tf

resource "kubernetes_namespace" "kong-namespace" {
  metadata {
    name = "kong"
  }
}

module "kong_ingresscontroller" {
  source  = "bennu/kong/helm"

  db_host = var.db_host
  db_name = var.db_name
  db_pass = var.db_pass
  db_user = var.db_user

  namespace = "kong"
  create_ingress_controller = true

  resources = {
    requests = {
      cpu    = "500m"
      memory = "2Gi"
    }
    limits = {
      cpu    = "750m"
      memory = "2Gi"
    }
  }
}

PLAN

  # module.kong.module.kong_ingresscontroller.helm_release.kong will be created
  + resource "helm_release" "kong" {
      + atomic                     = true
      + chart                      = "kong"
      + cleanup_on_fail            = false
      + create_namespace           = false
      + dependency_update          = false
      + disable_crd_hooks          = false
      + disable_openapi_validation = false
      + disable_webhooks           = false
      + force_update               = false
      + id                         = (known after apply)
      + lint                       = false
      + manifest                   = (known after apply)
      + max_history                = 0
      + metadata                   = (known after apply)
      + name                       = "kong-qbjyjt9i"
      + namespace                  = "kong"
      + recreate_pods              = false
      + render_subchart_notes      = true
      + replace                    = false
      + repository                 = "https://charts.konghq.com"
      + reset_values               = false
      + reuse_values               = false
      + skip_crds                  = false
      + status                     = "deployed"
      + timeout                    = 300
      + values                     = [
          + <<~EOT
                "admin":
                  "annotations": {}
                  "enabled": true
                  "http":
                    "enabled": true
                  "ingress":
                    "annotations": {}
                    "enabled": false
                    "hostname": "admin.local"
                    "path": "/"
                  "type": "ClusterIP"
                "affinity":
                  "podAntiAffinity":
                    "requiredDuringSchedulingIgnoredDuringExecution":
                    - "labelSelector":
                        "matchLabels":
                          "app.kubernetes.io/component": "app"
                          "app.kubernetes.io/instance": "kong-qbjyjt9i"
                          "app.kubernetes.io/name": "kong"
                      "topologyKey": "kubernetes.io/hostname"
                "autoscaling":
                  "enabled": false
                  "maxReplicas": "2"
                  "metrics":
                  - "resource":
                      "name": "cpu"
                      "target":
                        "averageUtilization": 70
                        "type": "Utilization"
                    "type": "Resource"
                  - "resource":
                      "name": "memory"
                      "target":
                        "averageUtilization": 75
                        "type": "Utilization"
                    "type": "Resource"
                  "minReplicas": "1"
                "env":
                  "database": "postgres"
                  "pg_database": "postgres"
                  "pg_host": "xx.xxx.xxx.xxx"
                  "pg_port": "5432"
                  "pg_user": "xxxx"
                "image":
                  "pullSecrets": []
                  "repository": "kong"
                  "tag": "2.2.1-alpine"
                "ingressController":
                  "enabled": true
                  "ingressClass": "kong"
                  "installCRDs": false
                "migrations":
                  "postUpgrade": true
                  "preUpgrade": true
                  "resources": {}
                "priorityClassName": "system-cluster-critical"
                "proxy":
                  "annotations": {}
                  "enabled": true
                  "ingress":
                    "annotations": {}
                    "enabled": false
                    "hostname": "api.local"
                    "path": "/"
                  "tls":
                    "enabled": false
                  "type": "ClusterIP"
                "replicaCount": "1"
                "resources":
                  "limits":
                    "cpu": "750m"
                    "memory": "2Gi"
                  "requests":
                    "cpu": "500m"
                    "memory": "2Gi"
                "waitImage":
                  "repository": "bash"
                  "tag": 5
            EOT,
        ]
      + verify                     = false
      + version                    = "1.15.0"
      + wait                       = true
      + wait_for_jobs              = false

      + set_sensitive {
          + name  = "env.pg_password"
          + value = (sensitive value)
        }
    }

apparently, when I remove that line, it resolves the error

priorityClassName = "system-cluster-critical

Can you help me? Thanks