kubevious / helm

Helm charts repository for Kubevious

Kubevious Helm Charts

This repo is focused only on the installation of Kubevious using Helm chart. To learn more about Kubevious and how it can help with preventing Kubernetes disasters, please refer to the root repository: https://github.com/kubevious/kubevious

🔨 Prerequisites

• Kubernetes v1.13 or higher
• Helm v3.2 or higher

🚀 Installation

First create a namespace:

kubectl create namespace kubevious

Add Kubevious repository and install the Helm chart:

helm repo add kubevious https://helm.kubevious.io
helm upgrade --atomic -i kubevious kubevious/kubevious --version 1.2.1 -n kubevious 

🚒 Troubleshooting

If Kubevious crashes with OOM, try giving it a little bit extra memory:

helm upgrade --atomic -i -n kubevious \
    --version 1.2.1 \
    --set collector.resources.requests.memory=4G \
    --set collector.v8MaxOldSpace=4000 \
    --set parser.resources.requests.memory=4G \
    --set parser.v8MaxOldSpace=4000 \
    kubevious kubevious/kubevious

🌐 Accessing Kubevious

Kubevious runs within your cluster. Upon successful completion of helm chart installation, you will see commands to access Kubevious UI. There are two ways to access Kubevious UI.

Option 1. Access using port forwarding

The easiest but not most convenient method. Wait few seconds before pods are up and running. Setup port forwarding:

kubectl port-forward service/kubevious-ui-clusterip 8080:80 -n kubevious

Access from browser: http://localhost:8080

Option 2. Expose using Ingress

Enable Ingress deployment using dedicated value parameters. See full list of helm chart values to cofigure Ingress parameters.

helm upgrade --atomic -i -n kubevious \
    --version 1.2.1 \
    --set ingress.enabled=true \
    kubevious kubevious/kubevious

💤 Uninstalling the Chart

Undeploy from cluster:

helm uninstall kubevious -n kubevious

IMPORTANT: As requested by the community, now Kubevious Helm charts generate random MySQL root and user passwords. The Helm uninstall leaves behind the MySQL persistent volume. The same volume will be mounted if Kubevious is reinstalled into the same namespace using the same release name. That creates a big problem because Helm chart will generate new passwords for the backend to connect to MySQL, but the connection would fail because the mounted volume is initialized using the password generated using the initial installation. There are few solutions to this:

1. Delete the PersistentVolumeClain after helm uninstall:

$ kubectl delete pvc data-kubevious-mysql-0 -n kubevious

2. Install Kubevious providing your own root and user passwords. See mysql.root_password and mysql.db_password configuration values below.

3. Bit more complicated way is to update passwords in kubevious-mysql-secret and kubevious-mysql-secret-root Kubernetes secrets. Though wouldn't recommend going that route.

📈 Anonymous Analytics

A two-way feedback mechanism was added to Kubevious. It includes version checks, news updates, useful hints and tips, and reporting of errors, cluster size metrics, and internal time counters. Participants can also see their clusters on a https://worldvious.io leaderboard map. Location is anonymized to the nearest city/zip. No IP address is stored or logged. We calculate the SHA256 hash of the IP address and use it as a key in the backend. If, for some reason, you do not want to participate, please see details of reporting configurations parameters and instructions to opt-out (it's super easy).

🐘 Scale Setup

When running Kubevious in large Kubernetes clusters with lots of Nodes, Pods, Events or other resources, consider providing adequate resources to following chart settings:

• collector.resources.*
• collector.v8MaxOldSpace
• parser.resources.*
• parser.v8MaxOldSpace

For details see https://nodejs.org/docs/latest-v14.x/api/cli.html#cli_useful_v8_options

⚙️ Configuration

The following table lists the configurable parameters of the kubevious chart and their default values.

Value	Description	Default
nameOverride	Overrides the app.kubernetes.io/name label value
fullnameOverride	Overrides name of the app
kubevious.api.skipEvents	Indicates whether Kubernetes Events should be collected by Kubevious. On some systems, that can significantly increase memory, processing, and storage requirements.	True
kubevious.api.skipSecrets	Indicates whether Kubevious should collect Kubernetes Secrets. Values of Secrets are always sanitized and replaced with null. Collecting Secrets helps detect inconsistencies between data keys and their references.	False
kubevious.api.skipped	List of APIs to be skipped from the collection. Use <apiVersion>:<kind> format. For example: apps/v1:ControllerRevision or discovery.k8s.io/v1:EndpointSlice	[]
ingress.enabled	Whether to expose Kubevious using Ingress gateway.	false
ingress.annotations	Dictionary of Ingress annodations.	{kubernetes.io/ingress.allow-http: "true"}
ingress.hosts	Array of hosts and paths for ingress	[{host: "", paths: [{ path: "", pathType: ImplementationSpecific }] }]
ingress.tls	Array of ingress tls configurations. Fields are hosts array and secretName
backend.podAnnotations	Backend pod annotations
backend.image.pullPolicy	Backend PodSpec pullPolicy	IfNotPresent
backend.image.imagePullSecrets	Backend PodSpec imagePullSecrets
backend.service.type	Backend type of service	ClusterIP
backend.service.port	Backend port of service	4000
backend.resources.requests.cpu	Backend request CPU	100m
backend.resources.requests.memory	Backend request Memory	200Mi
backend.resources.limits.cpu	Backend limit CPU
backend.resources.limits.memory	Backend limit Memory
backend.podSecurityContext	Backend PodSpec securityContext
backend.nodeSelector	Backend PodSpec nodeSelector
backend.tolerations	Backend PodSpec tolerations
backend.affinity	Backend PodSpec affinity
backend.log.level	Backend log level. Values are: error, warn, info, verbose, debug, silly	Info
collector.historyRetentionDays	Number of days to maintain history	15
collector.podAnnotations	Collector pod annotations
collector.image.pullPolicy	Collector PodSpec pullPolicy	IfNotPresent
collector.image.imagePullSecrets	Collector PodSpec imagePullSecrets
collector.service.type	Collector type of service	ClusterIP
collector.service.port	Collector port of service	4000
collector.resources.requests.cpu	Collector request CPU	100m
collector.resources.requests.memory	Collector request Memory	200Mi
collector.resources.limits.cpu	Collector limit CPU
collector.resources.limits.memory	Collector limit Memory
collector.v8MaxOldSpace	Collector V8 old memory section (in megabytes)
collector.podSecurityContext	Collector PodSpec securityContext
collector.nodeSelector	Collector PodSpec nodeSelector
collector.tolerations	Collector PodSpec tolerations
collector.affinity	Collector PodSpec affinity
collector.log.level	Collector log level. Values are: error, warn, info, verbose, debug, silly	Info
guard.podAnnotations	Guard pod annotations
guard.image.pullPolicy	Guard PodSpec pullPolicy	IfNotPresent
guard.image.imagePullSecrets	Guard PodSpec imagePullSecrets
guard.service.type	Guard type of service	ClusterIP
guard.service.port	Guard port of service	4000
guard.resources.requests.cpu	Guard request CPU	100m
guard.resources.requests.memory	Guard request Memory	200Mi
guard.resources.limits.cpu	Guard limit CPU
guard.resources.limits.memory	Guard limit Memory
guard.v8MaxOldSpace	Guard V8 old memory section (in megabytes)
guard.podSecurityContext	Guard PodSpec securityContext
guard.nodeSelector	Guard PodSpec nodeSelector
guard.tolerations	Guard PodSpec tolerations
guard.affinity	Guard PodSpec affinity
guard.log.level	Guard log level. Values are: error, warn, info, verbose, debug, silly	Info
parser.podAnnotations	Parser pod annotations
parser.image.pullPolicy	Parser PodSpec pullPolicy	IfNotPresent
parser.image.imagePullSecrets	Parser PodSpec imagePullSecrets
parser.service.type	Parser type of service	ClusterIP
parser.service.port	Parser port of service	4000
parser.resources.requests.cpu	Parser request CPU	100m
parser.resources.requests.memory	Parser request Memory	200Mi
parser.resources.limits.cpu	Parser limit CPU
parser.resources.limits.memory	Parser limit Memory
parser.v8MaxOldSpace	Parser V8 old memory section (in megabytes)
parser.podSecurityContext	Parser PodSpec securityContext
parser.nodeSelector	Parser PodSpec nodeSelector
parser.tolerations	Parser PodSpec tolerations
parser.affinity	Parser PodSpec affinity
parser.log.level	Parser backend log level. Values are: error, warn, info, verbose, debug, silly	Info
parser.serviceAccount.create	Indicates whether a service account should be created for Parser	true
parser.serviceAccount.annotations	Annotations to add to Parser service account
parser.serviceAccount.name	The name of the service account to use. If not and create is true, a name is generated
parser.serviceAccount.skipRoleBinding	Skip creation of RoleBinding and Role for Parser. You would have to create a RoleBinding and Role manually and allow Kubernetes API access for Parser ServiceAccount.	false
ui.podAnnotations	UI pod annotations
ui.image.pullPolicy	UI PodSpec pullPolicy	IfNotPresent
ui.image.imagePullSecrets	UI PodSpec imagePullSecrets
ui.service.type	UI type of service	ClusterIP
ui.service.port	UI port of service	80
ui.resources.requests.cpu	UI request CPU	25m
ui.resources.requests.memory	UI request Memory	50Mi
ui.resources.limits.cpu	UI limit CPU
ui.resources.limits.memory	UI limit Memory
ui.podSecurityContext	UI PodSpec securityContext
ui.nodeSelector	UI PodSpec nodeSelector
ui.tolerations	UI PodSpec tolerations
ui.affinity	UI PodSpec affinity
mysql.external.enabled	Indicates whether an existing MySQL database should be used. When enabled a new MySQL database would not be deployed.	false
mysql.external.host	Host for external MySQL server
mysql.external.port	Port for external MySQL server
mysql.external.database	Database name for external MySQL server. The database should be manually created.
mysql.external.user	User name. User should have access to the database specified above.
mysql.external.password	Password.
mysql.db_name	MySQL database name	kubevious
mysql.db_user	MySQL database user	kubevious
mysql.generate_passwords	Indicates whether a random password should be generated for root and kubevious users	false
mysql.db_password	MySQL database password	"kubevious" or a random password if generate_passwords is set
mysql.root_password	MySQL root user password	"kubevious" or a random password if generate_passwords is set
mysql.persistence.enabled	Allows disabling of persistence	true
mysql.persistence.accessMode	MySQL persistent volume access mode	ReadWriteOnce
mysql.persistence.size	MySQL persistent volume size	20Gi
mysql.persistence.storageClass	MySQL persistent volume storage class name
mysql.image.pullPolicy	MySQL PodSpec pullPolicy	IfNotPresent
mysql.image.imagePullSecrets	MySQL PodSpec imagePullSecrets
mysql.service.type	MySQL type of service	ClusterIP
mysql.service.port	MySQL port of service	3306
mysql.resources.requests.cpu	MySQL request CPU	250m
mysql.resources.requests.memory	MySQL request Memory	1000Mi
mysql.resources.limits.cpu	MySQL limit CPU
mysql.resources.limits.memory	MySQL limit Memory
mysql.podAnnotations	MySQL pod annotations
mysql.podSecurityContext	MySQL PodSpec securityContext
mysql.nodeSelector	MySQL PodSpec nodeSelector
mysql.tolerations	MySQL PodSpec tolerations
mysql.affinity	MySQL PodSpec affinity
redis.image.pullPolicy	Redis PodSpec pullPolicy	IfNotPresent
redis.image.imagePullSecrets	Redis PodSpec imagePullSecrets
redis.service.type	Redis type of service	ClusterIP
redis.service.port	Redis port of service	6379
redis.resources.requests.cpu	Redis request CPU	100m
redis.resources.requests.memory	Redis request Memory	128Mi
redis.resources.limits.cpu	Redis limit CPU
redis.resources.limits.memory	Redis limit Memory
redis.podAnnotations	Redis pod annotations
redis.podSecurityContext	Redis PodSpec securityContext
redis.nodeSelector	Redis PodSpec nodeSelector
redis.tolerations	Redis PodSpec tolerations
redis.affinity	Redis PodSpec affinity
worldvious.opt_out_version_check	Disables version check. As a part of the version check, Kubevious deployments are added to the leaderboard at https://worldvious.io. Reporting is anonymized to the nearest city/zip. No IP address is stored or logged. We calculate the SHA256 hash of the IP address and use it as a key. As a part of this request, we also added news notification and a feedback request mechanism.	false
worldvious.opt_out_error_report	Disables automatic exception and error reporting.	false
worldvious.opt_out_counters_report	Disables periodic reporting of cluster metrics, such as: number of nodes, pods, ingresses, configmaps, etc. The number of pods and nodes would appear on the https://worldvious.io leaderboard. Those are the same counters you would see in the console log of kubevious and parser pods.	false
worldvious.opt_out_metrics_report	Disables periodic reporting of internal time metrics. In conjunction with counters reporting, this would help identify internal bottlenecks and improve overall performance. Those are the same metrics you would see in the console log of kubevious and parser pods.	false
worldvious.opt_out_all	Opt out from all of the above reportings.	false

helm's Issues

arm64 support

Hi,
I am trying to install Kubevious with helm chart on EKS with arm64 node arch, I think Kubevious don't have arm support yet.
I am getting this error from the backend pod logs:

$ kubectl logs kubevious-backend-54b67c94-qcr62 -n kubevious
exec /usr/local/bin/docker-entrypoint.sh: exec format error

Support ingress v1

As of k8s 1.22 ingress is only available in v1.
It would be nice to add support for it.

cluster domain is hardcoded

The default cluster domain cluster.local is hardcoded in the templates.
This breaks deployment on clusters where this domain was customized.
The frontend ui shoiws an unmeaningfull 502 erro.
The ui pods logs a message like

2020/07/05 13:16:27 [error] 13#13: *4 kubevious-svc.kubevious.svc.cluster.local could not be resolved (3: Host not found), client: 127.0.0.1, server: , request: "GET /socket/?EIO=3&transport=polling&t=NCV7km9 HTTP/1.1", host: "localhost:3000", referrer: "http://localhost:3000/?tme=false&tmdt=U3VuIEp1bCAwNSAyMDIwIDE1OjE2OjA5IEdNVCswMjAwIChDZW50cmFsIEV1cm9wZWFuIFN1bW1lciBUaW1lKQ==&tmd=MjQ=&tmdaf=U2F0IEp1bCAwNCAyMDIwIDE1OjE2OjA5IEdNVCswMjAwIChDZW50cmFsIEV1cm9wZWFuIFN1bW1lciBUaW1lKQ=="

If you just leave out the cluster.local part, Kubernetes DNS resolving will find the right ip without you needing to know the cluster domain name.

MYSQL password not handled by mysql statefulset

Just followed the install instructions using helm chart.

i've configured a password for mysql connection and kubevious loads it properly

  envFrom:
     - configMapRef:
          name: {{ .Values.mysql.name }}-client

but this password is not configured into the mysql statefulset.

   env:
        - name: MYSQL_DATABASE
          value: kubevious
        - name: MYSQL_ALLOW_EMPTY_PASSWORD
          value: "1"

This cause an error on kubevious stating Error 500: Not Connected

UI service is created as a NodePort even if you specify ClusterIP in values.yaml

When I tried to install from the official helm repo, the ingress never worked so I downloaded the entire chart and found the following. It seems like if the ingress.type is ClusterIP then we would not want to return NodePort as the type for the service. After fixing this to us ne instead of eq, it installed correctly and I was able to use the ingress.

helm/kubernetes/templates/_helpers.tpl

Line 196 in c9091cb

{{- if (eq .Values.ui.service.type "ClusterIP") }}

Nginx Ingress Issues

I am having an issue getting the UI working in my cluster. I am running in GCP and already have an nginx ingress that I am trying to re-use for Kubevious. I can get Kubevious to create its own ingress which creates a new GCP load balancer but I can't seem to get it to use the existing nginx.

I have added this to my ingress:

• host: kubevious.example.com
  http:
  paths:
  • path: /
    backend:
    serviceName: kubevious-ui-clusterip
    servicePort: 80

I have tried several installation options but here is where I am right now:
helm upgrade --atomic -i -n kubevious
--version 0.8.15
--set ingress.domain=example.com
kubevious kubevious/kubevious

When hitting the URL I get a 503 Service Temporarily Unavailable from nginx. I'm sure this is something silly I am overlooking.

Any assistance would be greatly appreciated.

Provide `global.affinity` and `global.tolerations` options to apply for all kubevious components

It will be convenient to have global.affinity and global.tolerations option so that we don't have to define them at each component
example:

    global: {
      affinity: {
        nodeAffinity: {
          requiredDuringSchedulingIgnoredDuringExecution: {
            nodeSelectorTerms: [{
              matchExpressions: [{
                key: 'deployment',
                operator: 'In',
                values: ['my-affinity'],
              }],
            }],
          },
        },
      },
      tolerations: [{
        effect: 'NoSchedule',
        key: 'dedicated',
        operator: 'Equal',
        value: 'my-toleration',
      }],
    },

use bitnami charts as dependencies

Suggestion for improvement and simplification

Instead of defining MySQL and Redis "charts" as subcharts just use existing charts from Bitnami:

• https://artifacthub.io/packages/helm/bitnami/mysql
• https://artifacthub.io/packages/helm/bitnami/redis

storageClass set, PVC pending

If I set the storage class like this:
helm upgrade --atomic -i kubevious kubevious/kubevious --version 0.5.9 -n kubevious --set mysql.storageClass=slow

I would expect the pv to be created and pvc bound to it. But this happens:

kubectl describe pvc -n kubevious
Name:          data-kubevious-mysql-0
Namespace:     kubevious
StorageClass:
Status:        Pending
Volume:
Labels:        k8s-app=kubevious-mysql
Annotations:   <none>
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode:    Filesystem
Mounted By:    kubevious-mysql-0
Events:
  Type    Reason         Age                        From                         Message
  ----    ------         ----                       ----                         -------
  Normal  FailedBinding  4m54s (x16624 over 2d21h)  persistentvolume-controller  no persistent volumes available for this claim and no storage class is set

Am I missing something?

Support TLS for Ingress

Hey guys,

Thanks for looking into the annotations for ingress. With the testing I ran into an issue where using a custom host name you will always get the default 404 backend. For our ingresses, we need to use tls in order to get it to work properly.

For our fix we edited the ingress and added the following stanza:

tls:
  - hosts:
    - example.com
    secretName: kubevious-tls

Again, would most likely be an optional parameter to add your tls in with some default values.

Examples:
https://github.com/helm/charts/blob/master/stable/jenkins/templates/jenkins-master-ingress.yaml#L33
https://github.com/helm/charts/blob/master/stable/mediawiki/templates/ingress.yaml#L31
https://github.com/elastic/helm-charts/blob/master/elasticsearch/templates/ingress.yaml#L18

Reference:
kubernetes/ingress-nginx#3971

Kindly update the mysql version as 8.0.22 not available any more

Kindly update the mysql version as 8.0.22 is not available anymore suggesting to change it to 8.0.30

helm/kubernetes/values.yaml

Line 209 in e778006

tag: 8.0.22

kubevious-ui crashed

 helm upgrade --install  --version=0.6.14 --namespace kubevious kubevious kubevious/kubevious  --set mysql.storageClass=openebs-cstor-sparse

Log from kubevious-ui pod:

2020/06/13 13:44:30 [emerg] 8#8: host not found in upstream "kubevious-svc.kubevious.svc.cluster.local" in /etc/nginx/conf.d/default.conf:15
nginx: [emerg] host not found in upstream "kubevious-svc.kubevious.svc.cluster.local" in /etc/nginx/conf.d/default.conf:15

livenessProbe problem with Collector

Hello,

When I install helm chart in my cluster, collector pod can start and stay in CrashLoopBackOff state .

After investigation, periodSeconds in livenessProbe section is too short and is not editable from chart values, so I have to manually edit the deployment to increase this value.

Defaults values:

    livenessProbe:
      failureThreshold: 3
      httpGet:
        path: /
        port: http
        scheme: HTTP
      periodSeconds: 10
      successThreshold: 1
      timeoutSeconds: 1
    name: kubevious
    ports:
    - containerPort: 4002
      name: http
      protocol: TCP
    readinessProbe:
      failureThreshold: 3
      httpGet:
        path: /
        port: http
        scheme: HTTP
      periodSeconds: 10
      successThreshold: 1
      timeoutSeconds: 1

Is it possible to add the possibility to control the livenessProbe and readinessProbe from the values ​​of the helm charts?

Best regards

unable to change the storage class of mysql

There is no way in current release to change the storage class of the PVC created for the Mysql include inside of kubevious.
If no storageclass default is defined in a K8S the deployment failed as the PVC stays in pending mode

Unused fields from mysql.external

It looks like the helm chart do not use any of the fields under mysql.external to allow kubevious to connect to already existed mySQL deployment. Even with that flag set to enabled and provided all the mysql auth credentials, the deployment will still create the mysql pod.

NAME                                    READY   STATUS    RESTARTS   AGE
pod/kubevious-6d4d4c94f5-nvl6v          1/1     Running   0          8m21s
pod/kubevious-mysql-0                   1/1     Running   0          8m21s
pod/kubevious-parser-75c6f7d65d-rbfcc   1/1     Running   0          8m21s
pod/kubevious-ui-5856b6b864-bqqfd       1/1     Running   0          8m21s

Following is a quick grep for the external. key.

~/helm$ grep -r external.
README.md:| mysql.external.enabled | Indicates whether an existing MySQL database should be used. When enabled a new MySQL database would not be deployed. | false |
README.md:| mysql.external.host | Host for external MySQL server | |
README.md:| mysql.external.port | Port for external MySQL server | |
README.md:| mysql.external.database | Database name for external MySQL server. The database should be manually created. | |
README.md:| mysql.external.user | User name. User should have access to the database specified above. | |
README.md:| mysql.external.password | Password. | |
kubernetes/values.yaml:  external:

BUG: mysql ReadinessProbe and LivenessProbe fail

Readiness probe failed: mysql: [Warning] Using a password on the command line interface can be insecure. ERROR 1045 (28000): Access denied for user 'UNKNOWN_USER'@'localhost' (using password: YES)

BUG: UI-Pod not starting with SecurityContext set

When using SecurityContext and PodSecurityContext to run the ui-pod as nonRoot, i receive the following error in the pod logs:

sed: can't create temp file '/etc/nginx/conf.d/default.confXXXXXX': Permission denied
sed: can't create temp file '/etc/nginx/conf.d/default.confXXXXXX': Permission denied
2021/03/30 14:36:27 [warn] 9#9: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
2021/03/30 14:36:27 [emerg] 9#9: host not found in resolver "$DNS_SERVERS" in /etc/nginx/conf.d/default.conf:12
nginx: [emerg] host not found in resolver "$DNS_SERVERS" in /etc/nginx/conf.d/default.conf:12

The easiest solution to make this run in a cluster with PodSecurityPolicies activated woule be to be able to set the serviceaccount under which the pod should be started and give this serviceaccount the privilege to run as root.