scientanl / k8s-proxysql-cluster

Kubernetes ProxySQL Cluster

k8s-proxysql-cluster

Container to run proxysql in cluster mode in kubernetes.

Usage

By default te the container acts as a normal proxysql instance. To enable cluster mode the following is needed.

Service

the container uses a service to discover if there are already master nodes up. This service can be found in

/k8s/proxysql.service.yaml

use the following command to use the service in kubernetes

kubectl create -f ./k8s/proxysql.service.yaml

Statefulset

To deploy the actual nodes a statefulset should be added to the kubernetes cluster This statefulset can be found in

/k8s/proxysql.statefulset.yaml

use the following command to use the statefulset in kubernetes

kubectl create -f ./k8s/proxysql.statefulset.yaml

Helm Install

You can install using the helm repository using the following command:

helm install --name db-proxy deploy/charts/proxysql-cluster 

Parameter	Description	Default
image.repository	proxysql image repo	scienta/k8s-proxysql-cluster
image.tag	proxysql image tag	1.0.0
numReplicas	Number of replicas to create in StatefulSet	3
proxysql.admin.username	Admin username for proxysql	admin
proxysql.admin.password	Admin password for proxysql	admin
proxysql.admin.iface	Listen network for proxysql service	0.0.0.0
proxysql.admin.port	Listen port for proxysql service	6032
proxysql.clusterAdmin.username	Cluster user username used by proxysql nodes to sync	cluster1
proxysql.clusterAdmin.password	Cluster user password used by proxysql nodes to sync	secret1pass
proxysql.queryCacheSizeMb	(Optional) Query cache size	nil
proxysql.dataDir	Directory to store proxysql tables, etc.	/var/lib/proxysql
proxysql.webEnabled	Enable proxysql web dashboard	true
mysql.iface	Listen network for mysql service connections	0.0.0.0
mysql.port	Listen port fo mysql service connections	3306
mysql.monitor.username	Monitor username on MySQL instances for proxysql health checks	monitor
mysql.monitor.password	Monitor password on MySQL instances for proxysql health checks	monitor
mysql.admin.username	Root / admin username on MySQL instances	root
mysql.admin.password	Root / admin password on MySQL instances	insecurepassword
cronjob.enabled	Enable k8s CronJob to set the default hostgroup to the backend having the least # databases	false
resources	CPU / Memory Limits and Requests	{}
tolerations	Pod tolerations	'{}'

Workings

When the container is run with the --k8s-cluster argument the following happens:

• the container checks if there is a node available at the proxysql service

• If no node is available this container becomes the first master node of the cluster

  • The node deletes the proxysql server from the proxysql_servers table
  • The node adds its own ip to its proxysql_servers table

• Else the node wil join the existing master nodes

  • The node's proxysql_servers table is synced with that of the existing master nodes
  • The node adds its own ip to the proxysql_servers table of the node exposed by the proxysql service.
  • The node deletes the proxysql service from the proxysql_servers table

• The cluster is up and running!

k8s-proxysql-cluster's Issues

Add proxysql-admin tool

Would it make sense to integrate with Percona's proxysql-admin tool so that proxysql-cluster works with Galera / PXC?

Make admin user and admin password variable

There are no pvc created

Why there are no pvc created ?? is this stateless proxysql mysql cluster ??

Add option to specify other other options in proxysql.cnf

Hi there!

There should be an option to specify other config stuff like mysql_servers to be used in the ConfigMap through values.yaml.

Invalid indent syntax in helm chart

Description

The helm chart is using incorrect syntax to indent resources and tolerations.

Needs to be prefixed with toYaml.

Better init done check in init-k8s-cluster.sh

The init done of a node still depends on some sleeps. This could probably do with some improvement.

Option to use own proxysql.cnf file

Update readme

• describe proxysql-cli
• update the part about inner working
• add helm deployment

Pods do not start on deploy

I simply deployed the helm chart with:
cronjob:
enabled: false

Deployment: On OpenShift
The pods do not start, it simply goes to "Completed" state.

[root@bastion nesh]# kubectl get po -n test2
db-proxy-proxysql-cluste-0 0/1 Completed 1 17s

[root@bastion nesh]# kubectl logs -n test2 db-proxy-proxysql-cluste-0
2020-12-01 09:14:12 [INFO] Using config file /etc/proxysql.cnf
Renaming database file /var/lib/proxysql/proxysql.db
2020-12-01 09:14:12 [INFO] ProxySQL version 1.4.14-28-g0f6701c5
2020-12-01 09:14:12 [INFO] Detected OS: Linux db-proxy-proxysql-cluste-0 4.18.0-147.8.1.el8_1.x86_64 #1 SMP Wed Feb 26 03:08:15 UTC 2020 x86_64
Standard ProxySQL Cluster rev. 0.4.0906 -- ProxySQL_Cluster.cpp -- Fri Jan 25 16:04:59 2019
2020-12-01 09:14:12 sqlite3db.cpp:45:open(): [ERROR] SQLITE CRITICAL error: Unable to open /var/lib/proxysql/proxysql_stats.db. Shutting down.

Update proxysql node peers when statefulset scaled down

Description

Currently, when the StatefulSet is scaled down or pods are terminated, the ProxySQL node peers continuously attempt to sync with the missing pods.

How to reproduce

Create a new cluster:

helm install --name db-proxy deploy/charts/proxysql-cluster

Wait for all 3 pods to be up and running:

❯ kubectl get pods -l app=db-proxy-proxysql-cluste
NAME                         READY   STATUS    RESTARTS   AGE
db-proxy-proxysql-cluste-0   1/1     Running   0          3m49s
db-proxy-proxysql-cluste-1   1/1     Running   0          3m30s
db-proxy-proxysql-cluste-2   1/1     Running   0          27s

Query the proxysql nodes:

 1 ❯ kubectl exec -ti db-proxy-proxysql-cluste-0 -- proxysql-cli query:nodes
+-------------+------+--------+-------------+
| hostname    | port | weight | comment     |
+-------------+------+--------+-------------+
| 172.17.0.31 | 6032 | 0      | 172.17.0.31 |
| 172.17.0.26 | 6032 | 0      | 172.17.0.26 |
| 172.17.0.27 | 6032 | 0      | 172.17.0.27 |
+-------------+------+--------+-------------+

Scale down the StatefulSet to 2 replicas:

❯ kubectl patch sts db-proxy-proxysql-cluste --patch='{"spec":{"replicas":2}}'

Query the proxysql nodes again:

❯ kubectl exec -ti db-proxy-proxysql-cluste-0 -- proxysql-cli query:nodes
+-------------+------+--------+-------------+
| hostname    | port | weight | comment     |
+-------------+------+--------+-------------+
| 172.17.0.31 | 6032 | 0      | 172.17.0.31 |
| 172.17.0.26 | 6032 | 0      | 172.17.0.26 |
| 172.17.0.27 | 6032 | 0      | 172.17.0.27 |
+-------------+------+--------+-------------+

proxysql_servers still contain all nodes and will result in noisy logs on remaining pods:

2019-04-10 08:21:30 ProxySQL_Cluster.cpp:215:ProxySQL_Cluster_Monitor_thread(): [WARNING] Cluster: unable to connect to peer 172.17.0.27:6032 . Error: Lost connection to MySQL server at 'handshake: waiting for inital communication packet', system error: 110
2019-04-10 08:21:31 ProxySQL_Cluster.cpp:215:ProxySQL_Cluster_Monitor_thread(): [WARNING] Cluster: unable to connect to peer 172.17.0.27:6032 . Error: Lost connection to MySQL server at 'handshake: reading inital communication packet', system error: 107

...to infinity and beyond...

How to fix

Add preStop pod lifecycle hook to remove the terminating pod from proxysql_servers in the cluster.

License information

Currently there is no license information in the repository. Could you possibly add it?

Create helm chart

sync command duplicates query rules

Description

When you run proxysql-cli sync, a complete duplicate set of query rules are inserted.

Can anyone confirm they are having the same issue?