pixiu-io / kubez-ansible

To provide quick deployment tools for kubernetes cluster and cloud native application by ansible

Kubez-ansible Overview

To provide quick deployment tools for kubernetes cluster and cloud native application

This session has been tested on Rocky 8.5, Debian 11 and Ubuntu 20.04+ which supported by python3.

More distribution supported see more

Getting Started

Learn about Kubez Ansible by reading the documentation online kubez-ansible.

Supported Components

• 集群指南

  • 单节点集群 单节点集群的快速部署
  • 多节点集群 多节点集群部署 (1master + 多node)
  • 高可用集群 高可用集群部署 (3master + 多node)
  • 扩容
  • 销毁

• 网络插件

  • flannel
  • calico

• 容器运行时

  • docker
  • containerd

• 存储插件

  • NFS 文件存储
  • Ceph 块存储
  • MinIO 对象存储

• 云原生应用

  • 基础应用
    • Helm3
    • Nginx Ingress
    • Dashboard
    • Metrics Server
    • MetalLB
    • Cilium&Hubble
  • 日志监控
    • Loki
    • Grafana
    • Promtail
    • Prometheus
  • 中间件
    • OLM
    • PostgreSQL
    • Redis
    • Kafka
    • RabbitMQ
    • MongoDB
  • 微服务
    • Isito
  • CICD
    • Tekton
    • Jenkins
    • Harbor
    • Jfrog-Artifactory
  • 健康检查
    • Kuberhealthy
  • Certificates
    • Cert-manager
  • 混沌工程
    • ChaosMesh

• 自研云原生

  • Pixiu
  • Localstorage
  • Pixiu-autoscaler
  • PodSet

• 组件集成手册

  • 组件集成

学习分享

• go-learning

沟通交流

• 搜索微信号 yingjuncz, 备注（github）, 验证通过会加入群聊
• bilibili 技术分享

Copyright 2019 caoyingjun ([email protected]) Apache License 2.0

kubez-ansible's Issues

修改 all-in-one 默认的 cri 为 containerd

Kubeconfig配置文件，没有指定外网地址地址。

Kubeconfig配置文件，没有指定外网地址地址，需要增加md说明

启动coredns失败，usage per Container is 90Mi, but request is 70Mi

Error creating: pods "coredns-546565776c-f6r98" is forbidden: minimum memory usage per Container is 90Mi, but request is 70Mi
62s Warning FailedCreate replicaset/coredns-546565776c Error creating: pods "coredns-546565776c-dhbcc" is forbidden: minimum memory usage per Container is 90Mi, but request is 70Mi

1.24 之后，"master" label and taint keys to "control-plane"

宿主机存在时间不同步问题，建议使用ntp或者chrony处理

centos 7.8 在安装 helm 的时候异常： cp 的时候存在大文件问题

TASK [kubernetes : Apply helm3 command immediately for master nodes(the default timeout is 600s)] **********************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "non-zero return code", "rc": 1, "stderr": "error: unexpected EOF\nfailed to copy helm from helm-toolbox\n", "stderr_lines": ["error: unexpected EOF", "failed to copy helm from helm-toolbox"], "stdout": "helm-toolbox-0 1/1 Running 0 21m\nDropping out copy after 0 retries\n", "stdout_lines": ["helm-toolbox-0 1/1 Running 0 21m", "Dropping out copy after 0 retries"]}

升级 helm 版本到3.9.0

文档缺失较多

application consul is needed but missing

bump the default kubernetes version to 1.23.6

metrics-server started failed

pods "metrics-server-v0.3.6-cbb9b8dff-gvn87" is forbidden: [minimum cpu usage per Container is 100m, but request is 5m, minimum memory usage per Container is 90Mi, but request is 50Mi]

ubantu版本兼容性问题

只支持centos系统，不支持其他系统，比如Ubuntu,希望英俊的楼主能新出一个支持Ubuntu版本的工具

优化 helm_toolbox 模块

1. helm_toolbox 模块支持 helm repo 的幂等增加，等同于 helm repo add xx xxx
2. helm_toolbox 模块 支持指定 chart的幂等安装，等同于 helm install release xx -n ns --set k1=v1 --set k2=v2 ...
3. 支持幂等 卸载，更新等功能

启用云平台的lb时，init 集群之后需要登录lb生效，否则会中断部署

harbor is missing but needed

清理 gerrit 的支持

清理 gerrit 的支持， gerrit 用的不多，不在提供维护

没有运维功能-etcd的备份与恢复

比方说etcd的备份与恢复，etcd的迁移等功能

支持 operator 生命周期管理

helm_toolbox 模块支持 hook 功能

helm_toolbox 模块支持 hook 功能，用于支持在执行前或者后执行指定的程序

高可用时，节点是 master 也是node时，taint无效

通过 helm 模块安装 efk

优化 postgres 中间件配置参数

优化 redis 中间件配置参数

nfs 安装的时候，因为版本问题，需要手动修改 apisever的配置，建议自动化

基础组件 olm 首次安装失败

TASK [kubectl-apply : Apply kubernetes application | olm] ********************************************************************************************************************************************************************************************************************
fatal: [localhost -> localhost]: FAILED! => {"changed": true, "msg": "'Traceback (most recent call last):\n File "/tmp/ansible_kube_toolbox_payload_4xLles/ansible_kube_toolbox_payload.zip/ansible/modules/kube_toolbox.py", line 297, in main\n File "/tmp/ansible_kube_toolbox_payload_4xLles/ansible_kube_toolbox_payload.zip/ansible/modules/kube_toolbox.py", line 265, in run\n File "/tmp/ansible_kube_toolbox_payload_4xLles/ansible_kube_toolbox_payload.zip/ansible/modules/kube_toolbox.py", line 88, in _run\nCalledProcessError: Command \'kubectl apply -f /tmp/pixiuspace/olm.yml\' returned non-zero exit status 1\n'"}

优化 kafka 中间件配置参数

Ingress对外提供服务没有vip

需要有个vip对外提供服务，达到node节点挂了不影响客户使用

建议支持python3

优化 helm 的安装模式，helm_toolbox 依赖cni的完成，效率太低

执行kubez-ansible的时候，如果出现yes，则忽略直接执行

1. 执行kubez-ansible的时候，如果出现yes，则忽略直接执行
2. 选择如果已经做过免密，但是没有登录过，在执行的时候，会有卡住的情况

启动metrics-server失败usage per Container is 100m, but request is 5m

pods "metrics-server-v0.3.6-cbb9b8dff-gvn87" is forbidden: [minimum cpu usage per Container is 100m, but request is 5m, minimum memory usage per Container is 90Mi, but request is 50Mi]

添加 hpa-operator

提供自动 hpa 功能

执行helm命令的时候，会有warnning 告警

[root@kubez~]# helm list -A

WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION

重构文档的结构，增加可读性

优化 rabbitmq 中间件配置参数

验证 grafana + loki + promail 的开箱即用和文档验证

启用 harbor chart 的安装和优化

建议新增ingress的安装文档，和场景配置调整

ubuntu 在update cache的时候，因为缺少helm的key而失败

优化 helm 命令行的安装模式，通过 kubectl cp 屏蔽不同 cri 的差异

新增 istio组件支持

依赖 helm_toolbox 的新功能

支持最新版本 istio的一键安装

提供外挂式的tools sidercaer 容器

优化：使用下载zip包的方式下载代码

验证istio的可用性

istio-gateway 还没支持

helmv3 安装时未等待pod创建就直接获取状态

TASK [kubernetes : Apply helm3 command immediately for master nodes] *********************************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "non-zero return code", "rc": 1, "stderr": "Error from server (NotFound): pods "helm-toolbox-0" not found\nError from server (NotFound): pods "helm-toolbox-0" not found\nError from server (NotFound): pods "helm-toolbox-0" not found\nError from server (NotFound): pods "helm-toolbox-0" not found\nError from server (NotFound): pods "helm-toolbox-0" not found\nError from server (NotFound): pods "helm-toolbox-0" not found\nError from server (NotFound): pods "helm-toolbox-0" not found\nError from server (NotFound): pods "helm-toolbox-0" not found\nfailed to wait helm-toolbox up\n", "stderr_lines": ["Error from server (NotFound): pods "helm-toolbox-0" not found", "Error from server (NotFound): pods "helm-toolbox-0" not found", "Error from server (NotFound): pods "helm-toolbox-0" not found", "Error from server (NotFound): pods "helm-toolbox-0" not found", "Error from server (NotFound): pods "helm-toolbox-0" not found", "Error from server (NotFound): pods "helm-toolbox-0" not found", "Error from server (NotFound): pods "helm-toolbox-0" not found", "Error from server (NotFound): pods "helm-toolbox-0" not found", "failed to wait helm-toolbox up"], "stdout": "", "stdout_lines": []}

新增pgsql的中间件支持

新增中间件redis的支持

提供一个toolbox的镜像，里面安装了各种工具和客户端，方便调试

kubez destroy报语法错误

TASK [kubernetes : include_tasks] ****************************************************************************************************************************
fatal: [localhost]: FAILED! => {"reason": "the role 'loadbalancer' was not found in /usr/share/kubez-ansible/ansible/roles:/root/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/usr/share/kubez-ansible/ansible\n\nThe error appears to be in '/usr/share/kubez-ansible/ansible/roles/kubernetes/tasks/destroy.yml': line 26, column 11, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n import_role:\n role: loadbalancer\n ^ here\n"}

helm 模块 安装的部分代码复用度比较低，需抽象优化