koderover / zadig

Zadig is a cloud native, distributed, developer-oriented DevOps platform

Developer-oriented, self-service, cloud-native DevOps platform

⁣

English | 简体中文

Table of Contents

• Zadig
  • Table of Contents
  • What is Zadig
  • Quick start
    • How to use?
    • How to make contribution?
  • Getting help
  • More resources
  • License

What is Zadig

Zadig is a self-service, cloud-native DevOps platform based on Kubernetes developed by KodeRover. Zadig is available in Basic and Professional editions, and offers flexible and scalable workflow support, multiple release policy orchestration, and one-click security auditing. The platform also supports customized enterprise XOps agile performance dashboards, deep integration with multiple enterprise platforms, and rapid batch access through project templating to achieve one-click governance of thousands of services.

Its main goal is to help enterprises realize the digital transformation of production and research, enable engineers to become innovation engines, and provide support for the unlimited value links of the digital economy.

Our vision is: Developer + Zadig = Business success

The business architecture is as follows:

For more details about system architecture, see Zadig System Architecture Overview.

The Highlighted Features:

High Concurrency

Based on cloud-native design, through simple configuration, the system automatically generates workflows to achieve high concurrent execution for continuous delivery relevant tasks such as building, testing and deployment, across multiple services. It significantly improves the efficiency of multi-services deployment in microservice architecture.

Service-oriented Environment

With just one set of service configuration, multiple encapsulated environments will be provided automatically within minutes, empowering independent environments for developers, QAs and product managers.

Minimum to none migration cost of existing environments -- just hosting with one click, the system allows browsing and adjusting all the services at your fingertips.

Non-intrusive Testing Automation

Zadig can easily and non-intrusively embed existing testing automation frameworks, and achieve continuous building, testing and deployment via GitHub/GitLab Webhook.

It also integrates with productivity bots to provide instant quality report, which effectively applies shift-left testing best practices.

Powerful Template Library

Shared K8s YAML templates, Helm Chart templates, build templates, etc., across projects to achieve unified configuration management. Based on a set of templates, hundreds of microservices can be created, and with minimal configuration by development engineers, self-service usage is enabled, significantly reducing the burden of operations and maintenance management.

Secure and Reliable Release Management

Customized workflows that integrate engineer, process, and internal and external system compliance approvals, supporting flexible orchestration of deployment strategies such as blue-green deployment, canary release, phased gray release, and Istio release. Presenting the state of the production environment from a multi-cluster, multi-project perspective, it achieves transparency and reliability in the release process.

Stable and Efficient Customer Delivery

Simplify the implementation process for vendors to customer versions, customer private clouds, public clouds, and offline environments, as well as product license management. The supplier management plane collaborates with the customer console to complete the implementation, update, and maintenance process for customer environments, enhancing the quality of external services provided by the enterprise.

Objective and Accurate Performance Insights

Comprehensive understanding of the system's operational status, including data overviews on clusters, projects, environments, workflows, and key process pass rates. It provides objective performance metrics data for various aspects like builds, tests, and deployments at the project level, enabling precise analysis of development efficiency bottlenecks and promoting steady improvement.

Convenient Development CLI/IDE Plugin

Zadig also provides a convenient toolkit with development commandline interface which allows compiling, building and deploying the changes to dev environment with one command. It enables collaborated debugging and testing with minimum manual toil, reduces cognitive load and allows teams to focus more on business.

Quick start

How to use?

Please follow Quick Start

Bootcamps

With many hands-on tips, case studies and demos of different application types, Zadig bootcamp gives developers easy to follow best practices in a cloud native CI/CD solution

Just follow Tutorials to try it step by step.

How to make contribution?

Please check out our contributing guideline.

Getting help

• More about Zadig, see here
• Submit bugs or feature requests following contributing instructions
• Email：[email protected]
• Slack channel

License

Apache 2.0 License

zadig's Issues

Error: failed to download "koderover-chart/storage"

error: error executing jsonpath "{.items[0].spec.clusterIP}": Error executing template: array index out of bounds: index 0, length 0. Printing more information for debugging the template:
template was:
{.items[0].spec.clusterIP}
object given to jsonpath engine was:
map[string]interface {}{"apiVersion":"v1", "items":[]interface {}{}, "kind":"List", "metadata":map[string]interface {}{"resourceVersion":"", "selfLink":""}}

it is fail that set up the new user.

/kind bug

What steps did you take and what happened:
I install it with k8s

http://ip:30001/setup

I create new user with the form.

What did you expect to happen:

500 : Can't add more than one organization

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

architecture overview needed

/kind enhancement

An overview of Zadig architecture would help potential contributors a lot.

A typical overview of architecture could includes the following sections or more:

• Components overview/diagram including components in Zadig and third-party dependencies
• Introduction of each component including the role, functions and sub components
• important protocols between components
• deployment layout

What should I do if I forget my admin password?

/kind question

General Question

管理员密码忘记了，现在登陆不进去了

关于离线部署

当前提供的三种部署方式均为在线部署，请问如果想离线部署的话，有什么好的建议？

could you tell what support kubernetes version ?

/kind question

General Question

could you tell what support kubernetes version ?

when i integrate github oauth that problems arose

/kind bug

the clientid Unable to enter '1e6f6a085a3187c85a03'

eg.
Client ID
1e6f6a085a3187c85a03

install error

when run ./install_with_k8s.sh
Error: Kubernetes cluster unreachable: Get "http://localhost:8080/version?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused

about gitlab Authorize

/kind question

General Question

环境：
centos7.4，
zadig一键脚本安装

gitlab内网部署 版本13.9

集成gitlab失败（gitlab地址映射到公网之后也是失败）

Does Zadig support deployment on ARM?

Zadig支持在ARM上部署吗？

Support ordering rules for pipelines or other similar lists.

/kind feature

Is your feature request related to a problem? Please describe.
Now we can't sort our pipelines by other ordering rules except create time.

Describe the feature you'd like
Currently, we need sort by name. But I think maybe we can think it over to support more common scenes.

Integrate with golint

/kind enhancement

To keep improving Zadig's source code quality. Should we consider to integrate with some linters ? For example: https://github.com/golangci/golangci-lint

If the build step support pause to check result of step, and then trigger by manual？

/kind question

General Question

It is difficult for k8s beginners to add services

Is your feature request related to a problem? Please describe.

It is troublesome to add services, because k8s yaml has many configuration items and is prone to errors. For those who are not familiar with k8s, it is more difficult to use

Describe the feature you'd like

In the future, we will provide custom service template feature to reduce the difficulty of service configuration

@nighca

给大家贡献一个测试环境，有想体验的可以来试一试

http://47.94.168.23:30001/

用户名 ： [email protected]
密码：aa123456

The domain has been set, why can't I use the domain? thanks

无法使用域名访问，我要怎么查看设置对不对呢？

gogs integration support

/kind feature

I hope to support the GIT warehouse of gogs. Gitlab is too heavy. We have used gogs for more than 3 years. Thank you

请问，为啥微服务版的看不到，创建项目的时候里面没有微服务选项呀？谢谢

/kind question

General Question

The real-time log loading of the service is very slow

Is your enhancement proposal related to a problem? Please describe.

The real-time log loading of the service is very slow, even unable to display

Describe the solution you'd like

Zadig Team:

The efficiency has been optimized, and the performance is OK under a certain amount of data. If the amount of data is too large, we do not recommend using the page to view real-time logs. In the future, we will strengthen the Kodespace CLI function to support the situation of excessive data.

main 代码运行不了

go 版本

go version go1.16.4 darwin/amd64

执行命令

go mod tidy
报下面的错误

请问

github.com/koderover/zadig/pkg/types/pipeline 这个包哪去了；
这个项目是真开源的吗？？？

zadig-portal web也无法运行

node版本

v16.2.0

运行

yarn install
yarn start
报错：

不会是来骗星星的吧

The start and end of the build are not fast enough

Is your enhancement proposal related to a problem? Please describe.

The start and end of the build are not fast enough.

Describe the solution you'd like

Factors affecting efficiency: k8s resources and current feature implementation. The current bottleneck is that the system time depends on the k8s startup and resource destruction time, so there is room for optimization.

@xen0n

Is there a trial key for zadig enterprise version?

Is there a trial key for zadig enterprise version?

uninstall script delete the namespace, and no ack

zadig 安装在运维专用的namespace，卸载的时候没有提示，直接把ns删除了。这里需要做个提示，或者不要直接删除ns

A small bug when switching code platforms.

/kind bug

切换代码平台时，如果请求速度较慢，拥有者一栏还是切换前平台的选项，正常逻辑应该是切换的时候拥有者一栏清空？

Unable to pull github code

More richable feishu notification

/kind enhancement

Currently, feishu supports more richable bot notification. Refer to : https://open.feishu.cn/document/ukTMukTMukTM/uMDMxEjLzATMx4yMwETM?op_tracking=hc.

Maybe we can improve this point.

添加外部集群后，环境无法删除

Path: zh_cn/10.系统设置/09.cluster_manage.md

ldap integration failure

/kind bug
1.2.0版本
在系统设置用户账户集成添加 ldap，填写完成后点击测试并保存报错：400 : AD auth failed
openldap server 运行正常
如果选择 ad，能保存成功但同步用户信息失败

The quality efficiency center was not found?

/kind feature

Is your feature request related to a problem? Please describe.
[A clear and concise description of what the problem is. Ex. I'm always frustrated when ... ]

Describe the feature you'd like
[A clear and concise description of what you want to happen.]

Describe alternatives you've considered
[A clear and concise description of any alternative solutions or features you've considered.]

Additional context
[Add any other context or screenshots about the feature request here.]

Minio issue after upgraded to 1.3.0

/kind bug

我之前是用了自定义的 Minio 组件，升级到1.3.0之后每次构建完成后界面都报”saveContainerLog s3 Upload error: RequestError: send request failed caused by...“ 看起来是 minio 的url规则变了，以前是 "http://接入点地址//...", 现在变成了”http://.接入点地址/", 所以请求失败。请问如何解决

Resource quotas and default resource request/limit support for projects/namespaces

/kind feature

Thanks @hoozecn for the advice #156 . Welcome to contribute PR for this issue 😊

Is your feature request related to a problem? Please describe.

If your namespace has a resource quota, it is helpful to have a default value in place for memory limit and cpu limit.

Describe the feature you'd like

The k8s resource limit and request quota can be configured via the Zadig UI.

[bug or question] Cannot install in kind because of the PVC

/kind bug

What steps did you take and what happened:
[A clear and concise description of what the bug is.]

kind create cluster
export KUBECONFIG=~/.kube/config
curl -SsL https://github.com/koderover/zadig/releases/latest/download/install.sh | bash

kubectl -n zadig describe pvc
Name:          zadig-nfs-server
Namespace:     zadig
StorageClass:  standard
Status:        Pending
Volume:
Labels:        app.kubernetes.io/instance=zadig-nfs-server
               app.kubernetes.io/managed-by=Helm
               app.kubernetes.io/name=nfs-server
               helm.sh/chart=nfs-server-0.9.5
Annotations:   volume.beta.kubernetes.io/storage-provisioner: rancher.io/local-path
               volume.kubernetes.io/selected-node: kind-control-plane
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode:    Filesystem
Used By:       zadig-nfs-server-858477dfb7-fpdx2
Events:
  Type     Reason                Age                     From                                                                                                Message
  ----     ------                ----                    ----                                                                                                -------
  Normal   ExternalProvisioning  4m29s (x523 over 134m)  persistentvolume-controller                                                                         waiting for a volume to be created, either by external provisioner "rancher.io/local-path" or manually created by system administrator
  Normal   Provisioning          2m56s (x14 over 134m)   rancher.io/local-path_local-path-provisioner-78776bfc44-9cz45_91e5ec11-ec4b-45b9-9057-38f5c580f26b  External provisioner is provisioning volume for claim "zadig/zadig-nfs-server"
  Warning  ProvisioningFailed    2m56s (x14 over 134m)   rancher.io/local-path_local-path-provisioner-78776bfc44-9cz45_91e5ec11-ec4b-45b9-9057-38f5c580f26b  failed to provision volume with StorageClass "standard": Only support ReadWriteOnce access mode

What did you expect to happen:

The system is deployed successfully.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

pod “zadig-mongodb-865f554c8-wwmv9” status always “CrashLoopBackOff”，detail Message is “Back-off restarting failed container”and Exit Code is “100”，how figure out？

Tried https://github.com/koderover/zadig/releases/download/v1.2.0/install.sh and successed include jira 集成，
but pod “zadig-mongodb-865f554c8-wwmv9” status always “CrashLoopBackOff”，detail Message is “Back-off restarting failed container” and Exit Code is “100”，how figure out？

state:
waiting:
message: back-off 5m0s restarting failed container=mongodb pod=zadig-mongodb-865f554c8-wwmv9_zadig(bc509f98-13c9-40c7-a6de-2a8b7e737ae0)
reason: CrashLoopBackOff
Exit Code: 100
Events:
Type Reason Age From Message

---

Normal Scheduled 21m default-scheduler Successfully assigned zadig/zadig-mongodb-865f554c8-wwmv9 to localhost.localdomain
Normal Pulled 20m (x5 over 21m) kubelet Container image "docker.io/bitnami/mongodb:4.4.6-debian-10-r8" already present on machine
Normal Created 20m (x5 over 21m) kubelet Created container mongodb
Normal Started 20m (x5 over 21m) kubelet Started container mongodb
Warning BackOff 103s (x101 over 21m) kubelet Back-off restarting failed container

Originally posted by @janetTiramisu in #77 (comment)

Please provide documents for cloud hosting services

Path: zh_cn/04.项目配置/00.service.md

Build script variables supports workflow's id.

pipeline 的构建脚本中变量还不支持pipeline的ID，期望支持获取ID

质效中心-单测覆盖率数据总览

请问文档里面的质效中心-单测覆盖率数据总览从哪里看？

How to uninstall Zadig completely？

/kind question

General Question

helm uninstall zadig -n zadig
Error: uninstall: Release not loaded: zadig: release: not found
helm list 为空

I can‘t get the ip of the node where the Pod is located on Zadig Portal

Is your feature request related to a problem? Please describe.

Sometimes，I need to confirm whether the service discovery address is correct. But now I can't get the ip of the node where the pod is located on the Zadig Portal.

Describe the feature you'd like

Zadig Team:
We will support viewing the pod ip on the zadig Portal.

Cannot cache maven build context

/kind question

General Question

开启使用工作空间缓存和配置缓存自定义目录后，下次构建依然会重新拉取依赖包

After compiled the Agent service from the source code, an error is reported and the hubserver can't be connected.

/kind bug
通过下载源码master进行编译后的agent无法接入集群：

time="2021-07-22T09:20:38+08:00" level=info msg="Connecting to proxy" url="ws:///api/hub/connect"
time="2021-07-22T09:20:38+08:00" level=error msg="Failed to connect to proxy" error="dial tcp :80: connect: connection refused"
time="2021-07-22T09:20:38+08:00" level=error msg="Failed to connect to proxy" error="dial tcp :80: connect: connection refused"

cat agent.yaml

  serviceAccountName: koderover-agent
  containers:
  - name: agent
    image: ccr.ccs.tencentyun.com/koderover-rc/hub-agent:20210706195834-9-main-pr-52
    imagePullPolicy: Always
    env:
    - name: AGENT_NODE_NAME
      valueFrom:
        fieldRef:
          fieldPath: spec.nodeName
    - name: HUB_AGENT_TOKEN
      value: "43d0ebdaf29d7f878e276e712a6acad3187c20096fd0469915d4b69603d33e19cfa4537444beda45"
    - name: HUB_SERVER_BASE_ADDR
      value: "ws:///api/hub"
    resources:
      limits:
        cpu: 1000m
        memory: 1Gi
      requests:
        cpu: 100m
        memory: 256Mi

updateStrategy:
type: RollingUpdate

---

HUB_SERVER_BASE_ADDR:ws:///api/hub
没有自动生成HUB_SERVER_BASE_ADDR地址

直接通过helm安装下载镜像是正常的.
helm list不显示

installing zadig ...
"koderover-chart" already exists with the same configuration, skipping
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "koderover-chart" chart repository
Update Complete. ⎈Happy Helming!⎈
Release "zadig-zadig" does not exist. Installing it now.
NAME: zadig-zadig
LAST DEPLOYED: Thu Jul 22 09:28:18 2021
NAMESPACE: zadig
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:

1. Zadig has successfully been installed on your system.
2. Use 10.129.39.90:30080 to access zadig system.
   [root@tke04 helm]# helm list
   NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
   [root@tke04 helm]# kubectl version
   Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.9", GitCommit:"4fb7ed12476d57b8437ada90b4f93b17ffaeed99", GitTreeState:"clean", BuildDate:"2020-07-15T16:18:16Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
   Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.9", GitCommit:"4fb7ed12476d57b8437ada90b4f93b17ffaeed99", GitTreeState:"clean", BuildDate:"2020-07-15T16:10:45Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
   [root@tke04 helm]# helm version
   version.BuildInfo{Version:"v3.6.1", GitCommit:"61d8e8c4a6f95540c15c6a65f36a6dd0a45e7a2f", GitTreeState:"clean", GoVersion:"go1.16.5"}
   [root@ helm]#

为什么一直Pending

Case 'voting' docker build failed.

2021-07-13T15:38:01.861+0800 build start
2021-07-13T15:38:01.862+0800 wait for docker daemon to start ...
2021-07-13T15:38:01.893+0800 login docker registry https://ccr.ccs.tencentyun.com
2021-07-13T15:38:02.436+0800 extracting workspace ...
2021-07-13T15:38:02.441+0800 no previous cache is found: cache not found
2021-07-13T15:38:02.446+0800 git init
Initialized empty Git repository in /workspace/voting-app/.git/
2021-07-13T15:38:02.450+0800 git fetch origin +refs/heads/main --depth=1
From https://github.com/guohechain/voting-app

• branch main -> FETCH_HEAD
• [new branch] main -> origin/main
  2021-07-13T15:38:05.048+0800 git checkout -qf FETCH_HEAD
  2021-07-13T15:38:05.059+0800 git --no-pager log --oneline -1
  6cc1d7f one
  Sending build context to Docker daemon 163.3kB
  Step 1/7 : FROM python:3.6-alpine
  ---> e5d44943603f
  Step 2/7 : WORKDIR /app
  ---> Using cache
  ---> 474644cc727e
  Step 3/7 : ADD requirements.txt /app/requirements.txt
  ---> Using cache
  ---> d4daaf475e62
  Step 4/7 : RUN pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
  ---> Running in 36e7fa1ff017
  Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='mirrors.aliyun.com', port=443): Read timed out. (read timeout=15)",)': /pypi/simple/flask/
  WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='mirrors.aliyun.com', port=443): Read timed out. (read timeout=15)",)': /pypi/simple/flask/
  WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='mirrors.aliyun.com', port=443): Read timed out. (read timeout=15)",)': /pypi/simple/flask/
  WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='mirrors.aliyun.com', port=443): Read timed out. (read timeout=15)",)': /pypi/simple/flask/
  WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='mirrors.aliyun.com', port=443): Read timed out. (read timeout=15)",)': /pypi/simple/flask/
  ERROR: Could not find a version that satisfies the requirement Flask (from versions: none)
  ERROR: No matching distribution found for Flask
  The command '/bin/sh -c pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/' returned a non-zero code: 1

2021-07-13T15:39:49.774+0800 build end. duration: 107.91 seconds
2021/07/13 15:39:49 Failed to run reaper, the error is: 1 error occurred:
* exit status 1

Originally posted by @zyu in #72 (comment)

The offline feature of the service

Is your feature request related to a problem? Please describe.

In the process of actual use of the test environment, some services need to be offline. During the process of updating the environment, I do not want to start and update this offline service, but this service may be online after a period of time.

Describe the feature you'd like

Zadig Team：

In the furture, we will provide the offline feature of the service

如何在不卸载以前版本的基础上，直接安装升级到最新版v1.2.0 Release？

如何在不卸载以前版本的基础上，直接安装升级到最新版v1.2.0 Release？
执行以下脚本后，打开未看到新版本功能，比如jira集成等。
curl -LO https://github.com/koderover/zadig/releases/latest/download/install_with_k8s.sh
chmod +x ./install_with_k8s.sh
export IP=

服务器 IP 地址，用于访问 Zadig 系统

如果您有自己的域名，可以配置您可用的域名 export DOMAIN =

export PORT=< 30000 - 32767 任一端口>

如果出现端口占用情况，换一个端口再尝试

./install_with_k8s.sh

Request status code gets 500 after installation.

Path: zh_cn/02.安装与配置/01.all-in-one.md

After configuring the trusted domain name for private docker registry integration, an error will still be reported.

/kind question

General Question

集成了docker仓库之后，「新建环境」会报错

1. 在宿主机的 /etc/docker/daemon.json 添加insecure-registries 宿主机执行docker login可以成功登录
2. 一开始以为zadig的docker使用的是dockerindocker 然后在 「dind-0」 这个pod里也添加了域名可信，进入「dind-0」pod执行docker login 也可登录成功，回到zadig页面还是会报错
3. 其他： 证书是自己生成的，私有docker仓库使用harbor搭建

The cmd main server graceful shutdown code seems does not meet expectations?

/kind question

the cmd main server graceful shutdown code seems does not meet expectations?

the problem code is here >>>

I wrote an example based on your code, it cant`t graceful shutdown !

package main

import (
	"context"
	"log"
	"net/http"
	"os/signal"
	"syscall"
	"time"
)

func main() {
	ctx, _ := signal.NotifyContext(context.Background(), syscall.SIGTERM, syscall.SIGINT)

	http.HandleFunc("/ping", ping)
	server := &http.Server{Addr: ":6666", Handler: nil}

	go func() {
		<-ctx.Done()
		t, _ := context.WithTimeout(context.TODO(), 5*time.Second)
		if err := server.Shutdown(t); err != nil {
			log.Printf("Failed to stop server, error: %s", err)
		}
	}()

	if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
		log.Printf("Failed to start http server, error: %s", err)
		return
	}
}

func ping(w http.ResponseWriter, r *http.Request) {
	time.Sleep(4 * time.Second)
	_, _ = w.Write([]byte("success"))
}

After adding an external cluster, the environment can't be created.

Path: zh_cn/10.系统设置/09.cluster_manage.md

When will gitee be supported?

Zadig 什么时候支持gitee

stuck when server added is duplicate

/kind bug

button remains loading when I reopen the modal dialog