huawei / containerops

DevOps Orchestration Platform

ContainerOps - DevOps Orchestration

Why DevOps orchestration?

There are many tools, projects, plugins, services adopted in the DevOps workflow. However, no one can cover all DevOps tasks. When developers move from one tool to another, they are facing the huge risk of reinvestment. How should we promote the DevOps process and make iteration more speedy? The principle of DevOps orchestration is to keep your original process working without any changes and just assemble tools, projects, plugins, services into orchestration engine. Improvements to the DevOps process can be done by gradually adding or replacing tools, projects, plugins or services for a smooth migration, and overall, DevOps needs to be promoted step by step, invasive changes are dangerous.

How DevOps Orchestration?

Combo the different DevOps services, tools and plugins to implementing DevOps orchestration is very complex, and it should resolve many challenges like deliver data between jobs, or resolve environment consistency for tools or plugins.

ContainerOps VS. Jenkins

1. Jenkins servers become snowflakes. Maintaining a DevOps workflow needs pasting shell script into the textboxes of UI, uploading and installing multiple plugins and dependencies in the slaves. The ContainerOps could clearly define the DevOps workflow by drawing and dragging through a GUI IDE. It uses the component endpoint URLs and doesn't need to download and install. All dependencies are already self-contained in the component.

2. Jenkins has no first class support for pipelines, and Jenkins 2.0 tries to address this by introducing a Pipeline plugins. However, it misses the point of DevOps. There are many tools and services throughout the whole Jenkins process, and each has its own pipeline. With the cloud native app becomes more and more sophisticated, this increasing complexity makes tools to be isolated in the chain and finding a good way to pass context in the whole workflow becomes critical. These are the problems ContainerOps wants to resolve.

3. Plugins accomplish everything you care about in Jenkins, and the plugin has dependencies on the environment. The ContainerOps use container-encapsulated components instead of plugins, and all dependencies have been self-contained in the component. The resources of running a component are handled by the ContainerOps engine through Kubernetes.

Architecture of project

The ContainerOps is a DevOps orchestration platform, and its architecture is micro services. All the codes in one repository, each service has an own folder and maintainer.

• component - Some components maintained by ContainerOps team.
• pilotage - The orchestration engine service.
• crew - It's a RBAC service.
• dockyard - It's a artifact & container repository.
• assembling - It's a artifact & container build service.
• scaler - The third service manage service.
• nucleus - The component manage service.
• singular - The deployment and operations tools.
• logarithm - The log service collect from platform.

Database

The components of ContainerOps all need SQL database support, we suggest use TiDB/TiKV for distribute deployment. For debug, you could create a database in your localhost.

GRANT USAGE ON *.* TO 'containerops'@'localhost' IDENTIFIED BY 'containerops' WITH GRANT OPTION;
CREATE DATABASE `containerops` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
GRANT ALL PRIVILEGES ON containerops.* TO containerops@localhost IDENTIFIED BY 'containerops';
FLUSH PRIVILEGES;

Contribute

If you interest the ContainerOps and want to get involved in developing. Getting start with this reading:

• The contributor guidelines

Community

containerops's Issues

Could you please support the CentOS7 for the tools of kubernetes?

I want to install kubernetes on the CentOS7,but the tools only support for the Ubuntu.

Could the tools for installing kubernetes with general user?

The tools for installing kubernetes only can excute by root,but I want general user to use it,could you please support for it?

Add a rest API to run a flow in pilotage

• Method: POST
• Body: Flow yaml data
• Return: 201/400

Add database support and refactoring Singular

mistake with password and db 's value in example

https://github.com/Huawei/containerops/blob/master/common/containerops.toml.example
password = "containerops_database"
db = "containerops_password"
We should swap the value.

pipeline display

1. "Action Name", "Action Timeout" label will change "Component Name"， "Component Timeout"
2. input json in inputbox, display imperfect

Correct Docker CE version?

From Docker CE release note page, we can see Docker CE's stable releases include:
17.09.0-ce (2017-09-26)
17.06.2-ce (2017-09-05)
17.06.1-ce (2017-08-15)
17.06.0-ce (2017-06-28)
17.03.1-ce (2017-03-27)
17.03.0-ce (2017-03-01)

And in Cloud Native Stack Integration Status page, current Docker CE version is 17.04.0, which is an edge release.

Is it better to put stable releases in higher priority?

Link output and input value between components

1. Parse the data link relation between components.
2. Collect the output data from a component.
3. Assign data from above as input data with a component.

And ping @wangkirin

issue: i think “”$GOPATH:/bin“” is “”$GOPATH/bin“”

In this file：https://github.com/Huawei/containerops/blob/master/demo/cncf/kubernetes/Dockerfile
Line 31 ：ENV PATH $PATH:$GOROOT/bin:$GOPATH**:**/bin:$GOPATH/src/github.com/Huawei/containerops

I think it is：
ENV PATH $PATH:$GOROOT/bin:$GOPATH**/**bin:$GOPATH/src/github.com/Huawei/containerops

Add database support in pilotage

Deploy cluster in parallel

Current deploying cluster in parallel with singular will fail, the reason is that the SSH commands direct both stdio and stderr to the same stream, which is not handled by a locker, so somethings the stream content would be empty(stdio flushes the correct result, no error occurs, then stderr flushes the empty content).

To solve this, just pass 2 readers to SSH commands, receive stdio and stderr separately.

must manually create the kubernetes namespaces demo

must manually create the kubernetes namespaces demo

很希望现在国内技术比较强的公司能够多使用中文

很希望现在国内技术比较强的公司能够多使用中文.
发扬光大**, 带动国内技术的发展也是一件好事!!!

人家外国人, 都开始很多文档支持中文了, 但是国人就是不用中文.

华为, 要为中华有为!

Where can I find manual or document ?

Where can I find manual or document ?

Deploy cncf.build with Pilotage

Deploy cncf.build site with pilotage flow.

ping @crystaldust

Could you please support service and component of k8s check func

for example, such as Etcd.
step 1 : Verify whether that etcd service exists on the server. Just skip this step for installing etcd service when it exists.
step 2 : Verify whether that etcd binary exists on the server, then ask user replace it or continue.

Installation Documentation Missing

How is this built and installed?

workflow ON/OFF status without refresh

change field name and save workflow, workflow ON/OFF status without refresh

check if ufw is enable when use singular to install kubernetes on ubuntu

when excute singular to install kubernetes in ubuntu, if there is no ufw or ufw is not load , stop ufw is unneccsary and singular will exit with error
Failed to stop ufw.service: Unit ufw.service not loaded. .

I think it's better check ufw first

component must open more than one port

component must open more than one port

Minimum memory limit allowed is 4MB

Minimum memory limit allowed is 4MB

without conf folder

without conf folder, must manual create

A suggestion for the dirctory (“containerops/component/images/” )

containerops/component/images/
-------------------------------------- golang/
----------------------------------------------runtime_base/
----------------------------------------------golint_tool/
----------------------------------------------gofmt_tool/
----------------------------------------------XX_tool/
--------------------------------------javalang/
--------------------------------------XXlang/

there is't any documentation for end users.

There is't any documentation for end users, how to use the contanerops?

Add an 'append' command

Sometimes we need to append a node to an existing cluster. For the existing template, we can add one or more node info, then apply it with append subcommand instead of deploy. When reading the template yaml with append subcommand, we can first check if the cluster's master and workers are there, then apply the infras to the newly added nodes.

check pipeline steps

1. create component, set input, output, env data
2. create pipeline, add stage, add action, use Component, set data
3. link component and set relationship
4. run the pipeline
5. check code check result
6. check code build result
7. check code run result
8. check code test result
9. check input output para paras result

containerops.sh down

containerops.sh is down. DNS does not resolve.

delete index on event_definition

delete event index on event_definition

Could you test k8s1.7.5 + istio 0.2

We are using tectonic 1.7.5, which will install k8s 1.7.5.
Could you test istio 0.2, see whether it works fine.

Thanks a lot.

My mail: [email protected]

“clusterrolebinding ”should is "cluster role binding"?

if err := utils.SSHCommand("root", key, ip, 22, "kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap", os.Stdout, os.Stderr); err != nil {