argoproj / argo-cd Goto Github PK

Declarative Continuous Deployment for Kubernetes

Home Page: https://argo-cd.readthedocs.io

License: Apache License 2.0

Makefile 0.33% Go 80.55% Shell 0.69% Dockerfile 0.20% Lua 1.47% JavaScript 0.06% TypeScript 15.34% HTML 0.01% SCSS 1.18% CSS 0.03% Procfile 0.13% Mustache 0.01%

argo kubernetes continuous-deployment gitops continuous-delivery docker cd cicd pipeline devops

argo-cd's Introduction

Releases:

Code:

Social:

Argo CD - Declarative Continuous Delivery for Kubernetes

What is Argo CD?

Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes.

Why Argo CD?

Application definitions, configurations, and environments should be declarative and version controlled.
Application deployment and lifecycle management should be automated, auditable, and easy to understand.

Who uses Argo CD?

Official Argo CD user list

Documentation

To learn more about Argo CD go to the complete documentation. Check live demo at https://cd.apps.argoproj.io/.

Community

Contribution, Discussion and Support

You can reach the Argo CD community and developers via the following channels:

Q & A : Github Discussions
Chat : The #argo-cd Slack channel
Contributors Office Hours: Every Thursday | Agenda
User Community meeting: First Wednesday of the month | Agenda

Participation in the Argo CD project is governed by the CNCF Code of Conduct

Blogs and Presentations

argo-cd's People

Contributors

Stargazers

Watchers

Forkers

jessesuen alexmt georgeck shrinandj snikolaev1 wanghong230 dthomson25 jazminelkan-gonzalez varsharaju jbrette rr-on-gh timurfayruzov sctsang chocopowwwa lilwan capier sindhusrinivasan27 nnbishoper geddamsatish jovebirdy firethewhole mikuhop hlj205 suwakokochiya conorfennell liusixin davidmr001 kuy stephydeng1998 askulkar ddddddd1215 twz123 yaojunyu ferhimedamine vczh2 tyler2350 mduarte l4ttic3 aglcaicai mygirl8893 kevdowney swiftdiaries ninefive amarrella wmedlar solidnerd jotacamou 40a zcahana 33944284 marketxing1 narg95 dmazhukov roytman go-ky chansimper limnick tweakmy bunjiboys eupestov woodprogrammer yujieda franklinharry infinitydon liberty-song adieu duboisf 2226265772 memorymemo jennysharps zuole jacobheidelbachdfds zyylele123 agile-science alexec morristech ericsgagnon jecho ifmorrison hartman17 edlee2121 amarruedo redbadger paulrudin dulltz fli bmahule amudhanayakia masayamatsuda mikias coffeesn0b ubi-mirrors smasseman zlx-github nabadger fabito cherwin primeroz bderickson yann-soubeyrand

argo-cd's Issues

Deployed application resources should be annotated

Deployed application resources should be annotated with a label which has application name and application revision. This allows displaying following additional information about the application: a version of each deployed resource, unexpected resources (e.g. resources which are deployed but not defined in the specification)

Add application and revision labels to resources
Populated 'unexpected' resources while comparing target and live states
Render 'unexpected' resources and resource revision in web ui

UI needs to support auth tokens

Taking the liberty of filling this in. We have a few options:

Set and parse cookies server-side. These cookies could contain the JWT required by Argo CD. This has the advantage of being secure and we can set the HttpOnly flag on them since the client-side JavaScript won't need to (and, indeed, cannot) interact with them. A tradeoff is that logout require server-side action to delete the cookie.
Generate tokens server-side (as above), but send them to the client. This is the current implementation for token generation. The client can use JavaScript to create a cookie from the token, then look that cookie up before sending the token to the server. This is less secure since JavaScript can access the cookie (i.e., no HttpOnly flag), but JavaScript can be used for the logout process. To ensure safe access and encryption, probably best to use a secure cookie and avoid HTML5 local storage.

As a side note, a best practice is for all cookies to be secure cookies, at least in production.

For the first option, the frontend integrates by having an API endpoint for login (already done!) that needs to set a cookie in the response (not yet done), rather than returning the token. Logout requires a second endpoint to delete the cookie.

For the second option, I believe that everything is in place for implementation right now.

@alexmt, interested in your thoughts.

argocd context management

For managing and switching between argocd contexts (servers|users), I am proposing the following to the CLI:

List all contexts, highlight or star the current one. Display context name, server, and user info

$ argocd (context|ctx)

Switch context. Rewrite ~/.argocd/config to update currentContext:

$ argocd (context|ctx) <name>

Implement live application CRD update

In addition to periodical application syncronization, Application controller should watch for all application resources changes and update application CRD.

Example app

Add example app and record small demo video

Resources tree view on application details page

Application details page should contains resources tree view:

application controller should persist resources tree in app CRD
application details page should render resources tree

Rollback ignores parameter overrides

Rollback to previous deployment means use previously deployed revision and overridden parameter. Currently Rollback only deploy using previous revision and ignore overidden parameters

Argo workflow integration

The argo workflow engine will be integrated as part of the argo-cd application in order to drive things like user defined application actions, and event hooks.

Support for customizable resource actions as Lua scripts

Repurposing this issue for a different proposal:

A use case is to perform some kind of custom action on an application resource. Some example actions are:

pause / resume an Argo Rollout
suspend / resume a CronJob
set some istio traffic shaping weights on a VirtualService

This proposal is to support customized actions on application resources. These actions would be made available from the API and CLI. The "action" would be a lua script that mutates the resource object in some way.

Some considerations:

we may need a way to list available actions on a resource (to present in UI). For example, if a cronjob is suspended, then it should show "resume" as an available action but not "suspend"
some actions may require inputs. For example, to set weights for a Istio VirtualService
actions may return errors. For example, if someone supplied invalid input to an action

Workflow for blue-green deployment while preserving original deployment name

Problem:
I want to define a single deployment and service in ksonnet/git. The resource names of the corresponding deployment and service would be jesse-deploy and jesse-svc, respectively. I will have 5 replicas of the deployment. I want the ability to perform a blue-green deployment where I surge to 10 concurrently running pods, half of which are running the newer version, then perform a DNS/service level cutover to the new version via an update of the jesse-svc service resource with the new label selector.

The challenge is that we want to preserve the deployment resource/name, while also not wanting to redefine the same deployment twice as two deployment components in the ksonnet app (i.e. blue and green). In other words, I do not want jesse-deploy-green.yaml and jesse-deploy-blue.yaml deployment manifests checked in anywhere in git.

Current blue-green kubernetes techniques describe the creation of a new deployment with a new name and updating the service to point to the new deployment. However, this does not work well with GitOps because it implies the new deployment and new name would need to be checked-in with a new name.

I am proposing the following workflow to achieve blue-green deployment while preserving the original deployment resource, and without requiring a user to manage two duplicate deployment manifests in their git repo.

a deployment jesse-deploy and service jesse-svc start out running version 1 of the image, with selector app=jesse and 5 replicas.
blue-green deploy action is invoked via argo-cd API
a new, ephemeral deployment jesse-deploy-tmpXYZ is created at version 1, with selector app=jesse-tmpXYZ
jesse-deploy-tmpXYZ becomes healthy with 5 replicas. At this point in time, we have 10 pods running version 1 of the image, split equally between the two deployments. But no traffic is directed to the ephemeral deployment jesse-deploy-tmpXYZ because the label is unique.
service jesse-svc is patched to use selector app=jesse-tmpXYZ (which is still running version 1). At this point, all traffic is directed to the ephemeral deployment jesse-tmpXYZ running version 1
jesse-deploy is updated with the new manifest to run version 2
jesse-deploy becomes healthy and with 5 replicas. At the point, we have 5 pods running version 2, and 5 pods running version 1. No traffic is being directed to version 2 yet.
The service jesse-service is updated with the original app selector app=jesse. At this point, all traffic is directed back to jesse-deploy which is running version 2.
jesse-deploy-tmpXYZ is no longer receiving any traffic. It is now safe to delete the ephemeral deployment. Once deleted, the live state of the deployment and service should now reflect the state in git and the app should be considered synced.

Upgrading installs fails on final step

Steps to reproduce:

Set up a new argocd installation (I'm using Minikube).
Build the project (make, although make cli may be sufficient).
Run ./dist/argocd install
Run ./dist/argocd install --upgrade

Expected output does not include the final line:

$ ./dist/argocd install --upgrade
Namespace 'argocd' up-to-date
CustomResourceDefinition 'applications.argoproj.io' up-to-date
ServiceAccount 'application-controller' up-to-date
Role 'application-controller-role' up-to-date
RoleBinding 'application-controller-role-binding' up-to-date
Deployment 'application-controller' up-to-date
ServiceAccount 'argocd-server' up-to-date
Role 'argocd-server-role' up-to-date
RoleBinding 'argocd-server-role-binding' up-to-date
Deployment 'argocd-server' up-to-date
Service 'argocd-server' up-to-date
Deployment 'argocd-repo-server' up-to-date
FATA[0000] Service "argocd-repo-server" is invalid: [metadata.resourceVersion: Invalid value: "": must be specified for an update, spec.clusterIP: Invalid value: "": field is immutable]

This may be related to kubernetes/kubernetes#11237. It's possible that this could be addressed with a tweak to how we invoke K8s.

Argo CD may benefit from a code of conduct

A code of conduct can promote inclusiveness and describe step-by-step processes for addressing conflict within the project. See https://help.github.com/articles/adding-a-code-of-conduct-to-your-project/.

SSO integration

Need to integrate with various identity providers and provide RBAC controls for mapping user/groups access to clusters, repos, and applications. We should use dex for SSO.

Update app via `argocd app set`

Need a way to update an app via the CLI. For example, if I want to switch the branch my app is tracking, the source repository, etc...

There should be an argocd app set command which will update various fields similar to the app create:

argocd app set --repo https://github.com/argoproj/argo-cd.git --path examples/guestbook --env minikube

Migrate to ksonnet 0.10

ksonnet 0.10 is not out yet, but we want the feature for drop-in yamls as soon as possible. In the interim, build tip of ksonnet master until there is a tagged release.

Repo browsing to improve app creation process

The current way of enabling applications via the CLI can be error prone. It would be nice to have the ability to create apps via the UI by browsing the git repo for the ksonnet app. This requires the repo server to provide listdir functionality, possibly using the GitHub APIs or by checking out the repository and doing filesystem syscalls.

UI design: https://projects.invisionapp.com/share/J8IUZD96ZCP#/screens/291683071

UI tickets:

Redesign application list page (#243)
Repositories list page (#241)
Application creation wizard (#242)

Support 'ephemeral' applications

Argo CD should support deploying and destroying 'ephemeral' applications. The ephemeral application is not version controlled and supposed to be deployed and destroyed after a short time. Typical use case is e2e tests.

`argocd app add` should support an -f option

In the end, Argo CD app defs are just k8s manifests, whether they are created using UI or CLI. We should additionally support creation of the app instance via explicit manifest. Then we could do something like:

argocd app add -f http://github.com/argoproj/argo-cd/tree/master/examples/guestbook-minikube.yaml

or:

kubectl create -f http://github.com/argoproj/argo-cd/tree/master/examples/guestbook-minikube.yaml

Add ability to display unexpected resources

Spawned from issue #25. Need a way to display unexpected resources:

Populated 'unexpected' resources while comparing target and live states
Render 'unexpected' resources and resource revision in web ui

Argocd install failure

With the latest release v0.2.0, following the cmd to run argocd install and got resources created in argocd ns.
However the pods are not coming up:

kc get pod -n argocd
NAME                                     READY     STATUS             RESTARTS   AGE
application-controller-f85b84c9f-ddkqj   0/1       CrashLoopBackOff   6          7m
argocd-repo-server-85b697d4f7-dcggf      0/1       ImagePullBackOff   0          7m
argocd-server-5bb977f748-vhqb5           0/1       CrashLoopBackOff   6          7m

any hint?

TLS and authentication support on API Server

Need TLS support for the API server with initial basic auth for the admin user.

Rollback feature enhancement

Rollback feature should be enhanced:

User should be able to view recently deployed revisions.
User should be able to select recently deployed revision and view parameter values.
User should be able to rollback to selected revision
Persist revision info including env parameter values in history
Implement rollback UI

Delete all the kube object once app is removed

After adding the app and rm the app, the service, pod, deployment objects are still there. We should delete all the objects related to the app.

BTW, should we rename "rm" to "delete"? argo is using delete.

DeletionPropagation policy when deleting an application

Just like in kubernetes, it is sometimes desirable to delete a parent resource, without the cascaded deletion of all its children. I think we need to incorporate k8s-like DeletionPropagation policy so that someone can delete an app and orphan its resources.

Support ssh git URLs and ssh key authentication

HTTPS access and password authentication are not always available. We need to support authentication using SSH key.

Deployment of resources needs to use `kubectl apply` semantics

The way we are updating resources is not correct. kube.go::ApplyResource() is doing a simple Update() but it needs to do a Patch() using 3-way strategic merge like kubectl. Kubectl achieves this by storing last-applied-configuration as an annotation and incorporate that into the calculation. We need to do the same.

App controller incorrectly report that app is out of sync

The issue is not reproducible consistently. Sometimes app controller detects a difference in api version of some app resources even if the application is fully synchronized. This happens because controller loads same object twice using different API versions and then randomly uses one to compare with expected object.

Application create/update should validate cluster/repo/app/environment

Today, it is possible to create an application with invalid cluster/repo/app information. The API server should work in tandem with repo server to ensure that all the information which was supplied, is valid. This will prevent a large source of issues.

When discussing with @alexmt, we will still allow a force option as an escape hatch, to bypass validation.

Rework local config to support multiple servers/credentials

The local argocd config should to support multiple servers at the same time, switching between configuration contexts, as well as optionally storing self-signed certificates into the local ~/.argocd/config. In other words, ~/.argocd/config should look more like ~/.kube/config.

The resulting change will be the following user experience:

$ argocd login my-argocd-server.example.com --context-name my-argocd
WARNING: this server uses a self-signed certificate. Store the certificate and proceed? (y/n) y
Username: admin
Password:

The resulting config will look like (up for discussion):

current-context: my-argocd

contexts:
- name: my-argocd
  server: my-argocd-server.example.com
  user: my-argocd-server.example.com

servers:
- server: my-argocd-server.example.com
  certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSU.......LS0tLQo=

users:
- name: my-argocd-server.example.com
  session-token: eyJhbGciOiJI....l-6XOUVAxy8

We would then introduce some commands for switching between contexts. e.g.

argocd config use-context CONTEXTNAME

Auto-sync option in application CRD instance

The application instance needs an option to automatically sync on every push. Currently this is an separate action performed by the user.

Delete pods using ArgoCD UI

A user should be able to delete pod using argo UI. Delete action should be available via drop down menu on resouce tree and on pod details panel.

Implement API to delete pod
Render delete menu/button on application details page

Argo CD server and repo server fails to start if installed into custom namespace

ERROR: logging before flag.Parse: E0409 23:04:38.881595       1 reflector.go:205] 
github.com/argoproj/argo-cd/controller/controller.go:76: Failed to list *v1alpha1.Application: applications.argoproj.io is forbidden:
User "system:serviceaccount:argocd-fusion:application-controller" cannot list applications.argoproj.io in the namespace "argocd-fusion"

`app add/rm` should be `app create/delete`

The semantics of adding and removing repo/cluster is different from creating and deleting an application. In the former, we are not creating any resources, we are just inputing credentials. In the later, the application will result in k8s services being created and deleted.

Renaming the CLI for app add/rm to app create/delete better conveys this difference.

ksonnet show should order resources by k8s proper creation order

A promised feature of ArgoCD is that it should create things in proper kubernetes ordering (e.g. namespaces, configmaps, secrets earlier than deployments, etc...)

argocd-server service should expose 80 and 443 during install

Now that we have TLS, 443 should be exposed as well. Both map to 8080 on the pod, which muxes between both protocols.

gRPC interceptor to transform kubernetes errors to gRPC errors

We should have an gRPC interceptor which translates kubernetes errors into the corresponding gRPC codes. The interceptor would have logic like:

apierr "k8s.io/apimachinery/pkg/api/errors"

if apierr.IsNotFound(err) {
    err = grpc.Errorf(codes.NotFound, err.Error())
} else if apierr.IsAlreadyExists(err) {
    err = grpc.Errorf(codes.AlreadyExists, err.Error())
}

Support for event hooks as Argo workflows

There are use case is to invoke a certain action after an Argo CD event has occurred. Events may include (but not limited to):

Pre-Sync
Post-Sync
Pull Request Open
Pull Request Close

Allow CLI tokens to be specified manually

Per suggestion from @jessesuen in #84:

have other ways of creating a client without requiring there exists .argocd directory (so long as a token is supplied)

Support rollback without updating spec in git repo

Currently api server supports only application sync operation, which deploys target revision specified in git repo. That means a user has to commit different revision into git repo to deploy different revision. This is limitation does not work well with rollbacks.

In order to support rollbacks API server should allow deploying any revision.

Support any revision in application/sync API
Implement ability to deploy any revision using web ui

Project should have a license

Since argo-cd is a public-facing, open-source project, we should have a license in place (either called LICENSE or LICENSE.md in the root of the project):

To encourage involvement by providing peace-of-mind to contributors in the community regarding access to and use of their code.
To encourage adoption, as a show of good faith. In prospective corporate adopters, a license may be required by management or procurement departments, even (or especially) for open-source code.

Persist environment parameters during deployment

Each environment has parameters which are used to generate resources manifests. It would be useful to persist and display parameters which were used to deploy the application.

Persist parameters during deployment
Render parameters on application details page

Server fails to read config map from namespace on cluster with RBAC enabled

Steps to reproduce:

Install argo to any cluster with enabled RBAC. Argocd server crashes at start while trying to read configmap

Argo CD should try to clone app repo without password if repo is not registered

Currently Argo CD fails to sync app if repo is not registered. It does not make sense to require registering public repos. Argo CD should try to clone app repo without password if repo is not registered

Unable to add repo if repo URL contains upper case characters

FATA[0000] rpc error: code = Unknown desc = Secret "repo-fusionBlueOcean-1908697072" is invalid: 

metadata.name: Invalid value: "repo-fusionBlueOcean-1908697072": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character
(e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')

application-controller stuck in loop (Failed to watch clusters: unknown (get secrets), retrying in 10s)

While testing ArgoCD, my controller gets into an unrecoverable state:

time="2018-04-22T07:42:39Z" level=info msg="Application Controller (version: v0.1.0+befa43e.dirty) starting (namespace: jesse-test)"
time="2018-04-22T07:42:39Z" level=warning msg="Failed to watch clusters: unknown (get secrets), retrying in 10s"
time="2018-04-22T07:42:39Z" level=info msg="Refreshing application 'guestbook' status (force refreshed: false)"
time="2018-04-22T07:42:41Z" level=info msg="Comparing app guestbook state in cluster https://REDACTED (namespace: default)"
time="2018-04-22T07:42:43Z" level=info msg="App guestbook comparison result: prev: OutOfSync. current: Synced"
time="2018-04-22T07:42:43Z" level=info msg="Application update successful"
time="2018-04-22T07:42:49Z" level=warning msg="Failed to watch clusters: unknown (get secrets), retrying in 10s"
time="2018-04-22T07:42:59Z" level=warning msg="Failed to watch clusters: unknown (get secrets), retrying in 10s"
time="2018-04-22T07:43:09Z" level=warning msg="Failed to watch clusters: unknown (get secrets), retrying in 10s"
time="2018-04-22T07:43:19Z" level=warning msg="Failed to watch clusters: unknown (get secrets), retrying in 10s"
time="2018-04-22T07:43:29Z" level=warning msg="Failed to watch clusters: unknown (get secrets), retrying in 10s"

The only way to fix this is to kill the controller. The issue seems to be in the watchAppsResources() method.

Pods logs on application details page

A user should be able to see live pod logs on application details page.

(https://github.com/argoproj/argo-cd/wiki/Application-details-page-design)

Implement stream logs API
Implement logs rendering on application details page

ArgoCD failed to add cluster

ArgoCD cli failed to add cluster with following error:

username/password or bearer token may be set, but not both

`argocd login` and local ~/.argocd/config for CLI

After auth is added, all CLI command will need to be authenticated. This implies we need argocd login command which would generate this file in the user's home directory. The credentials would be supplied for future CLI requests.

Move Kubernetes manifest generation into separate service

Both controller and API server needs to generate kubernetes manifests for applications. In order to improve performance we should move that functionality into separate service.

Implement repo server
Add repo server to installer

Argo-cd perimeter/integration

What is the scope of argo-cd over Jenkins X plans and Spinnaker interaction with k8s.
Also, is it planned any integration with the new emerging k8s dev tools that are starting to be discussed in sig-apps?