jonashackt / tekton-argocd-eks Goto Github PK

How to install and configure ArgoCD, Tekton (incl. Tekton Triggers) & a Cloud Native Buildpacks powered Pipeline on Amazon EKS and integrate with GitLab & GitHub

Home Page: http://tekton.tekton-argocd.de/#/pipelineruns

License: MIT License

TypeScript 100.00%

eks aws tekton tekton-pipelines gitlab kubernetes argocd tekton-triggers tekton-tutorial tekton-dashboard

tekton-argocd-eks's People

Contributors

Stargazers

Watchers

Forkers

rmdg88 ahmedqazi444 jinvishal klapcsik brighton-momo avi202020 thien-nguyenthanh pulumi-ce jasric89

tekton-argocd-eks's Issues

Add GitHub as Trigger resource and reporting instance

We not only want to be able to have our Tekton Pipelines triggered by GitLab, but also by GitHub - and the application project originated from GitHub also https://github.com/jonashackt/microservice-api-spring-boot

ToDos:

EventListener for GitHub
Trigger-Secret for GitHub WebHook
GitHub repository WebHook in https://github.com/jonashackt/microservice-api-spring-boot
Use GitHub Container Registry in Buildpacks Task to publish image (needs separate Pipeline?)
Report status back using https://hub.tekton.dev/tekton/task/github-set-status

Provide Tekton Dashboard URL to gitlab-set-status dynamically

Currently it's fixed inside the default parameter inside our pipeline.yml:

    - name: TEKTON_DASHBOARD_HOST
      description: The Tekton Dashboard URL for the status reports in GitLab
      default: "http://adfae510ff50e4949b7c7a5949099632-1408242414.eu-central-1.elb.amazonaws.com"

Maybe we could use a ConfigMap here? (see https://martinheinz.dev/blog/47)

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

Update Helm release traefik to v26

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

ansible

tekton/tasks/kustomize-manifests.yml

tekton/tasks/task-hello-world.yaml

github-actions

.github/workflows/provision.yml

actions/setup-node v3.6.0

actions/cache v3

pulumi/action-install-pulumi-cli v2.0.0

helmv3

traefik/install/Chart.yaml

traefik 22.3.0

kustomize

argocd/install/kustomization.yaml

argoproj/argo-cd v2.6.7

npm

eks-deployment/package.json

@pulumi/aws 5.38.0

@pulumi/eks 1.0.2

@pulumi/pulumi 3.66.0

@types/node 18.16.4

Check this box to trigger a request for Renovate to run again on this repository

Publish Tekton Dashboard as GitHub Actions Environment

Right now we create a environment from our Pulumi created EKS cluster address which has minimal use right now. It would be much more aligned with the project's goals to have the Tekton Dashboard available which could be also linked inside the GitHub Actions log.

homebrew not in PATH in GitHub Actions virtual environments anymore

Builds run into the following error when using brew command

/home/runner/work/_temp/3d77d38c-dec3-43f1-8d28-77bc8c7c9668.sh: line 1: brew: command not found

see https://github.com/jonashackt/tekton-argocd-eks/actions/runs/3536379914/jobs/5935381463

Check this box to trigger a request for Renovate to run again on this repository

Exposing tekton dashboard

Hey Jonas! Super cool repo, I love how it covers the whole thing end to end, it was really instructive to me.

Not sure if it was your intention to expose publically your tekton dashboard here, but it gives anyone on the internet some scary powers over your cluster. I hate those stories of people that try to do something cool and by mistake end up with a giant cloud provider bill, that's why I had to give you a heads up.

If this was intentional all along feel free to ignore and close this issue

Configure Traefik with Let's Encrypt certs and use https for all dashboards and services

Right now only ArgoCD has https enabled - but also lacking a propert certificate. All other services have a great Host name based routing through Traefik but only use http. Having everything https-based would complement the setup.

Traefik Helm installation with pinned version and managed by Renovate

Ingress for all: ArgoCD dasboard/server & Tekton Dashboard (as already with the Tekton Triggers Eventlistener)

Currently the ArgoCD and Tekton dashboards are simply Services with type LoadBalancer which are exposed through AWS ELBs. This could be done better like the Ingress configuration of the Tekton EventListener.

See https://blog.pipetail.io/posts/2020-05-04-most-common-mistakes-k8s/

Here we also have to mind the waiting for the availability of the ArgoCD server to be ready for login, which was quite complex and needs to work with the Ingress version also.

Maybe we should also switch to Traefik instead of Nginx for all 3 services?! But thats also another issue...

Remove the need for bulky replace-with-yq tasks in the Tekton pipeline

Right now - only for later deployment - we have quite a bulky implementation of 3 replace-with-yq tasks inside our Tekton pipeline, that inherit quite a bunch of yq expressions, we need to maintain in the future:

   - name: replace-deployment-name-branch-image
      taskRef:
        name: replace-yaml-value-with-yq
      runAfter:
        - switch-config-repository-branch
      workspaces:
        - name: source
          workspace: config-workspace
      params:
        - name: YQ_EXPRESSIONS
          value:
            - ".metadata.name = \"$(params.PROJECT_NAME)-$(params.SOURCE_BRANCH)\""
            - ".spec.template.spec.containers[0].image = \"$(params.IMAGE):$(params.SOURCE_REVISION)\""
            - ".spec.selector.matchLabels.branch = \"$(params.SOURCE_BRANCH)\""
            - ".spec.template.metadata.labels.branch = \"$(params.SOURCE_BRANCH)\""
        - name: FILE_PATH
          value: "./deployment/deployment.yml"

    - name: replace-service-name-branch
      taskRef:
        name: replace-yaml-value-with-yq
      runAfter:
        - replace-deployment-name-branch-image
      workspaces:
        - name: source
          workspace: config-workspace
      params:
        - name: YQ_EXPRESSIONS
          value:
            - ".metadata.name = \"$(params.PROJECT_NAME)-$(params.SOURCE_BRANCH)\""
            - ".spec.selector.branch = \"$(params.SOURCE_BRANCH)\""
        - name: FILE_PATH
          value: "./deployment/service.yml"

    - name: replace-ingress-name-route
      taskRef:
        name: replace-yaml-value-with-yq
      runAfter:
        - replace-service-name-branch
      workspaces:
        - name: source
          workspace: config-workspace
      params:
        - name: YQ_EXPRESSIONS
          value:
            - ".metadata.name = \"$(params.PROJECT_NAME)-$(params.SOURCE_BRANCH)-ingressroute\""
            - ".spec.routes[0].match = \"Host(`$(params.PROJECT_NAME)-$(params.SOURCE_BRANCH).$(params.TRAEFIK_DOMAIN)`)\""
            - ".spec.routes[0].services[0].name = \"$(params.PROJECT_NAME)-$(params.SOURCE_BRANCH)\""
        - name: FILE_PATH
          value: "./deployment/traefik-ingress-route.yml"

It would be really nice to replace this with something smarter - not necessarily Helm, but a more declarative approach?!

0/2 nodes are available: 2 node(s) had volume node affinity conflict.

Sorry in German, since this is from an internal chat:

Mir wird gerade (nach 6 Monaten) klar, warum mein EKS cluster sich immer wieder an die Wand fährt. Ich habe gerade zufällig mal wieder auf die nodes in k9s geschaut - und siehe da: ganze 457 Pods hängen auf EINER node, die anderen haben nur jeweils 14 abbekommen

Tekton Buildpacks PVC gives persistentvolume-controller waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator

The Tekton buildpacks pipeline isn't running properly anymore.

Running k describe pvc buildpacks-source-pvc shows the problem, that no PersistentVolume seems to be created anymore:

 Normal  ExternalProvisioning  14s (x8 over 101s)  persistentvolume-controller  waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator

tkn pipelinerun logs $PIPELINE_RUN_NAME --follow returns: "unable to decode an event from the watch stream: unable to decode watch event: no kind \"PipelineRun\" is registered for version \"tekton.dev/v1\"

Just starting a PipelineRun with:

PIPELINE_RUN_NAME=$(kubectl create -f tekton/pipelines/pipeline-run.yml --output=jsonpath='{.metadata.name}')

and then trying to follow the logs using the tkn cli:

tkn pipelinerun logs $PIPELINE_RUN_NAME --follow --all

But this results in the tkn pipelinerun logs command getting stuck with the following errors (and thus forever running GitHub Actions):

tkn pipelinerun logs $PIPELINE_RUN_NAME --follow --all
W0317 10:25:40.242703   61304 reflector.go:347] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: watch of *v1.PipelineRun ended with: an error on the server ("unable to decode an event from the watch stream: unable to decode watch event: no kind \"PipelineRun\" is registered for version \"tekton.dev/v1\" in scheme \"github.com/tektoncd/pipeline/pkg/client/clientset/versioned/scheme/register.go:32\"") has prevented the request from succeeding
W0317 10:25:54.896587   61304 reflector.go:347] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: watch of *v1.PipelineRun ended with: an error on the server ("unable to decode an event from the watch stream: unable to decode watch event: no kind \"PipelineRun\" is registered for version \"tekton.dev/v1\" in scheme \"github.com/tektoncd/pipeline/pkg/client/clientset/versioned/scheme/register.go:32\"") has prevented the request from succeeding
W0317 10:26:24.137912   61304 reflector.go:347] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: watch of *v1.PipelineRun ended with: an error on the server ("unable to decode an event from the watch stream: unable to decode watch event: no kind \"PipelineRun\" is registered for version \"tekton.dev/v1\" in scheme \"github.com/tektoncd/pipeline/pkg/client/clientset/versioned/scheme/register.go:32\"") has prevented the request from succeeding
W0317 10:27:26.656261   61304 reflector.go:347] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: watch of *v1.PipelineRun ended with: an error on the server ("unable to decode an event from the watch stream: unable to decode watch event: no kind \"PipelineRun\" is registered for version \"tekton.dev/v1\" in scheme \"github.com/tektoncd/pipeline/pkg/client/clientset/versioned/scheme/register.go:32\"") has prevented the request from succeeding

Speed up GitHub Actions workflow

Currently all renovate action lead to long pipeline runs, since there are the most updates right now.

So let's break the GitHub Actions workflow into 2 - one for Pulumi provisioning - and one for Tekton and Argo

Renovate should keep Tekton and Argo k8s manifests up-to-date

Currently the Kubernetes manifests in our GitHub Actions workflow aren’t kept up to date.