argoproj-labs / rollouts-plugin-trafficrouter-contour

The Argo Rollouts plugin implementing the Contour HTTPProxy traffic control in progressive delivery scenarios.

About Contour

Contour is an ingress controller for Kubernetes that works by deploying the Envoy proxy as a reverse proxy and load balancer. Contour supports dynamic configuration updates out of the box while maintaining a lightweight profile.

Contour supports multiple configuration APIs in order to meet the needs of as many users as possible:

• Ingress - A stable upstream API that enables basic ingress use cases.
• HTTPProxy - Contour's Custom Resource Definition (CRD) which expands upon the functionality of the Ingress API to allow for a richer user experience as well as solve shortcomings in the original design.
• Gateway API (beta) - A new CRD-based API managed by the Kubernetes SIG-Network community that aims to evolve Kubernetes service networking APIs in a vendor-neutral way.

How to integrate Contour with Argo Rollouts

Install Rollouts Using Helm

Add the following code to your valuse.yaml file when install the argo-rollouts by helm:

controller:
    initContainers:                                   
      - name: copy-contour-plugin
        image: release.daocloud.io/skoala/rollouts-plugin-trafficrouter-contour:v0.3.0
        command: ["/bin/sh", "-c"]                    
        args:
          - cp /bin/rollouts-plugin-trafficrouter-contour /plugins
        volumeMounts:                                 
          - name: contour-plugin
            mountPath: /plugins
    trafficRouterPlugins:                             
      trafficRouterPlugins: |-
        - name: argoproj-labs/contour
          location: "file:///plugins/rollouts-plugin-trafficrouter-contour"  
    volumes:                                           
      - name: contour-plugin
        emptyDir: {}
    volumeMounts:                                      
      - name: contour-plugin
        mountPath: /plugins

if argo-rollouts helm chart version >= [2.32.6], just set the providerRBAC.contour to true in the values.yaml file. Otherwise, you need to follow the steps below to create RBAC for operate on the HTTPProxy:

kubectl apply -f https://raw.githubusercontent.com/argoproj-labs/rollouts-plugin-trafficrouter-contour/main/yaml/rbac.yaml

or

kubectl patch clusterrole argo-rollouts --type='json' -p='[{"op": "add", "path": "/rules/-", "value": {"apiGroups":["projectcontour.io"],"resources":["httpproxies"],"verbs":["get","list","watch","update","patch","delete"]}}]'

Stand-alone installation

NOTES:

1. The file as follows (and the codes in it) just for illustrative purposes only, please do not use directly!

2. The argo-rollouts >= v1.5.0-rc1

Steps:

1. Run the yaml/rbac.yaml to add the role for operate on the HTTPProxy.
2. Build this plugin.
3. Put the plugin somewhere & mount on to the argo-rolloutscontainer (please refer to the example YAML below to modify the deployment):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: argo-rollouts
  namespace: argo-rollouts
spec:
  template:
    spec:
      ...
      volumes:
        ...
         - name: contour-plugin
           hostPath:
             path: /CHANGE-ME/rollouts-plugin-trafficrouter-contour
             type: ''
      containers:
        - name: argo-rollouts
        ...
          volumeMounts:
             - name: contour-plugin
               mountPath: /CHANGE-ME/rollouts-plugin-trafficrouter-contour

4. Create a ConfigMap to let argo-rollouts know the plugin's location:

apiVersion: v1
kind: ConfigMap
metadata:
  name: argo-rollouts-config
  namespace: argo-rollouts
data:
  trafficRouterPlugins: |-
    - name: "argoproj-labs/contour"
      location: "file://CHANGE-ME/rollouts-trafficrouter-contour-plugin/contour-plugin"
binaryData: {}

5. Create the CR/Rollout and put it into the operated services` namespace:

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: rollouts-demo
  namespace: rollouts-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app.kubernetes.io/instance: rollouts-demo
  strategy:
    canary:
      canaryService: canaryService
      stableService: stableService
      steps:
        - setWeight: 30
        - pause:
            duration: 10
      trafficRouting:
        plugins:
          argoproj-labs/contour:
            httpProxies:
              - rollouts-demo
  workloadRef:
    apiVersion: apps/v1
    kind: Deployment
    name: canary

6. Enjoy It.

Use it by Docker image

From v0.2.3, you can use this plugin from a init container, the plugin artifact location in the image is:

/bin/rollouts-plugin-trafficrouter-contour

The docker image with its artifact both support amd64 and arm64.

Contributing

Thanks for taking the time to join our community and start contributing!

• Please familiarize yourself with the Code of Conduct before contributing.
• Check out the open issues.

rollouts-plugin-trafficrouter-contour's Issues

Production ready

Hi,

I ended up here searching about argo rollout + contour, but I don't see Contour being part of the doc like Nginx, around here for example. So I wonder if this is production ready ?

Best regards,

New release needed

There's been no new release since January and a lot of improvements since then, can we get a 0.5.0 release tag pushed?

The plugin should only consider HTTPProxies from the same namespace as the Rollout resource

Summary

The plugin should only consider HTTPProxies from the same namespace as the Rollout resource.

Motivation

As of right now, the plugin is capable of changing the weight of HTTPProxies from other namespaces, i.e.:

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: api
  namespace: default
spec:
  strategy:
    canary:
      canaryService: api-canary
      stableService: api
      trafficRouting:
        plugins:
          argoproj-labs/contour:
            namespace: another-namespace
            httpProxy: api
      steps:
        - setWeight: 25

The Rollout above can alter the HTTPProxy from the another-namespace, even though it's in the default namespace.

This allows anyone to deploy a Rollout in any namespace and alter any HTTPProxy within the cluster.

Proposal

The plugin should be limited to HTTPProxies from the same namespace as the Rollout resource.

Argo Rollouts follow the same principle, all traffic management configurations only allow changing resources from the same namespace as the Rollout. As an example, we can turn to the NGINX doc:

The stable Ingress field is a reference to an Ingress in the same namespace of the Rollout

Implement the callback: SetHeaderRoute

Unable to find service in HTTPProxy with more than one route

Describe the bug

When trying to set the weight of an HTTPProxy with more than one route, it is only able to find the services of the first route:

'TrafficRoutingError' failed to set weight via plugin: the service: api-canary is not found in HTTPProxy

To Reproduce

Create an HTTPProxy with more than one route, and try to set the weight of the second route.

apiVersion: projectcontour.io/v1
kind: HTTPProxy
metadata:
  name: app
spec:
  virtualhost:
    fqdn: example.com
    tls:
      secretName: ingress
  routes:
    - conditions:
        - prefix: /
      services:
        - name: web
          port: 80
        - name: web-canary
          port: 80
    - conditions:
        - prefix: /api
      services:
        - name: api
          port: 80
        - name: api-canary
          port: 80

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: app
spec:
  strategy:
    canary:
      canaryService: api-canary
      stableService: api
      trafficRouting:
        plugins:
          argoproj-labs/contour:
            namespace: api
            httpProxy: api

Version

argoproj/argo-rollouts:v1.5.0
argoproj-labs/rollouts-plugin-trafficrouter-contour:v0.1.1

Handle multiple HTTPProxies

Summary

Handle multiple HTTPProxies

Motivation

We run contour with many HTTPProxies pointed at a single service. If we were to use this plugin in the current state, it would only adjust the weight of one HTTPProxy, leaving 100 weight on the rest of them.

Proposal

Instead of:

trafficRouting:
  plugins:
    argoproj-labs/contour:
      httpProxy: my-proxy

Configuration could look like:

trafficRouting:
  plugins:
    argoproj-labs/contour:
      httpProxies:
        - my-proxy
        - my-other-proxy

Then we can simply loop over this list of proxies.

Implement VerifyWeight function

Summary

Argo Rollouts weight verification feature allows the controller to verify the traffic routers after the setWeight step. This is useful to avoid problems when the Contour controller is unavailable or has issues updating the HTTPProxy.

Motivation

Currently, the plugin does not make any verification regarding this feature, as seen below:

rollouts-plugin-trafficrouter-contour/pkg/plugin/plugin.go

Lines 146 to 148 in bbfbd96

	func (r *RpcPlugin) VerifyWeight(rollout *v1alpha1.Rollout, desiredWeight int32, additionalDestinations []v1alpha1.WeightDestination) (pluginTypes.RpcVerified, pluginTypes.RpcError) {
	return pluginTypes.Verified, pluginTypes.RpcError{}
	}

Proposal

This verification can be implemented by looking at the status field of the HTTPProxy resource since the Contour operator constantly keeps it up-to-date with the proxy's current state.

Docs: need usage sample in README

Need a usage sample include yaml file and some steps in README.

No license file for the project

Update repo name

Hey @wilsonwu & @izturn

Just wanted to make you aware of argoproj/argo-rollouts#2720

We are updating the naming convention for plugins for rollouts.

So in your case, the repo would need to move to rollouts-plugin-trafficrouter-contour

I hope this isn't too much of an inconvenience.

No CI for unit test

Feature request: need lint

Set weight failed when HttpProxy has multi routes

Describe the bug
Set weight failed when HttpProxy has multi routes.
This is a part of HTTPProxy of conditions.

spec:
  ingressClassName: gateway-zeng-up
  routes:
    - conditions:
        - header:
            name: ':method'
            regex: GET|POST|DELETE|PUT|PATCH|OPTIONS|HEAD
          prefix: /
        - header:
            exact: eureka
            name: route
      jwtVerificationPolicy: {}
      loadBalancerPolicy:
        strategy: RoundRobin
      rateLimitPolicy: {}
      services:
        - name: sesame-b5cde76a93-sesame
          port: 30902
          weight: 100
    - conditions:
        - header:
            name: ':method'
            regex: GET|POST|DELETE|PUT|PATCH|OPTIONS|HEAD
          prefix: /
      jwtVerificationPolicy: {}
      loadBalancerPolicy:
        strategy: RoundRobin
      rateLimitPolicy: {}
      services:
        - name: rollouts-demo
          port: 8080
          weight: 100
        - name: rollouts-demo-canary
          port: 8080

Logs

time="2024-01-23T06:44:58Z" level=error msg="rollout syncHandler error: failed to set weight via plugin: failed to create patch : the total weight must equals to 100" namespace=skoala-zeng rollout=rollouts-demo
time="2024-01-23T06:44:58Z" level=info msg="rollout syncHandler queue retries: 258 : key \"skoala-zeng/rollouts-demo\"" namespace=skoala-zeng rollout=rollouts-demo
time="2024-01-23T06:44:58Z" level=error msg="failed to set weight via plugin: failed to create patch : the total weight must equals to 100\n" error="<nil>"

Feature request: need issue template

Feature request: need CI process

Publish the docker image into an org managed registry

Publish the docker image into an org managed registry.

Make the Log level is customizable

Finish the README.md

The READEME.md has the section which marked with TODO need to be done

[Feature] Support httpproxies with multiple canary routes

Is your feature request related to a problem? Please describe.
Given an HTTPProxy template that looks like the following:

    spec:
      routes:
      - conditions:
        - prefix: /foo
        services:
        - name: web
          weight: 100
        - name: web-canary
          weight: 0
      - conditions:
        - prefix: /bar
        services:
        - name: web
          weight: 100
        - name: web-canary
          weight: 0

We only update the weights on a single route
Describe the solution you'd like
We should update all of the canary services for a proxy across all routes
Describe alternatives you've considered

Additional context

How to update weight when HttpProxy has mutil services

Summary

when HttpProxy routes has mutil services, how to update service weight to make sure canary & stable service a determined proportion of traffic is assigned.

Motivation

httpProxy:

apiVersion: projectcontour.io/v1
kind: HTTPProxy
metadata:
  name: demo
spec:
  routes:
    - conditions:
        - header:
            name: ':method'
            regex: POST|DELETE|PUT|PATCH|OPTIONS|HEAD
          prefix: /user
      services:
        - name: stable-service
          port: 80
          weight: 10
        - name: canary-service
          port: 80
          weight: 90
        - name: other-service
          port: 80
          weight: 10  

when update stable & canary service weight, how to update other-service weight?

Proposal

maybe delete other-service or set other-service wight to 0 is a reasonable way.

or someone has some good idea?

The (YAML) codes in README.md doesn't work

The YAML codes in README.md's format is wrong, it needs to fix

[Feature] ARM architecture support

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Describe alternatives you've considered

Additional context