cybozu-go / accurate Goto Github PK

What

Using Server-Side Apply in Kubernetes requires that controllers set a non-dummy field manager. Below is an extract of a namespace created by the accurate controller in one of our clusters. As you can see, the field manager for fields set by accurate is marked with Go-http-client as the manager. This will cause issues if another controller does not set a non-default manager value. We also use managedFields a lot when debugging issues in our clusters.

Accurate controller should set a field manager value when creating/updating API resources. Suggested value accurate.

kind: Namespace
apiVersion: v1
metadata:
  name: erikbo-egb-test
  uid: f98a73f3-04dc-4abc-b673-f2b6970ca318
  resourceVersion: '698120538'
  creationTimestamp: '2023-09-25T14:20:40Z'
  labels:
    accurate.cybozu.com/parent: erikbo
    app.kubernetes.io/created-by: accurate
    kubernetes.io/metadata.name: erikbo-egb-test
  managedFields:
    - manager: Go-http-client
      operation: Update
      apiVersion: v1
      time: '2023-09-25T14:20:40Z'
      fieldsType: FieldsV1
      fieldsV1:
        'f:metadata':
          'f:labels':
            .: {}
            'f:accurate.cybozu.com/parent': {}
            'f:app.kubernetes.io/created-by': {}
            'f:kubernetes.io/metadata.name': {}

How

Describe how to address the issue.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository.

• WARN: Error updating branch

Errored

These updates encountered an error and will be retried. Click on a checkbox below to force a retry now.

• Update dependency goreleaser/goreleaser to v1.19.2
• Update dependency helm/helm to v3.12.2
• Update dependency kubernetes-sigs/controller-tools/controller-gen to v0.12.1
• Update dependency kubernetes-sigs/kubebuilder to v3.11.1
• Update dependency kubernetes/kubectl to v1.27.4
• Update dependency mikefarah/yq to v4.34.2
• Update dependency rust-lang/mdBook to v0.4.32
• Update kubernetes packages to v0.27.4 (patch) (k8s.io/api, k8s.io/apimachinery, k8s.io/cli-runtime, k8s.io/client-go)
• Update module github.com/onsi/gomega to v1.27.10
• Update dependency aquaproj/aqua to v2.10.1
• Update dependency aquaproj/aqua-registry to v4.31.0
• Update dependency aquaproj/aqua-renovate-config to v1.7.0
• Update dependency bitnami-labs/sealed-secrets to v0.23.0

Other Branches

These updates are pending. To force PRs open, click the checkbox below.

• Update renovatebot/github-action digest to da51f39
• Update rajatjindal/krew-release-bot action to v0.0.46
• Update goreleaser/goreleaser-action action to v4.3.0
• Update helm/chart-testing-action action to v2.4.0
• Update helm/kind-action action to v1.8.0
• Update actions/setup-go action to v4

Detected dependencies

What

Currently, Accurate only propagates labels and annotations whose key matches
exactly with one of the keys defined in the configuration file.

It'd be handy if it allows shell glob patterns such as *.cybozu.com/*.

How

Use path.Match.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

What

Currently, Accurate does not allow tenant users to set new labels/annotations to sub-namespaces created from SubNamespace objects.

It is handy to allow tenant users to set new labels/annotations only when:

1. the label/annotation key is included in the configuration file, and
2. the label/annotation is not set for the parent namespace.

How

1. Add new fields to SubNamespace, for example, spec.labels and spec.annotations
2. Enhance accurate-controller to create a sub-namespace with the specified labels/annotations.

accurate-controller has to ignore labels/annotations whose key is not listed in the configuration file.

It also should not set labels/annotations if the same labels/annotations are set for the parent namespace.
Nevertheless, the value would be overwritten by the existing Namespace controller to the value of the parent labels/annotations.

Describe how to address the issue.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

Describe the bug

The cluster role bound to the controller is too permissive by default, and I think this is a bug: https://github.com/cybozu-go/accurate/blob/main/charts/accurate/templates/generated/generated.yaml#L77-L84

If this permissive RBAC is required for the controller to operate, I think why should be documented.

We are evaluating this project as an alternative to HNC and might file a few issues/PRs for minor fixes. I hope there is a maintainer team with some bandwidth and interest in "external" contributions. 😄

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository.

• WARN: Error updating branch

Errored

These updates encountered an error and will be retried. Click on a checkbox below to force a retry now.

• Update dependency goreleaser/goreleaser to v1.19.2
• Update dependency helm/helm to v3.12.2
• Update dependency kubernetes-sigs/controller-tools/controller-gen to v0.12.1
• Update dependency kubernetes-sigs/kubebuilder to v3.11.1
• Update dependency kubernetes/kubectl to v1.27.4
• Update dependency mikefarah/yq to v4.34.2
• Update dependency rust-lang/mdBook to v0.4.32
• Update kubernetes packages to v0.27.4 (patch) (k8s.io/api, k8s.io/apimachinery, k8s.io/cli-runtime, k8s.io/client-go)
• Update module github.com/onsi/gomega to v1.27.9
• Update dependency aquaproj/aqua to v2.10.1
• Update dependency aquaproj/aqua-registry to v4.29.1
• Update dependency aquaproj/aqua-renovate-config to v1.7.0
• Update dependency bitnami-labs/sealed-secrets to v0.23.0

Other Branches

These updates are pending. To force PRs open, click the checkbox below.

• Update renovatebot/github-action digest to 80f3301
• Update rajatjindal/krew-release-bot action to v0.0.46
• Update goreleaser/goreleaser-action action to v4.3.0
• Update helm/chart-testing-action action to v2.4.0
• Update helm/kind-action action to v1.8.0
• Update actions/setup-go action to v4

Detected dependencies

Describe the bug
Admitedly, I did not yet try to reproduce the issue as I am not sure if this is intended behavior or not, but currently when configuring Accurate's labelKeys/annotationKeys, no check is performed as to whether Accurate's own labels and annotations have been specified.

For instance, config.Validate only checks if the labels/annotations are syntatically valid:

As a result, an administrator could set up Accurate's config.yaml to propagate Accurate's own labels and annotations and it would happily oblige.

Accurate controller's propagateMeta will indeed propagate all configured labels/annotations:

Granted, this is purely user misconfiguration, though I think that adding a check for the mistaken specification of labels/annotations in the constants.MetaPrefix namespace (or specific labels/annotations that should definitely not be configured to propagate?) would be more user-friendly.

Environments

• Version:
• OS:

To Reproduce
Steps that should reproduce the behavior:

1. Add the following to config.yaml

labelKeys:
- accurate.cybozu.com/type

2. Deploy Accurate
3. Add a SubNamespace
4. It should have inherited the accurate.cybozu.com/type=root from its parent namespace

Expected behavior
I am not exactly sure if this is expected behavior, but it does not seem like it is originally intended to happen.

Additional context

Describe the bug

The following documentation says kubectl accurate template list is available to show all template namespaces.

https://cybozu-go.github.io/accurate/info.html#show-all-template-namespaces

However, kubectl accurate template does not have list subcommand.

Environments

• Version: v0.1.0
• OS: Ubuntu 18.04.5 LTS (x86_64)

To Reproduce

1. Download kubectl-accurate-list-amd64
   https://github.com/cybozu-go/accurate/releases/download/v0.1.0/kubectl-accurate-linux-amd64
2. Run ./kubectl-accurate-linux-amd64 template list

Expected behavior

All template namespaces are shown.

Additional context

sho_iizuka@s000817-bionic:/tmp$ ./kubectl-accurate-linux-amd64 template list
template subcommand

Usage:
  accurate template [command]

Available Commands:
  set         Set TEMPLATE as the template of NS namespace
  unset       Unset template for NS namespace

Flags:
  -h, --help   help for template

Use "accurate template [command] --help" for more information about a command.
sho_iizuka@s000817-bionic:/tmp$

What

Support krew release.

How

refs:
https://krew.sigs.k8s.io/docs/developer-guide/plugin-manifest/
https://krew.sigs.k8s.io/docs/developer-guide/release/automating-updates/

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

What

Just like kubectl accurate list that shows the list of root namespaces with its sub-namespaces hierarchically.

Also, add kubectl-accurate sub list as an alias for kubectl-accurate list.

How

Describe how to address the issue.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

What

Currently, the default manifest does not include RBAC to propagate ResourceQuota.
This is noted in the user manual.

ResourceQuota is very popular for soft multi-tenancy environments, so the RBAC should be included by default.

How

Describe how to address the issue.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

What

Currently, Accurate does not impose any naming policy on SubNamespaces.
This means that tenant users can name their sub-namespaces freely, which would cause chaos.

So, implement a naming policy and impose it with a validating webhook for SubNamespace.

How

This is just a thought. Add the following field to the configuration file.

namingPolicies:
- root: "tenant1"
  match: "tenant1-.*"
- root: ".*"
  match: "dev-.*"

root and match are both regular expressions.
When a SubNamespace is created in a tree starting from a root namespace and the root namespace's name matches root expression, the SubNamespace name is validated with match expression.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

What

When attempting to delete a namespace with subnamespaces, this is rejected by the webhook - which is good. But I think webhooks should be named properly - so a user can understand better what's going on.

$ k delete namespaces erikbo
Error from server (Forbidden): admission webhook "vnamespace.kb.io" denied the request: child namespaces exist

How

Describe how to address the issue.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository.

• WARN: Error updating branch

Errored

These updates encountered an error and will be retried. Click on a checkbox below to force a retry now.

• Update dependency aquaproj/aqua to v2.9.1
• Update dependency goreleaser/goreleaser to v1.19.2
• Update dependency helm/helm to v3.12.2
• Update dependency kubernetes-sigs/controller-tools/controller-gen to v0.12.1
• Update dependency kubernetes-sigs/kubebuilder to v3.11.1
• Update dependency mikefarah/yq to v4.34.2
• Update dependency rust-lang/mdBook to v0.4.32
• Update dependency aquaproj/aqua-registry to v4.27.0

Other Branches

These updates are pending. To force PRs open, click the checkbox below.

• Update renovatebot/github-action digest to 184f0e6
• Update rajatjindal/krew-release-bot action to v0.0.46
• Update goreleaser/goreleaser-action action to v4.3.0
• Update helm/chart-testing-action action to v2.4.0
• Update helm/kind-action action to v1.8.0
• Update actions/setup-go action to v4

Detected dependencies

What

The controller manager needs permission to view all resources in order to check the annotation for resource propagation.

How

Add permission to view all resources to manager-role

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

Renovate tried to run on this repository, but found these problems.

• WARN: Error updating branch

Errored

These updates encountered an error and will be retried. Click on a checkbox below to force a retry now.

• Update dependency helm/helm to v3.12.3
• Update dependency kubernetes-sigs/kubebuilder to v3.11.1
• Update dependency kubernetes-sigs/kustomize to v5.1.1
• Update dependency kubernetes/kubectl to v1.27.5
• Update dependency rust-lang/mdBook to v0.4.34
• Update kubernetes packages to v0.27.5 (patch) (k8s.io/api, k8s.io/apimachinery, k8s.io/cli-runtime, k8s.io/client-go)
• Update module github.com/onsi/gomega to v1.27.10
• Update dependency aquaproj/aqua to v2.10.1
• Update dependency aquaproj/aqua-registry to v4.44.0
• Update dependency aquaproj/aqua-renovate-config to v1.8.0
• Update dependency bitnami-labs/sealed-secrets to v0.23.1
• Update dependency goreleaser/goreleaser to v1.20.0
• Update dependency kubernetes-sigs/controller-tools/controller-gen to v0.13.0
• Update dependency mikefarah/yq to v4.35.1
• Update kubernetes packages to v0.28.1 (minor) (k8s.io/api, k8s.io/apimachinery, k8s.io/cli-runtime, k8s.io/client-go)
• Update module github.com/onsi/ginkgo/v2 to v2.12.0
• Update module go.uber.org/zap to v1.25.0
• Update module sigs.k8s.io/controller-runtime to v0.16.1

Other Branches

These updates are pending. To force PRs open, click the checkbox below.

• Update renovatebot/github-action digest to f1d7014
• Update rajatjindal/krew-release-bot action to v0.0.46
• Update goreleaser/goreleaser-action action to v4.4.0
• Update helm/chart-testing-action action to v2.4.0
• Update helm/kind-action action to v1.8.0
• Update actions/setup-go action to v4

Detected dependencies

What

When a SubNamespace is deleted, a related Namespace is deleted by the accurate controller.

However, sometimes the Namespace deletion does not complete, and it remains terminating.
At this time, the owner of the SbuNamespace may not manage the terminating namespace.

The reason why the Namespace cannot be deleted is that some namespaced resources in it cannot be deleted due to the finalizer or any other reasons.

How

Add validation when deleting a SubNamespace.
If there are any namespaced resources left in the related namespace, deny the deletion.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository.

• WARN: Error updating branch

Errored

These updates encountered an error and will be retried. Click on a checkbox below to force a retry now.

• Update dependency goreleaser/goreleaser to v1.19.2
• Update dependency helm/helm to v3.12.2
• Update dependency kubernetes-sigs/controller-tools/controller-gen to v0.12.1
• Update dependency kubernetes-sigs/kubebuilder to v3.11.1
• Update dependency kubernetes-sigs/kustomize to v5.1.1
• Update dependency kubernetes/kubectl to v1.27.4
• Update dependency mikefarah/yq to v4.34.2
• Update dependency rust-lang/mdBook to v0.4.34
• Update kubernetes packages to v0.27.4 (patch) (k8s.io/api, k8s.io/apimachinery, k8s.io/cli-runtime, k8s.io/client-go)
• Update module github.com/onsi/gomega to v1.27.10
• Update module sigs.k8s.io/controller-runtime to v0.15.1
• Update dependency aquaproj/aqua to v2.10.1
• Update dependency aquaproj/aqua-registry to v4.33.0
• Update dependency aquaproj/aqua-renovate-config to v1.7.0
• Update dependency bitnami-labs/sealed-secrets to v0.23.0
• Update module go.uber.org/zap to v1.25.0

Other Branches

These updates are pending. To force PRs open, click the checkbox below.

• Update renovatebot/github-action digest to 14937ce
• Update rajatjindal/krew-release-bot action to v0.0.46
• Update goreleaser/goreleaser-action action to v4.3.0
• Update helm/chart-testing-action action to v2.4.0
• Update helm/kind-action action to v1.8.0
• Update actions/setup-go action to v4

Detected dependencies

What

It's a bit tedious to customize manifests using kustomize.
Replace them with Helm charts.

How

Describe how to address the issue.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

What

As a cluster-admin I want to onboard and eventually terminate tenants in our cluster by creating and eventually deleting a root namespace for the tenant. The onboarding process works like a charm, but I am not too happy about the termination process - which we had an opportunity to try out this week. Let me explain:

We use a GitOps process where tenants are onboarded via a pull request to our "tenants" project. A tenant is in this project defined by some simple resources:

• An Accurate root namespace with some templated resources defined in our tenant-template Accurate template namespace.
• A Flux gitops-reconciler service account granted admin permissions in the tenant root namespace. Both the SA and admin role binding are configured with propagation to sub-namespaces. This allows the tenant to use Flux to provision most resources using a GitOps process in their namespace tree.
• Flux GitRepository and Kustomization resources pointing to a Git project controlled by the tenant - allowing the tenant to bootstrap their resources.

So far, so good. But this week we received a request for a tenant termination. Using a modern GitOps tool like Flux, with pruning enabled, we thought that it was as simple as reverting the onboarding PR in Git. So we did that, after getting the PR approved by the tenant responsible. What we forgot to do, was to check if there were sub-namespaces defined under the tenant root namespace. After merging the tenant termination PR, Flux immediately reported an error: It got (correctly) blocked by the Accurate namespace webhook:

delete failed, errors: Namespace/blnc delete failed: admission webhook "namespace.accurate.cybozu.io" denied the request: child namespaces exist;
kustomization/flux-tenants.flux-tenants

But this error was reported only once, which kind of surprised me - as Flux is usually constantly reconciling until the actual state equals the desired state. And as I suspected, the tenant namespace was still present - including the child namespaces that we did not think about. However the Flux controller resource had no knowledge of the resources it used to control anymore, which left the tenant root namespace (including children) as orphans in our cluster. This is something we are trying hard to avoid.

After cleaning up manually, I reached out to the Flux maintainers on Slack, and you can read all the details in the CNCF Slack thread - if you are interested. TL;DR: Flux maintainers think this is an issue with Accurate, and I tend to agree now.

To fix this for future tenant terminations in our clusters, I suggest adding an opt-in allowing us to configure the Accurate namespace webhook allowing cascade namespace DELETE requests.

How

I have some ideas after looking at the code, but please let me know what you think first! I will update this description once we agree on a solution. I'll be happy to submit a PR fixing this.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository.

• WARN: Error updating branch

Errored

These updates encountered an error and will be retried. Click on a checkbox below to force a retry now.

• Update dependency helm/helm to v3.12.3
• Update dependency kubernetes-sigs/kubebuilder to v3.11.1
• Update dependency kubernetes-sigs/kustomize to v5.1.1
• Update dependency kubernetes/kubectl to v1.27.4
• Update dependency rust-lang/mdBook to v0.4.34
• Update kubernetes packages to v0.27.4 (patch) (k8s.io/api, k8s.io/apimachinery, k8s.io/cli-runtime, k8s.io/client-go)
• Update module github.com/onsi/gomega to v1.27.10
• Update module sigs.k8s.io/controller-runtime to v0.15.1
• Update dependency aquaproj/aqua to v2.10.1
• Update dependency aquaproj/aqua-registry to v4.40.0
• Update dependency aquaproj/aqua-renovate-config to v1.8.0
• Update dependency bitnami-labs/sealed-secrets to v0.23.1
• Update dependency goreleaser/goreleaser to v1.20.0
• Update dependency kubernetes-sigs/controller-tools/controller-gen to v0.13.0
• Update dependency mikefarah/yq to v4.35.1
• Update kubernetes packages to v0.28.0 (minor) (k8s.io/api, k8s.io/apimachinery, k8s.io/cli-runtime, k8s.io/client-go)
• Update module go.uber.org/zap to v1.25.0

Other Branches

These updates are pending. To force PRs open, click the checkbox below.

• Update renovatebot/github-action digest to dc88b89
• Update rajatjindal/krew-release-bot action to v0.0.46
• Update goreleaser/goreleaser-action action to v4.4.0
• Update helm/chart-testing-action action to v2.4.0
• Update helm/kind-action action to v1.8.0
• Update actions/setup-go action to v4

Detected dependencies

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository.

• WARN: Error updating branch

Errored

These updates encountered an error and will be retried. Click on a checkbox below to force a retry now.

• Update dependency goreleaser/goreleaser to v1.19.2
• Update dependency kubernetes-sigs/kubebuilder to v3.11.1
• Update dependency aquaproj/aqua-registry to v4.25.0

Other Branches

These updates are pending. To force PRs open, click the checkbox below.

• Update renovatebot/github-action digest to 28ff2ef
• Update rajatjindal/krew-release-bot action to v0.0.46
• Update goreleaser/goreleaser-action action to v4.3.0
• Update helm/chart-testing-action action to v2.4.0
• Update helm/kind-action action to v1.7.0
• Update actions/setup-go action to v4

Detected dependencies

What

This is not possible for the time being.
This is possible with Kubernetes 1.26 or better.
ref: kubernetes/kubernetes#105867

spf13/cobra can auto-generate shell completion rules, so add completion
subcommand to kubectl-accurate for Bash, Zsh, PowerShell, Fish, etc.

Note that it should work when invoked as kubectl accurate ....

How

Describe how to address the issue.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

Describe the bug

NamingPolicy is now supported in #17.
However, namingpolicies in values.yaml is not reflected in configmap.

Because namingpolicies is not described in configmap.yaml.
https://github.com/cybozu-go/accurate/blob/da7508cf549fd7c49668d515e759345d6c6824a6/charts/accurate/templates/configmap.yaml

Environments

• Version: 0.3.0
• Chart Version: 0.2.2

To Reproduce
Steps to reproduce the behavior:

1. Prepare a values.yaml containing namingPolicies as below:

controller:
  config:
    namingPolicies:
      - root:  foo
        match: foo_.*

2. Deploy accurate using Helm with the values.yaml.

helm install --create-namespace --namespace accurate accurate -f values.yaml accurate/accurate

3. The generated configmap does not contain namingPolicies

Expected behavior
The configmap should contain namingPolicies.

Describe the bug

We have installed the latest version of Accurate (1.3.0), which includes migration to SSA (#112). While everything seems to work well, we have noticed that pre-existing resources (from before we installed the release with migration to SSA) have some suspicious managedFields. Example for a namespace controlled by a sub-namespace:

managedFields:
  - manager: accurate-controller
    operation: Apply
    apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      'f:metadata':
        'f:annotations':
          'f:collectord.io/index': {}
        'f:labels':
          'f:trust.statnett.no/inject-statnett-bundle': {}
  - manager: accurate-controller
    operation: Update
    apiVersion: v1
    time: '2023-11-08T15:01:07Z'
    fieldsType: FieldsV1
    fieldsV1:
      'f:metadata':
        'f:annotations':
          .: {}
          'f:collectord.io/index': {}
        'f:labels':
          .: {}
          'f:accurate.cybozu.com/parent': {}
          'f:app.kubernetes.io/created-by': {}
          'f:kubernetes.io/metadata.name': {}
          'f:trust.statnett.no/inject-statnett-bundle': {}

This results in unset fields in the SSA patch to retain in the owned resource, which is the issue the migration to SSA PR was supposed to solve (#98).

Environments

N/A

To Reproduce

Steps to reproduce the behavior:

1. Install one of the previous versions of Accurate before we migrated to SSA.
2. Create a SubNamespace adding a label/annotation to the sub-namespace.
3. Upgrade Accurate to the latest version including migration to SSA
4. Remove sub-namespace a label/annotation from SubNamespace created in 2.
5. Observe the label/annotation is still present on sub-namespace.

Expected behavior

The label/annotation removed from SubNamespace spec should be removed from sub-namespace.

Additional context

I know how to fix this and will prepare a PR. 🤠

What

Describe what this issue should address.

How

Describe how to address the issue.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

Describe the bug
When a propagated label/annotation is deleted from a parent namespace, the deletion is not propagated to its child namespaces.

Environments

• Version:
• OS:

To Reproduce
Steps to reproduce the behavior:

1. Configure Accurate to propagate the baz.glob/c label via config.yaml
2. Deploy Accurate
3. Create a root namespace with the baz.glob/c label and a child namespace
4. Delete the baz.glob/c label from the root namespace
5. Observe that the label is still present in the child namespace

Expected behavior
From a developer somewhat familiar with the inner workings of the controller's perspective, it makes sense that propagating deletions of labels/annotations is non-trivial. Furthermore, the absence of a label/annotation on the parent (via deletion of omission) does not necessarily imply that it must be deleted on the child namespace, so user intent is also hard to grasp.

On the other hand, from a reader's perspective, propagation means to me that creation, update and deletion is reconciled.

This can be addressed either by:

• implementing the propagation of label/annotation deletion
• updating the manual to clarify that the propagation of label/annotation deletions is not supported

As a predictable implementation of deletion propagation is non-trivial and may break user expectations, I am leaning towards the latter. Any opinions?

Additional context
Add any other context about the problem here.

What

We are òbserving some mysterious errors in the Accurate pods:

2023/10/26 13:30:27 http: TLS handshake error from 10.101.10.1:53540: EOF
2023/10/26 13:30:27 http: TLS handshake error from 10.101.10.1:53546: EOF
2023/10/26 13:40:41 http: TLS handshake error from 10.101.10.1:47184: EOF
2023/10/26 13:40:41 http: TLS handshake error from 10.101.10.1:47194: EOF
2023/10/26 14:20:31 http: TLS handshake error from 10.101.10.1:41206: EOF
2023/10/26 15:20:41 http: TLS handshake error from 10.101.10.1:58124: EOF
2023/10/26 15:39:52 http: TLS handshake error from 10.101.10.1:58912: read tcp 10.101.13.130:9443->10.101.10.1:58912: read: connection reset by peer
2023/10/26 15:50:17 http: TLS handshake error from 10.101.10.1:53224: EOF
2023/10/26 16:20:22 http: TLS handshake error from 10.101.10.1:47576: EOF

Are anyone else observing something similar? The operator seems to work as it should, so this is just annoying messages in the logs. I suspect this might be related to kubernetes/kubernetes#109022, and some suspect the root issue to be golang/go#50984.

I'll suggest addressing #103 to see if that can fix this issue.

How

Describe how to address the issue.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

What

The image used to build the operator binary seems to be a Cybozu Ubuntu Jammy based image:

Is there any particular reason for not using the official Go image, https://hub.docker.com/_/golang?

How

Replace the Go binary build image with golang:1.21, eventually an exact image digest if we want reproducible builds.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

What

We have been using accurate in production environments for a while.
Since there are no high priority issues, let's declare GA status and release v1.0.0.

How

Describe how to address the issue.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

What

We use FluxCD to provision our clusters, and FluxCD supports health checks for CRDs compatible with kstatus. And I believe ArgoCD also does this.

It would be nice if the SubNamespace CRD had a status subresource compatible with kstatus. You can read more details in the specification, but I would suggest the following requirements (at least as a start):

• A SubNamespace with empty status has not yet been picked up by the controller
• The controller should ensure .status.obeservedGeneration is equal to .metadata.generation when reconciling a SubNamespace
• If the actual state differs from the desired state, the controller should add status conditions indicating that a reconcile is in progress
• When the actual state equals the desired status, the controller should remove all status conditions
• If the controller is unable to reconcile successfully for any reason, like in a conflict situation, the controller should add conditions indicating what's wrong.

Since the status field on SubNamespace currently is a simple string, this will be a breaking change of status.

How

Describe how to address the issue.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

What

After adding a new watched resource type, we forgot to grant the controller RBAC to operator on the new resources. This made the operator workload roll over apparently with success. But after checking the logs, I see that it emits tons of errors. Example:

{"level":"error","ts":"2023-10-19T11:29:09Z","msg":"Reconciler error","controller":"namespace","controllerGroup":"","controllerKind":"Namespace","Namespace":{"name":"bsrv"},"namespace":"","name":"bsrv","reconcileID":"d8c88184-5aa8-4de9-961c-c57776f6bed0","error":"failed to reconcile a namespace: failed to propagate resource bsrv/default-limits of /v1, Kind=LimitRange with propagate=create: limitranges is forbidden: User \"system:serviceaccount:accurate:accurate-controller-manager\" cannot create resource \"limitranges\" in API group \"\" in the namespace \"bsrv\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller...

How

I think the controller should perform a check on startup, to see if it has the required RBAC to the configured watched resources, and crash/panic if it doesn't. The check could be based on the SelfSubjectAccessReview API.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

What

We are performing a POC of Accurate, and one of our goals is to allow end-users to set selected NS labels/annotations. This seems to work very well, but I could not find the precedence between labelKeys/subNamespaceLabelKeys and annotationKeys/subNamespaceAnnotationKeys documented anywhere. Looking at the code this looks as expected and I have also verified this in our cluster:

But I still think the precedence should be clearly documented (and tested) as a commitment to the current behavior.

How

• Document that SubNamespace annotations/labels will take precedence over any labels/annotations propagated from parent-NS to sub-NS.
• Add tests for this behavior (if not already exists)

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

What

The current implementation can propagate ServiceAccounts, but the propagated ServiceAccounts
won't work because the Secret tokens issued for them in the parent namespace are not propagated.

The issued Secrets do not have metadata.ownerRerefences, so accurate.cybozu.com/propagate-generated cannot be used either.

How

Propagate ServiceAccounts without secrets field.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

What

Support Kubernetes 1.24

How

Describe how to address the issue.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

Renovate tried to run on this repository, but found these problems.

• WARN: Error updating branch

Errored

These updates encountered an error and will be retried. Click on a checkbox below to force a retry now.

• Update dependency helm/helm to v3.12.3
• Update dependency kubernetes-sigs/kubebuilder to v3.11.1
• Update dependency kubernetes-sigs/kustomize to v5.1.1
• Update dependency kubernetes/kubectl to v1.27.5
• Update dependency rust-lang/mdBook to v0.4.34
• Update kubernetes packages to v0.27.5 (patch) (k8s.io/api, k8s.io/apimachinery, k8s.io/cli-runtime, k8s.io/client-go)
• Update module github.com/onsi/gomega to v1.27.10
• Update dependency aquaproj/aqua to v2.10.1
• Update dependency aquaproj/aqua-registry to v4.42.0
• Update dependency aquaproj/aqua-renovate-config to v1.8.0
• Update dependency bitnami-labs/sealed-secrets to v0.23.1
• Update dependency goreleaser/goreleaser to v1.20.0
• Update dependency kubernetes-sigs/controller-tools/controller-gen to v0.13.0
• Update dependency mikefarah/yq to v4.35.1
• Update kubernetes packages to v0.28.1 (minor) (k8s.io/api, k8s.io/apimachinery, k8s.io/cli-runtime, k8s.io/client-go)
• Update module github.com/onsi/ginkgo/v2 to v2.12.0
• Update module go.uber.org/zap to v1.25.0
• Update module sigs.k8s.io/controller-runtime to v0.16.0

Other Branches

These updates are pending. To force PRs open, click the checkbox below.

• Update renovatebot/github-action digest to ef8c78a
• Update rajatjindal/krew-release-bot action to v0.0.46
• Update goreleaser/goreleaser-action action to v4.4.0
• Update helm/chart-testing-action action to v2.4.0
• Update helm/kind-action action to v1.8.0
• Update actions/setup-go action to v4

Detected dependencies

Describe the bug

Secret "bar" sometimes not found in the namespace controller test.

https://github.com/cybozu-go/accurate/runs/3891153514?check_suite_focus=true

• Failure [0.614 seconds]
Namespace controller
/home/runner/work/accurate/accurate/controllers/namespace_controller_test.go:39
  should implement template namespace correctly [It]
  /home/runner/work/accurate/accurate/controllers/namespace_controller_test.go:84

  Unexpected error:
      <*errors.StatusError | 0xc000a69ea0>: {
          ErrStatus: {
              TypeMeta: {Kind: "", APIVersion: ""},
              ListMeta: {
                  SelfLink: "",
                  ResourceVersion: "",
                  Continue: "",
                  RemainingItemCount: nil,
              },
              Status: "Failure",
              Message: "secrets \"bar\" not found",
              Reason: "NotFound",
              Details: {Name: "bar", Group: "", Kind: "secrets", UID: "", Causes: nil, RetryAfterSeconds: 0},
              Code: 404,
          },
      }
      secrets "bar" not found
  occurred

  /home/runner/work/accurate/accurate/controllers/namespace_controller_test.go:199
------------------------------
SSSTEP: tearing down the test environment


Summarizing 1 Failure:

[Fail] Namespace controller [It] should implement template namespace correctly 
/home/runner/work/accurate/accurate/controllers/namespace_controller_test.go:199

Environments

• Version:
• OS:

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

What

The example of using propagate-generated is outdated:

As both cert-manager and sealed-secrets support adding labels/annotations to dependents:

• cert-manager secretTemplate
• sealed-secrets template

How

I actually wanted to submit a PR to update the docs with a more relevant example, but I am struggling to find any public operator that creates dependents without the ability to configure the owner to add labels/annotations to the dependents. So in order to resolve this doc issue, a good example of using this feature is required.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

What

Accurate can propagate any namespace resource between namespaces.
While copying the resource, all labels and annotations except for ones that contain kubernetes.io/ are inherited.

In some cases, the copied labels or annotations can cause problems.
For instance, if Argo CD is configured to track the managed resources by argocd.argoproj.io/instance label and the parent resource was created by Argo CD, the propagated resource would have the same label. Argo CD then tries to delete the propagated resource because the propagated resource is not found on the source Git repository.

So, add a feature to exclude particular labels or annotations from propagated resources.

How

Describe how to address the issue.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

What

Ref. docs:

Accurate currently does not delete previously propagated labels when deleted from the parent namespace to prevent unintended deletions. Users are expected to manually delete labels/annotations that are no longer needed.

I wonder which kind of "unintended deletions" are referred to here. I would expect Accurate to clean up a previously propagated namespace label/annotation (based on Kubernetes managedFields ownership) when the desired state of parent/template NS (or SubNamespace) indicates that a label/annotation should not be propagated.

Could a migration to SSA (Server Side Apply) aid in this? Since controller-runtime still has limited support for SSA, ref. kubernetes-sigs/controller-runtime#347, we probably must migrate to Unstructured instead of structured types. But IMO I think it would be worth it.

How

Describe how to address the issue.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository.

• WARN: Error updating branch

Errored

These updates encountered an error and will be retried. Click on a checkbox below to force a retry now.

• Update dependency helm/helm to v3.12.3
• Update dependency kubernetes-sigs/controller-tools/controller-gen to v0.12.1
• Update dependency kubernetes-sigs/kubebuilder to v3.11.1
• Update dependency kubernetes-sigs/kustomize to v5.1.1
• Update dependency kubernetes/kubectl to v1.27.4
• Update dependency rust-lang/mdBook to v0.4.34
• Update kubernetes packages to v0.27.4 (patch) (k8s.io/api, k8s.io/apimachinery, k8s.io/cli-runtime, k8s.io/client-go)
• Update module github.com/onsi/gomega to v1.27.10
• Update module sigs.k8s.io/controller-runtime to v0.15.1
• Update dependency aquaproj/aqua to v2.10.1
• Update dependency aquaproj/aqua-registry to v4.37.0
• Update dependency aquaproj/aqua-renovate-config to v1.8.0
• Update dependency bitnami-labs/sealed-secrets to v0.23.0
• Update dependency goreleaser/goreleaser to v1.20.0
• Update dependency mikefarah/yq to v4.35.1
• Update module go.uber.org/zap to v1.25.0

Other Branches

These updates are pending. To force PRs open, click the checkbox below.

• Update renovatebot/github-action digest to 353aacc
• Update rajatjindal/krew-release-bot action to v0.0.46
• Update goreleaser/goreleaser-action action to v4.4.0
• Update helm/chart-testing-action action to v2.4.0
• Update helm/kind-action action to v1.8.0
• Update actions/setup-go action to v4

Detected dependencies

What

If you want your repositories to be labeled as Verified Publisher, you can add a to each of them including the repository ID provided below. This label will let users know that you own or have control over the repository. The repository metadata file must be located at the path used in the repository URL.
https://artifacthub.io/control-panel/repositories?page=1

How

https://artifacthub.io/docs/topics/repositories/#ownership-claim

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

What

We find this project very useful and well-made, and we think it could be adopted by many more. When we consider adopting and/or contributing to new projects, it's a big plus if the project is a CNCF project. Others might think alike, and I would therefore suggest becoming a CNCF project.

How

If there is an interest in this, I found the following getting started resources:

• https://github.com/cncf/toc/blob/main/process/project_proposals.md
• https://github.com/cncf/toc/blob/main/process/sandbox.md

Describe the bug

We are running a proof-of-concept in one of our OpenShift clusters. And while the controller runs find without any watches, it fails badly when we enable watches of roles and role bindings. This issue might be relevant for other resource types, but namespace RBAC is the obvious start for propagating namespace resources for us.

It seems like the controller attempts to annotate resources in namespaces that are not annotated with any accurate.cybozu.com annotations, and this must be wrong! Errors are emitted constantly, and here is an example:

{"level":"error","ts":"2023-09-25T18:21:37Z","msg":"failed to check the controller reference","controller":"role","controllerGroup":"rbac.authorization.k8s.io","controllerKind":"Role","Role":{"name":"cluster-samples-operator","namespace":"openshift-cluster-samples-operator"},"namespace":"openshift-cluster-samples-operator","name":"cluster-samples-operator","reconcileID":"957d956a-b15e-4463-a458-ca5b1a32582b","error":"failed to add accurate.cybozu.com/propagate-generated annotation: roles.rbac.authorization.k8s.io \"cluster-samples-operator\" is forbidden: user \"system:serviceaccount:accurate:accurate-controller-manager\" (groups=[\"system:serviceaccounts\" \"system:serviceaccounts:accurate\" \"system:authenticated\"]) is attempting to grant RBAC permissions not currently held"

This is a total blocker for us. The controller must NOT touch resources in namespaces that are irrelevant for Accurate.

Environments

• Version: 1.1.0
• OS: N/A

To Reproduce

1. Install Accurate
2. Configure Accurate to watch roles and role bindings
3. Observe that Accurate adds (or attempts to add) annotations to resources in namespaces that are not observed/managed by Accurate. On Openshift this update is unsuccessful and the controller logs become extremely chatty.

Expected behavior

No attempt to update irrelevant namespace resources and no errors in controller logs.

Additional context

The controller errors might only occur on Openshift, which is a secure by default Kubernetes distribution. But the root cause here is the wrong attempt to change irrelevant resources.