Document RBAC permissions for Metacontroller about metacontroller

[rlguarino]

You should be able to give MetaController the minimal set of permissions it needs to do work. The strictest set of things that MetaController needs to access is its CRDs, but it wouldn't be very useful.

Since MetaController manages resources on behalf of its child controllers, MetaController needs rights to manage any of the resources that a child controller manages. So if there's no child controller that needs to list or manage pods then MetaController does not need access to Pods at all.

We could use an Aggregate role for meta controller and it could be standard practice that controller authors would package RBAC Roles that they need.

That could look something like this. We would package the following Aggregate Role with MetaController:

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: metacontroller
aggregationRule:
  clusterRoleSelectors:
  - matchLabels:
      rbac.authorization.k8s.io/aggregate-to-metacontroller: "true"
rules: 
- apiGroups: ["metacontroller.k8s.io"]
  resources: ["compositecontroller", "decoratorcontroller", "controllerrevisions"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

Then a controller author would create and distribute the following with their app:

In this example the controller needs to manage deployments resouces and the parent resources is a custom CRD.

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: aggregate-cron-tabs-view
  labels:
    # Add these permissions to the "metacontroller" default role.
    rbac.authorization.k8s.io/aggregate-to-metacontroller: "true"
rules:
- apiGroups: ["example.com"]
  resources: ["bluegreendeployment"]
  verbs: ["get", "list", "watch", "patch"]
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

[enisoc]

@Kalise Metacontroller can be run in any namespace. The main reason we suggest creating a new namespace is because there are cross-references in the installation and example controller manifests that need to specify an explicit namespace where things live. If we didn't create our own namespace with a known name, we'd have to ask you to fill in the namespace you chose in a bunch of places. I think it's also now discouraged to put extensions in kube-system, which they'd like to reserve for core k8s components.

@rlguarino Interesting idea! I didn't even know about aggregate roles. Another possibility I was thinking about is giving each controller its own service account, and having Metacontroller "act as" that service account when performing writes on behalf of that controller. This would let admins restrict permissions to only certain controllers.

However, it's a bit tricky to achieve this restriction for reads, since they are globally cached. One idea I had for that is to have Metacontroller perform manual authz checks for reads (with the per-controller service account) before fetching values from the cache on behalf of a given controller. We could cache the authz result with some TTL, which I believe webhook auth also does, to reduce query load.

I hadn't yet worked out how to give Metacontroller permission to populate the global cache on behalf of the set of controllers that need a given resource, but the aggregation feature you mentioned seems like it would help with that.

[rlguarino]

Are you familiar with how the kubernetes Controller-Manager work with RBAC? I think what you've suggested with using different service account for every controller sounds a lot like how k8s controller-manager works. I'll have to look into what they do about reads, because I thought they were using SharedInformers.

[enisoc]

Yeah my service account idea was intended to be a generalization of what kube-controller-manager does.

I guess it would make sense if they don't bother restricting cached reads in controller-manager, since the code running there is trusted. For example, even if StatefulSet became a confused deputy, we're pretty sure it doesn't read Secrets because the code doesn't call that API, so it would be unlikely to leak Secrets if compromised.

By contrast, Metacontroller is a multi-tenant service. Admins probably don't want to give the same global read access to all users who have permission to install controllers via Metacontroller.

[rlguarino]

Depending on assumptions of scale and the churn rate of a cluster it might be simpler and more robust to not use SharedInformers at all. We would almost certainly end up with more watches but no more than you would end up with if each controller was deployed independently without metacontroller.

[bsctl]

@rlguarino @enisoc Aggregate roles can be an immediate and good solution.
Also the Controller Manager approach sounds interesting.

Metacontroller is a multi-tenant service. Admins probably don't want to give the same global read access to all users who have permission to install controllers via Metacontroller.

As a system admin, completely agree. Thanks!

[terinjokes]

@enisoc It looks like (based on #9) that some resolution for this was expected in Q3 2018. Have you been able to experiment with solutions proposed here?