openebs-archive / helm-operator

Helm Operator for OpenEBS Installation

OpenEBS Helm Operator

Introduction

• This is a helm operator for openebs
• The content of values.yaml will be placed as spec items in OpenEBSInstallTemplate CR
• The operator watches all namespaces. There is no need for helm client/tiller to be installed. (the helm operator imports code from helm project)
• Changes to spec of the CR will behave similar to values override via helm
• All OpenEBS resources have ownerReference of OpenEBSInstallTemplate & are deleted upon removal of CR

Example of OpenEBSInstallTemplate CR can be found here

Limitations

• Release names are auto-generated based on CR UID. Impact is character-length will exceed limit if CR name is longer.

  • Workaround: Give shorter nams:
  • ER: operator-framework/operator-sdk#1094

• Release namespace doesn't have a field in the spec. (need more exploration)

  • Workaround: Create ns "openebs" or any desired before applying the CR

• Uninstall of a failed helm release (say, wrong spec) will be stuck.

  • Workaround: finalizer has to be removed manually
  • Snippet:

    status:
    conditions:
    - lastTransitionTime: "2019-06-28T02:12:46Z"
      status: "True"
      type: Initialized
    - lastTransitionTime: "2019-06-28T02:12:46Z"
      message: 'failed to get release history: release: "example-openebsinstalltemplate-3c4h3yqweylajqktnjs04e135"
        not found'
      reason: UninstallError
      status: "True"
      type: ReleaseFailed
  

• Image Maping

  helm-operator image	openebs release image
  v0.0.1	1.0.0
  v0.0.2	1.1.0
  v0.0.3	1.3.0
  v0.0.4	1.4.0
  v0.0.5	1.5.0
  v0.0.6	2.10.0
  v0.0.7	2.11.0
  v0.0.8	2.12.2
  v0.0.9	3.0.0

helm-operator's Issues

cr deletion in waiting state after upgrade to v1.25.3

mkdir: can't create directory because of permission denied

Issue

No storage is created when using openebs-hostpath StorageClass.

Actual result

The container init-pvc-<uuid> fails with:

mkdir: can't create directory '/data/pvc-3f7500d7-16ae-11ea-971b-fa163e796691': Permission denied

Expected result

Storage created

Workaround:

1. Permissive selinux in nodes
2. Modify init-pvc-<uuid> and edit: securityContext: {}:

spec:
  securityContext: 
    privileged: true

Environment

Openshift: 4.2
openebsoperator: v1.3.0

Other

I think the issue here is that init-pvc-<uuid> is not running as privileged. Running it as privileged is a better option that a permissive selinux. I just don't know yet which component is to be modified for that to happen.

Identify the workflow for pulling latest OpenEBS helm charts into the helm operator

Requirement

The helm operator is bootstrapped by specifying the OpenEBS stable helm chart, with the 1.0.0 files being unpacked and copied into the helm operator repository, which in turn is bundled into the operator image. We need to identify the right workflow to keep the operator repo synched with the latest stable charts without manual intervention, and at the same time maintain parity with the OperatorHub/OperatorLifeCycleManager CR (clusterServiceVersion) requirements.

• Some of the associated thoughts here are:

  • Maintaining versioned chart from stable with its own CSVs and Operator Images
  • Always pulling latest helm chart in the operator image (Dockerfile) while keeping CSV generic.

I agree, as part of the docker build - if we can pulling in the right version is a good option.

Originally posted by @kmova in #1

followed the steps and failed.

oc create namespace openebs
namespace/openebs created
oc adm policy add-scc-to-user privileged system:serviceaccount:openebs:openebs-maya-operator
clusterrole.rbac.authorization.k8s.io/system:openshift:scc:privileged added: "openebs-maya-operator"
oc adm policy add-scc-to-user privileged system:serviceaccount:openebs:default
clusterrole.rbac.authorization.k8s.io/system:openshift:scc:privileged added: "default"

After that, I installed and created the install template:

   failed installation (storageclasses.storage.k8s.io is forbidden: User
    "system:serviceaccount:openshift-operators:openebs-operator" cannot
    create resource "storageclasses" in API group "storage.k8s.io" at the
    cluster scope) and failed rollback: uninstallation completed with 2
    error(s): storageclasses.storage.k8s.io "openebs-hostpath" is forbidden:
    User "system:serviceaccount:openshift-operators:openebs-operator" cannot
    delete resource "storageclasses" in API group "storage.k8s.io" at the
    cluster scope; storageclasses.storage.k8s.io "openebs-device" is
    forbidden: User
    "system:serviceaccount:openshift-operators:openebs-operator" cannot
    delete resource "storageclasses" in API group "storage.k8s.io" at the
    cluster scope

Migrate CRDs to apiextensions.k8s.io/v1

Hi!

I tried to run openebs on a Kubernetes 1.22 cluster (upcoming release, tried with v1.22.0-rc.0) and it's currently not possible to deploy the CRDs, which leads most of the subsequent things to fail.
Kubernetes 1.22 dropped the support for apiextensions.k8s.io/v1beta1 which is currently used.

I figure that, in order to work on Kubernetes 1.22, the CRD need to migrate to apiextensions.k8s.io/v1

Cheers
Felix

OpenEBS 3.0.0 operator based install on OpenShift 4.8

• Installing through in-cluster OperatorHub on OpenShift 4.8.18 and 4.9.5
• OpenEBS 3.0.0
• Using a freshly installed OpenShift cluster on AWS (4.8) or bare-metal (4.9)
• Logged in as a user with the cluster-admin role when performing all steps
• Using the default settings for Operator install (channel=alpha; all namespaces; installed namespace=openshift-operators;update=automatic)
• Using the default YAML for OpenEBS Install Template
• openebs-maya-operator SA given privileged SCC using instructions in Operator description
  ** Side note, the instructions in the operator description differ slightly from the instructions on openebs.io which also differ from the instructions on GitHub. Differences are mainly in the namespace to install to, and I suspect have evolved over time. It would be good to make them consistent.

The Operator installs fine an creates the ServiceAccount openebs-operator.
However, when creating the OpenEBS Install Template instance, it fails with permissions errors for openebs-operator (NOT openebs-maya-operator) to create or destroy storage classes.

I have also tried giving the openebs-operator SA privileged SCC, but I get the same error. Also, tried installing the operator to a single namespace called openebs as per some of the other instructions pointed out above, but get the same failure.

open-ebs operator pod log

{"level":"info","ts":1636830926.867148,"logger":"cmd","msg":"Version","Go Version":"go1.16.9","GOOS":"linux","GOARCH":"amd64","helm-operator":"v1.14.0","commit":"78f08b4852faf344ad3ef457c54f86087aaa0a0a"}
{"level":"info","ts":1636830926.8682532,"logger":"cmd","msg":"Environment variable OPERATOR_NAME has been deprecated, use --leader-election-id instead."}
{"level":"info","ts":1636830926.868304,"logger":"cmd","msg":"Watching all namespaces.","Namespace":""}
I1113 19:15:27.921489       1 request.go:665] Waited for 1.021728884s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/authentication.k8s.io/v1?timeout=32s
{"level":"info","ts":1636830929.8728647,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":":8080"}
{"level":"info","ts":1636830929.8740401,"logger":"helm.controller","msg":"Watching resource","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","namespace":"","reconcilePeriod":"1m0s"}
{"level":"info","ts":1636830929.8744724,"msg":"starting metrics server","path":"/metrics"}
{"level":"info","ts":1636830929.874601,"logger":"controller.openebsinstalltemplate-controller","msg":"Starting EventSource","source":"kind source: openebs.io/v1alpha1, Kind=OpenEBSInstallTemplate"}
{"level":"info","ts":1636830929.8746696,"logger":"controller.openebsinstalltemplate-controller","msg":"Starting Controller"}
{"level":"info","ts":1636830929.9755356,"logger":"controller.openebsinstalltemplate-controller","msg":"Starting workers","worker count":16}
I1113 19:15:57.404478       1 request.go:665] Waited for 1.045015416s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/operator.openshift.io/v1?timeout=32s
{"level":"error","ts":1636830965.3285844,"logger":"helm.controller","msg":"Release failed","namespace":"openshift-operators","name":"oebs","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","release":"oebs","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830965.3765109,"logger":"controller.openebsinstalltemplate-controller","msg":"Reconciler error","name":"oebs","namespace":"openshift-operators","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
I1113 19:16:07.963847       1 request.go:665] Waited for 1.039821531s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/operators.coreos.com/v2?timeout=32s
{"level":"error","ts":1636830970.9624512,"logger":"helm.controller","msg":"Release failed","namespace":"openshift-operators","name":"oebs","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","release":"oebs","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830971.0066683,"logger":"controller.openebsinstalltemplate-controller","msg":"Reconciler error","name":"oebs","namespace":"openshift-operators","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830976.6246536,"logger":"helm.controller","msg":"Release failed","namespace":"openshift-operators","name":"oebs","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","release":"oebs","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830976.6556742,"logger":"controller.openebsinstalltemplate-controller","msg":"Reconciler error","name":"oebs","namespace":"openshift-operators","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
I1113 19:16:19.249108       1 request.go:665] Waited for 1.043298602s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/imageregistry.operator.openshift.io/v1?timeout=32s
{"level":"error","ts":1636830982.482207,"logger":"helm.controller","msg":"Release failed","namespace":"openshift-operators","name":"oebs","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","release":"oebs","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830982.519001,"logger":"controller.openebsinstalltemplate-controller","msg":"Reconciler error","name":"oebs","namespace":"openshift-operators","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830987.6638396,"logger":"helm.controller","msg":"Release failed","namespace":"openshift-operators","name":"oebs","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","release":"oebs","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830987.705868,"logger":"controller.openebsinstalltemplate-controller","msg":"Reconciler error","name":"oebs","namespace":"openshift-operators","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
I1113 19:16:30.279264       1 request.go:665] Waited for 1.04668003s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/helm.openshift.io/v1beta1?timeout=32s
{"level":"error","ts":1636830993.2904336,"logger":"helm.controller","msg":"Release failed","namespace":"openshift-operators","name":"oebs","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","release":"oebs","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830993.3285933,"logger":"controller.openebsinstalltemplate-controller","msg":"Reconciler error","name":"oebs","namespace":"openshift-operators","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830998.6364563,"logger":"helm.controller","msg":"Release failed","namespace":"openshift-operators","name":"oebs","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","release":"oebs","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830999.053493,"logger":"helm.controller","msg":"Failed to update status after install release failure","namespace":"openshift-operators","name":"oebs","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","release":"oebs","error":"Operation cannot be fulfilled on openebsinstalltemplates.openebs.io \"oebs\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830999.0535834,"logger":"controller.openebsinstalltemplate-controller","msg":"Reconciler error","name":"oebs","namespace":"openshift-operators","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
I1113 19:16:41.580579       1 request.go:665] Waited for 1.045188919s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/flowcontrol.apiserver.k8s.io/v1beta1?timeout=32s

Error upgrading from 2.11.4 to 3.0.0

First of all, thanks for this great project.

I´m using rancher 2.6 to upgrade a helm installation of version 2.11.4 > 3.0.0. The installation have nothing of special interest. It was using mainly the default values, and I only use the localstorage provider.

The error is the following, and I have no idea or hints on the net. Can you give my any pointers:

helm upgrade --history-max=5 --install=true --namespace=openebs --timeout=10m0s --values=/home/shell/helm/values-openebs-3.0.0.yaml --version=3.0.0 --wait=true openebs /home/shell/helm/openebs-3.0.0.tgz
checking 7 resources for changes
error updating the resource "openebs-ndm":
cannot patch "openebs-ndm" with kind DaemonSet: DaemonSet.apps "openebs-ndm" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"openebs", "component":"ndm", "release":"openebs"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
error updating the resource "openebs-localpv-provisioner":
cannot patch "openebs-localpv-provisioner" with kind Deployment: Deployment.apps "openebs-localpv-provisioner" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"openebs", "release":"openebs"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
error updating the resource "openebs-ndm-operator":
cannot patch "openebs-ndm-operator" with kind Deployment: Deployment.apps "openebs-ndm-operator" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"openebs", "release":"openebs"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
Error: UPGRADE FAILED: cannot patch "openebs-ndm" with kind DaemonSet: DaemonSet.apps "openebs-ndm" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"openebs", "component":"ndm", "release":"openebs"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable && cannot patch "openebs-localpv-provisioner" with kind Deployment: Deployment.apps "openebs-localpv-provisioner" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"openebs", "release":"openebs"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable && cannot patch "openebs-ndm-operator" with kind Deployment: Deployment.apps "openebs-ndm-operator" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"openebs", "release":"openebs"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable 

The yaml config is the following

analytics:
  enabled: true
  pingInterval: 24h
apiserver:
  affinity: {}
  enabled: true
  healthCheck:
    initialDelaySeconds: 30
    periodSeconds: 60
  image: openebs/m-apiserver
  imageTag: 2.12.2
  nodeSelector: {}
  ports:
    externalPort: 5656
    internalPort: 5656
  replicas: 1
  resources: {}
  sparse:
    enabled: 'false'
  tolerations: []
cleanup:
  image:
    imagePullSecrets: []
    registry: null
    repository: bitnami/kubectl
    tag: null
crd:
  enableInstall: true
cstor:
  enabled: false
  openebsNDM:
    enabled: false
  pool:
    image: openebs/cstor-pool
    imageTag: 2.12.2
  poolMgmt:
    image: openebs/cstor-pool-mgmt
    imageTag: 2.12.2
  target:
    image: openebs/cstor-istgt
    imageTag: 2.12.2
  volumeMgmt:
    image: openebs/cstor-volume-mgmt
    imageTag: 2.12.2
defaultStorageConfig:
  enabled: 'true'
featureGates:
  APIService:
    address: 0.0.0.0:9115
    enabled: false
    featureGateFlag: APIService
  ChangeDetection:
    enabled: false
    featureGateFlag: ChangeDetection
  GPTBasedUUID:
    enabled: true
    featureGateFlag: GPTBasedUUID
  UseOSDisk:
    enabled: false
    featureGateFlag: UseOSDisk
  enabled: true
helper:
  image: openebs/linux-utils
  imageTag: 3.0.0
image:
  pullPolicy: IfNotPresent
  repository: ''
imagePullSecrets: []
jiva:
  defaultStoragePath: /var/openebs
  enabled: false
  image: openebs/jiva
  imageTag: 2.12.2
  localpv-provisioner:
    openebsNDM:
      enabled: false
  openebsLocalpv:
    enabled: false
  replicas: 3
legacy:
  enabled: false
localprovisioner:
  affinity: {}
  basePath: /var/openebs/local
  enableDeviceClass: false
  enableHostpathClass: false
  enableLeaderElection: true
  enabled: true
  healthCheck:
    initialDelaySeconds: 30
    periodSeconds: 60
  image: openebs/provisioner-localpv
  imageTag: 3.0.0
  nodeSelector: {}
  replicas: 1
  resources: {}
  tolerations: []
  waitForBDBindTimeoutRetryCount: '12'
localpv-provisioner:
  enabled: false
  openebsNDM:
    enabled: false
lvm-localpv:
  enabled: false
ndm:
  enabled: true
  filters:
    enableOsDiskExcludeFilter: true
    enablePathFilter: true
    enableVendorFilter: true
    excludePaths: /dev/loop,/dev/fd0,/dev/sr0,/dev/ram,/dev/dm-,/dev/md,/dev/rbd,/dev/zd
    excludeVendors: CLOUDBYT,OpenEBS
    includePaths: ''
    osDiskExcludePaths: /,/etc/hosts,/boot
  healthCheck:
    initialDelaySeconds: 30
    periodSeconds: 60
  image: openebs/node-disk-manager
  imageTag: 1.7.0
  nodeSelector: {}
  probes:
    enableSeachest: false
  resources: {}
  sparse:
    count: '0'
    path: /var/openebs/sparse
    size: '10737418240'
    enabled: false
  tolerations: []
ndmExporter:
  clusterExporter:
    metricsPort: 9100
    name: ndm-cluster-exporter
    podLabels:
      name: openebs-ndm-cluster-exporter
  enabled: false
  image:
    pullPolicy: IfNotPresent
    registry: null
    repository: openebs/node-disk-exporter
    tag: 1.7.0
  nodeExporter:
    metricsPort: 9101
    name: ndm-node-exporter
    podLabels:
      name: openebs-ndm-node-exporter
ndmOperator:
  enabled: true
  healthCheck:
    initialDelaySeconds: 15
    periodSeconds: 20
  image: openebs/node-disk-operator
  imageTag: 1.7.0
  nodeSelector: {}
  readinessCheck:
    initialDelaySeconds: 5
    periodSeconds: 10
  replicas: 1
  resources: {}
  tolerations: []
  upgradeStrategy: Recreate
nfs-provisioner:
  enabled: false
openebs-ndm:
  enabled: false
policies:
  monitoring:
    enabled: true
    image: openebs/m-exporter
    imageTag: 2.12.2
provisioner:
  affinity: {}
  enableLeaderElection: true
  enabled: true
  healthCheck:
    initialDelaySeconds: 30
    periodSeconds: 60
  image: openebs/openebs-k8s-provisioner
  imageTag: 2.12.2
  nodeSelector: {}
  patchJivaNodeAffinity: enabled
  replicas: 1
  resources: {}
  tolerations: []
rbac:
  create: true
  kyvernoEnabled: false
  pspEnabled: false
release:
  version: 3.0.0
serviceAccount:
  create: true
  name: null
snapshotOperator:
  affinity: {}
  controller:
    image: openebs/snapshot-controller
    imageTag: 2.12.2
    resources: {}
  enableLeaderElection: true
  enabled: true
  healthCheck:
    initialDelaySeconds: 30
    periodSeconds: 60
  nodeSelector: {}
  provisioner:
    image: openebs/snapshot-provisioner
    imageTag: 2.12.2
    resources: {}
  replicas: 1
  tolerations: []
  upgradeStrategy: Recreate
varDirectoryPath:
  baseDir: /var/openebs
webhook:
  affinity: {}
  enabled: true
  failurePolicy: Fail
  healthCheck:
    initialDelaySeconds: 30
    periodSeconds: 60
  hostNetwork: false
  image: openebs/admission-server
  imageTag: 2.12.2
  nodeSelector: {}
  replicas: 1
  resources: {}
  tolerations: []
zfs-localpv:
  enabled: false
global:
  clusterName: c-dww6p
  projectName: p-mq4jz
defaultImage: true
defaultPorts: true

followed the steps and failed.

   failed installation (storageclasses.storage.k8s.io is forbidden: User
    "system:serviceaccount:openshift-operators:openebs-operator" cannot
    create resource "storageclasses" in API group "storage.k8s.io" at the
    cluster scope) and failed rollback: uninstallation completed with 2
    error(s): storageclasses.storage.k8s.io "openebs-hostpath" is forbidden:
    User "system:serviceaccount:openshift-operators:openebs-operator" cannot
    delete resource "storageclasses" in API group "storage.k8s.io" at the
    cluster scope; storageclasses.storage.k8s.io "openebs-device" is
    forbidden: User
    "system:serviceaccount:openshift-operators:openebs-operator" cannot
    delete resource "storageclasses" in API group "storage.k8s.io" at the
    cluster scope

Move the OpenEBS OperatorHub listing to upstream

Currently OpenEBS is listed as a Community Operator under the OperatorHub https://github.com/operator-framework/community-operators/tree/master/community-operators repository.
When OpenShift is installed, first, it shows the upstream operator on its OperatorHub menu and it takes time for community operators to show up.
OpenEBS is a commercially supported CNCF project and it would be a better fit if it is listed under the https://github.com/operator-framework/community-operators/tree/master/upstream-community-operators repository.