Comments (28)
I verified it in my development environment.
I deleted the grafana-operator-controller-manager
and the operator upgraded successfully to 5.6.1 and then to 5.6.3
from grafana-operator.
Thanks for the report @ginokok1996 , sorry about the issue.
This issue was introduced in #1373
We could potentially do as suggested here: operator-framework/operator-lifecycle-manager#952 to solve this issue.
We are working on a fix.
from grafana-operator.
If someone wants a Quick And Dirty™ solution to resolve this, all you need to do is remove deployment grafana-operator-controller-manager
from openshift-operators
namespace. The operator will re-create it automatically and proceed with update to 5.6.1 normally. I've noticed no other side-effects in any grafana instance, then again, the cluster I tested it on is not a critical/production one so your mileage may vary.
from grafana-operator.
That is a very good point @tkolo , the only thing that will happen is when the operator restarts it will reconcile any potential changes that could have been done while it was gone. This happens the same way when the operator is restarted.
So it should be fine to do in a production env.
from grafana-operator.
Not sure if its something on my end yet, will check further.
But it now says that its up to date even though there are new versions.
from grafana-operator.
OLM isn't making this easy haha.
Lets give it some time maybe it still needs to process some stuff.
from grafana-operator.
If you have the operator from community-operators catalog, you can try restarting (deleting) pods from openshift-marketplace/community-operatros
deployment. That usually does the trick for me.
from grafana-operator.
bug title is wrong, it's about upgrade from 5.6.0 -> 5.6.1. Other than that can confirm, I'm having the same issue
from grafana-operator.
Excuses, corrected it
from grafana-operator.
Thanks for the fast reply @NissesSenap , happy to know a fix is in the works.
from grafana-operator.
The fix has been applied and I have created the upstream PR to release it to OLM, hopefully it will be merged tonight.
redhat-openshift-ecosystem/community-operators-prod#4000
from grafana-operator.
Thanks for the fix @NissesSenap, only unfortunate thing is that 5.6.1 still exists on operatorhub.
meaning we cant follow the upgrade paths with automatic update on since we cant update to 5.6.1
We would then need to uninstall the operator and install 5.6.2 directly
from grafana-operator.
Can't you go from 5.6.0 manually over to 5.6.2?
from grafana-operator.
In openshift you can't, its either follow the different patches or uninstall the operator and install a specific version.
from grafana-operator.
I haven't used OCP in years. But back in the days you could change an object called something like operatorgroup https://docs.openshift.com/container-platform/4.10/operators/understanding/olm/olm-understanding-olm.html
You should be able to set the version manually. If you can try that, it would be great, so we at least know it works.
I will try to create a PR upstream to OLM, but it won't be able to merge automatically. So I will have to get in contact with someone at redhat and normally that isn't the quickest.
from grafana-operator.
I have created the upstream change here: redhat-openshift-ecosystem/community-operators-prod#4002
Let's see how it goes
from grafana-operator.
Thanks a bunch @NissesSenap
from grafana-operator.
What feels like my 10th try to fix this: redhat-openshift-ecosystem/community-operators-prod#4017
Hopefully the skip part will solve it.
Please give it a try @ginokok1996 when you got the time.
from grafana-operator.
Well, that is worrying....
Don't know if OLM got some cronjob to push updates and if they do, when does it come. But I think you should have seen the other versions any way.
But I have done changes to the patch flow it was , 5.6.0 -> 5.6.1 -> 5.6.2 -> 5.6.3
And in 5.6.3 it says that you should skip 5.6.1.
So I guess it is 5.6.0 -> 5.6.2 -> 5.6.3 at least that was my intention.
And if this makes OLM break it will drive me nuts since that was the suggested workaround :D
from grafana-operator.
@ginokok1996 Can you please navigate to the search tab, and search for ClusterServiceVersion
in the namespace o which you've installed the grafana-operator, that'll give us some clue as to what's going on
from grafana-operator.
I also reinstalled grafana 5.6.0 and restarted all the OLM pods in the lifecycle namespace.
It still says there are no updates.
from grafana-operator.
We are also facing the same issue in our clusters.
Automatic update from 5.6.0 fails with the same message
install strategy failed: Deployment.apps
"grafana-operator-controller-manager" is invalid: spec.selector: Invalid
value:
v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/managed-by":"olm",
"app.kubernetes.io/name":"grafana-operator"},
MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
from grafana-operator.
Hey folks, for now, can you try to delete the grafana-operator-controller-manager
deployment? This should unblock the update and allow it to proceed to the latest release.
Apologies for the inconvenience.
I've tested this on one of my clusters, and it unblocked the update path. Unfortunately, those that tried to update to 5.6.2 from 5.6.1 will have to do this. 5.6.3 shouldn't have this issue (I hope)
from grafana-operator.
Hey folks, for now, can you try to delete the
grafana-operator-controller-manager
deployment? This should unblock the update and allow it to proceed to the latest release. Apologies for the inconvenience.I've tested this on one of my clusters, and it unblocked the update path. Unfortunately, those that tried to update to 5.6.2 from 5.6.1 will have to do this. 5.6.3 shouldn't have this issue (I hope)
OLM should be able to re-deploy the operator with a new deployment, thus avoiding the "immutability issue". It shouldn't affect your running Grafana deployments
from grafana-operator.
Still facing the issue that when you're on version 5.6.0 openshift thinks its up to date and there are no new versions.
from grafana-operator.
There is no solution for this other than what is written here: #1399 (comment) @ginokok1996 .
OLM have its limitations, and we can't force your cluster from doing what it was doing.
The easiest way forward you have is to just delete the old operator deployment, and it will get solved.
We have created an issue upstream about this, but it's not like a fix will come out soon.
from grafana-operator.
Deleting the deployment will work to upgrade from 5.6.1 a higher version since it will then be allowed to change the labels.
However we can't even go from 5.6.0 to 5.6.1 now.
We would need to uninstall grafana and install the latest version.
Isn't the biggest issue but seems like a different problem than whats stated above.
from grafana-operator.
What @tkolo wrote.
No matter, there is nothing the grafana-opreator maintainers can do to stop having your cluster in bad state. All we can do is apologize for the inconveniences and point to the workarounds.
from grafana-operator.
Related Issues (20)
- Make the grafana operator able to create grafana service accounts and tokens from yaml HOT 3
- Documentation: PVC and Volumes example HOT 6
- Auto-enable app plugins via operator. HOT 4
- Installing custom plugin HOT 5
- Port NotificationChannel CRD from operator v4 to v5 HOT 3
- Prometheus data source for "Azure Monitor for Prometheus" with workload identity does not work HOT 5
- [Bug] instanceSelector.matchExpressions not working for GrafanaDatasource HOT 5
- Dashboard from configmap using selector HOT 4
- Grafana deployment with a Persistent Volume HOT 4
- Service account automountServiceAccountToken should be set to false HOT 3
- Service Account for grafana instance does not get annotations HOT 2
- [Bug] Unable to set custom secrets without defining grafana container image HOT 6
- watch namespaces using a label selector
- [Bug] Unable to upgrade from v5.6.0 on OpenShift HOT 11
- [doc] Grafana deployment with a Persistent Volume HOT 6
- Unable to limit which Grafana can use which dashboards (v5 operator) HOT 4
- Add make it possible to disable editing of a dashboard from a grafanadashboard CRD HOT 9
- ArgoCD healthcheck and GrafanaDashboard HOT 5
- Apply the default Grafana Version to the CR spec HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from grafana-operator.