[RFE] I need better logs and troubleshooting advice across the board. So many bugs and issues are completely unanswered with the same issue across generations of release. about rancher

[ns-rlewkowicz]

I ended up just having to scrap the cluster. I'm running out of time on an initiative and just have to rebuild, but not for lack of trying.

I wanted to share something fun I found. More or less, an undocumented api of sorts that you could cause a lot of damage with if you're not careful. I need to put this into a blog post.

You list your keys on the local cluster:

etcdctl get "" --prefix --keys-only

Grep around for cluster name, machine whatever etc and you just kinda feel out the keys.

R_CLUSTER=`etcdctl get "/registry/provisioning.cattle.io/clusters/fleet-default/c-gmd2b" --prefix | head -2 | tail -1 | jq .`

NEW_KEY=`echo $R_CLUSTER | jq '.status.conditions[0]="False"'`

etcdctl put "/registry/provisioning.cattle.io/clusters/fleet-default/c-gmd2b" "$(echo $NEW_KEY | jq . -c)"

I only got so far, and would like to play more if I can find the time, but the world is your oyster here. You can modify anything in place regardless of if you should or could otherwise. You can ready or unready nodes, try to trigger lifecycle events etc. Very powerful!