Upgrade to kube-prometheus-stack 40.0.0 failing on staging
After upgrading to kube-prometheus-stack 40.0.0 the existing kube-prometheus-stack fails to upgrade with:
Warning error 1s (x5 over 110m) helm-controller reconciliation failed: Helm upgrade failed: cannot patch "kube-prometheus-stack-prometheus-node-exporter" with kind DaemonSet: DaemonSet.apps "kube-prometheus-stack-prometheus-node-exporter" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"kube-prometheus-stack", "app.kubernetes.io/name":"prometheus-node-exporter"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
This is due to the major release which needs manual deletion of the kube-prometheus-stack-prometheus-node-exporter daemonset.
Edited by Varac