Grafana fails at startup because flux (re)moved dashboards

When the grafana pod from kube-prometheus-stack start, one of its init containers fails trying to download the flux grafana dashboard from github. Apparently they removed those dashboards:

⚠️ Removal Notice

Starting Flux v2.1.0, released August 24, 2023, the Flux monitoring configurations in this repository were marked as deprecated. The new monitoring docs are available at Flux monitoring docs with new example configurations in fluxcd/flux2-monitoring-example.

The deprecated configurations were removed in Flux v2.2 on December 13, 2023. All users of these configurations are advised to use the new monitoring setup, following the docs and the examples.

After collecting a lot of user feedback about our monitoring recommendation, in order to serve most of the needs of the users, we decided to create a new monitoring setup leveraging more of the kube-prometheus-stack, specifically kube-state-metrics, to enable configuring Flux custom metrics, see the Flux custom Prometheus metrics docs to learn more about it. Please refer to fluxcd/flux2/4128 for a detailed explanation about this change and the new capabilities offered by the new monitoring setup.

We need to check if we can move to the new dashboards, or failing that remove it.

I encountered this problem while setting up a new cluster. I'm not sure if this will also be a problem when the grafana pod starts on an existing cluster; that depends on how the init container is programmed.