Do not run k3s and host OS upgrades at the same time

.gitlab/ci_pipelines/default.yml

@@ -179,11 +179,10 @@ terminate-droplet:
       fi
   # We have problems with cosmos not allowing deletion even though the droplet
   # was just stopped.
-  retry: 1
+  retry: 2
   environment:
     name: $CI_COMMIT_REF_SLUG
     action: stop
-  retry: 1
 
 install-stackspin:
   stage: install-stackspin

.gitlab/issue_templates/release.md

@@ -11,10 +11,6 @@ In case of a major release, `Y = 0`.
 
 ### Update our custom charts
 
-Pull in upstream changes to:
-
-* [ ] [local-path-provisioner](https://open.greenhost.net/stackspin/local-path-provisioner)
-
 If necessary, do a release of the following charts. Don't forget to change the
 version used in Stackspin (or wait for renovatebot to do that automatically).
 
@@ -122,6 +118,15 @@ to check if the upgrade goes well there:
 * [ ] `watch flux get kustomization` to see components being upgraded. If
       necessary check the status using `kubectl describe hr ...` and debug.
 
+## Quality assurance
+
+Now is a good time to spend some time using the `stackspin.net` instance: do
+some basic manual testing of the dashboard, log in to all the apps, check if
+the Nextcloud apps are all working, check any major changes or new features,
+etc. Typically we keep the release candidate on `stackspin.net` for a week or
+so before pushing out the release, but that really depends on the type of
+changes.
+
 ## Release branch (only minor releases)
 
 * [ ] Merge the just-updated `pre-release/vX` into the release branch `vX`.

flux2/config/system-upgrade/host-os-upgrade-plan.yaml

@@ -11,7 +11,10 @@ spec:
       stackspin.net/provisioned-os: "debian"
   # Target Debian version.
   version: "12"
+  # Run on one node at a time.
   concurrency: 1
+  # Do not run concurrently with other exclusive Plans, notably k3s upgrades.
+  exclusive: true
   serviceAccountName: system-upgrade
   # Cordoning and draining before upgrade are not helpful in our single-node
   # setup.

flux2/config/system-upgrade/k3s-upgrade-plan.yaml

@@ -8,7 +8,10 @@ metadata:
   labels:
     k3s-upgrade: server
 spec:
+  # Run on one node at a time.
   concurrency: 1
+  # Do not run concurrently with other exclusive Plans, notably OS upgrades.
+  exclusive: true
   nodeSelector:
     matchExpressions:
     - key: node.kubernetes.io/instance-type