.gitlab/issue_templates/release.md

 # Release checklist
 
-## Before finalizing release
+We use a simple two-component version `x.y`, where `y` is updated for
+auto-update releases ("minor release"), and `x` is updated for releases that
+require manual intervention ("major release").
 
-Make these changes in the main branch before releasing:
+In the checklist below, we'll use a literal `X.Y` to stand for the new version.
+In case of a major release, `Y = 0`.
 
-* [ ] Ensure all applications/dependencies/charts are at their latest versions
-      see `.gitlab/issue_templates/update_all_components.md`
-* Update/review `docs/*` and make sure it matches the current state
-  * [ ] Update how to checkout the latest stable tag
-        `find docs/ -name '*.md' -o -name '*.rst' -exec grep -H '0.7' {} \;`
-  * [ ] Document how to upgrade in `docs/upgrading.rst`
-* [ ] Update dependencies in `requirements.txt` by following the
-      instructions in `requirements.in`
-* [ ] Do a manual upgrade test from last release version to `main`
-* [ ] Create a release merge request for the main branch
+## Pre-release chores
 
-## In the release merge request
+### Update our custom charts
 
+If necessary, do a release of the following charts. Don't forget to change the
+version used in Stackspin (or wait for renovatebot to do that automatically).
+
+* [ ] [dashboard](https://open.greenhost.net/stackspin/dashboard) (see also dashboard#10)
+* [ ] [nextcloud](https://open.greenhost.net/stackspin/nextcloud) (see also nextcloud#1012)
+* [ ] [wordpress](https://open.greenhost.net/stackspin/wordpress-helm)
+* [ ] [hedgedoc](https://open.greenhost.net/stackspin/hedgedoc-helm-chart)
+* [ ] [local-path-provisioner](https://open.greenhost.net/stackspin/local-path-provisioner)
+
+## Release candidate
+
+* [ ] Create a new branch, `release-candidate/vX.Y`, from main:
+
+  ```sh
+  git checkout main
+  git pull
+  git checkout -b release-candidate/vX.Y
+  ```
+
+### Only for major releases
+
+* [ ] Update how to checkout the latest stable tag, in
+  `docs/installation/install_cli.rst`.
+* [ ] Document how to upgrade in `docs/system_administration/upgrading.rst`
+
+### For all releases
+
+* [ ] Update the version number in the `VERSION` file
 * Update [CHANGELOG.md](https://keepachangelog.com)
   * [ ] Include all merged MR since last release, i.e. using [lab](https://github.com/zaquestion/lab#installation):
-        ```
-        lab mr list -s merged -a | awk '{first = $1; $1=""; print "*" $0, "(" first ")"}'
-        ```
-  * [ ] Include app charts and versions table (Use [tablemark-cli](https://github.com/haltcase/tablemark-cli)):
-        ```
-        helm ls -A -o json | jq 'map({name, chart, app_version})' | jq 'map(.chart |= split("-")[-1])' > /tmp/versions.json
-        tablemark /tmp/versions.json
-        ```
+
+    ```sh
+    lab mr list -s merged -a | \
+      awk '{first = $1; $1=""; print "*" $0, "(" first ")"}'
+    ```
+
+  * [ ] Include app charts and versions table. You can use
+    [tablemark-cli](https://github.com/haltcase/tablemark-cli), or
+    [an online tool](https://tableconvert.com/json-to-markdown):
+
+    ```sh
+    helm ls -A -o json | jq 'map({name, chart, app_version})' | \
+      jq 'map(.chart |= split("-")[-1])' > /tmp/versions.json
+    ```
+
+    Now look for image tag overrides and fix those app versions in the
+    table:
+
+    ```sh
+    find ./flux2 -name '*.yaml' | xargs grep -A4 '^ *image:'
+    ```
+
+    Then produce the final table:
+
+    ```
+    tablemark /tmp/versions.json
+    ```
+
   * [ ] Include `Known issues`
-* [ ] Update the version number in the `VERSION` file
-* [ ] Commit (signed)
-* [ ] Push to MR
-* [ ] Make sure main pipeline is successful before merging MR !
-* [ ] Wait for MR to get merged into main
+* [ ] Update app versions in `flux2/core/base/dashboard/dashboard-apps-configmap.yaml`.
+* [ ] Only for major versions: prepare a release blog post. You can
+      use the `CHANGELOG.md` as basis.  (It will be published at a later step.)
+* [ ] Decide where to link to from the dashboard "release notes" link. Could be
+      the release blog post, other release notes page, or as fallback the
+      CHANGELOG.md entry for this version.
+* [ ] Update `flux2/cluster/base/stackspin-static-info.yaml` to reflect the new
+      version and the release notes URL.
+* [ ] Commit, push to `release-candidate/vX.Y`.
+* [ ] Only for minor versions: create MR to merge `release-candidate/vX.Y` into
+      `pre-release/vX`
+
+### Manual pre-release upgrade testing (only major releases)
+
+* [ ] Try the proposed upgrade procedure on a fresh cluster installed with the
+      previous major version. Make sure to document any required actions and
+      potential problems in the upgrade docs.
+
+### Automatic pre-release upgrade testing (only minor releases)
+
+We have a special upgrade pipeline to test upgrading from the previous release
+to this new (candidate) release. This pipeline runs when the target branch of a
+MR matches `pre-release/*`. In principle you could make a MR to merge `main`
+into `pre-release/vX`, but that has the unfortunate side-effect that the
+pipeline will restart whenever the source branch changes, which happens all the
+time because of renovatebot automerging minor updates and colleagues that work
+too hard. Therefore:
+
+* [ ] Make sure that the resulting upgrade-test pipeline is successful
+* [ ] Before merging, notify #general on `stackspin.net` that we're about to
+  release, and that `stackspin.net` itself can experience short downtime while
+  the upgrade is in progress.
+* [ ] Wait until MR gets reviewed and merged
+* [ ] The CI machine created by the upgrade-test pipeline doesn't get destroyed
+  automatically, so please remove it yourself.
+
+### Check automatic upgrade on `stackspin.net` (only minor releases)
+
+Now that the new code has been merged to `pre-release/vX`, it will be picked
+up by the `stackspin.net` cluster which is set to follow that branch. Even
+though the upgrade pipeline already tested the upgrade process, it's still good
+to check if the upgrade goes well there:
+
+* [ ] do a `flux reconcile source git stackspin` so you don't have to wait
+      until flux decides it's time to reconcile;
+* [ ] `watch flux get kustomization` to see components being upgraded. If
+      necessary check the status using `kubectl describe hr ...` and debug.
+
+## Quality assurance
+
+Now is a good time to spend some time using the `stackspin.net` instance: do
+some basic manual testing of the dashboard, log in to all the apps, check if
+the Nextcloud apps are all working, check any major changes or new features,
+etc. Typically we keep the release candidate on `stackspin.net` for a week or
+so before pushing out the release, but that really depends on the type of
+changes.
+
+## Release branch (only minor releases)
+
+* [ ] Merge the just-updated `pre-release/vX` into the release branch `vX`.
+
+## Release branches (only major releases)
+
+* [ ] Create a new branch `vX` from `v(X-1)`.
+* [ ] Change all occurrences of the last release to the new release version:
+      `sed -i 's/v(X-1)/vX/g' .gitlab/ci_pipelines/upgrade_test.yml \
+        .gitlab/issue_templates/update_all_components.md \
+        .gitlab/issue_templates/deploy_stackspin.md \
+        .gitlab/issue_templates/release.md \
+        docs/conf.py`
+* [ ] Create a merge request for merging the release candidate
+      into the release branch `vX`. Ask for review and merge.
+* [ ] Create a new branch `pre-release/vX` from `vX`.
+
+### Upgrade `stackspin.net` (only major releases)
+
+Note that `stackspin.net` doesn't follow a release branch (`vX`) but a
+pre-release branch.
+
+* [ ] Apply the upgrade procedure to `stackspin.net`: following
+      https://docs.stackspin.net/en/latest/system_administration/upgrading.html
+      and any actions specific to this release.
+* [ ] Configure the cluster to follow the new pre-release branch: `pre-release/vX`.
+
+### Upgrade `staging.stackspin.net` (only major releases)
+
+Note that `staging.stackspin.net` doesn't follow a release branch (`vX`) but the
+`main` branch.
+
+* [ ] Apply the upgrade procedure to `staging.stackspin.net`, but do not change
+      the branch followed by flux (keep it at `main`).
 
 ## Push a signed tag
 
-* [ ] Make sure the main pipeline succeeds for the last commit before tagging. This
-      is important because tags should not get retagged !
-* [ ] Create and push signed tag (`git tag -s 0.7.0 -m 'Release 0.7.0'; git push --tags`)
+* [ ] Create and push signed tag
+      (`git tag -s X.Y -m 'Release X.Y'; git push --tags`)
+* If this is a major release:
+  * [ ] Log into <https://readthedocs.org> and update documentation for tagged versions.
+        If the new branch is not shown, rebuild the docs for any existing branch,
+        which will as a side-effect fetch all branches from our gitlab.
+  * [ ] Publish the release blog post.
+  * [ ] Create an issue for upgrading managed clusters:
+    * our own production clusters;
+    * our managed customer clusters.
+  * [ ] Create a new CI pipeline image:
+        https://docs.stackspin.net/en/latest/for_developers/contributing.html#ci-pipeline-image
+
+## Post-release chores
+
+* [ ] Announce the release in the public Stackspin matrix room.
+* [ ] Notify Greenhost sysops that an upgrade to their cluster will happen
+      overnight.
+* [ ] Update the "stable" CI docker image (used as base for the upgrade pipeline):
+    - `tag="open.greenhost.net:4567/stackspin/stackspin/stackspin-ci:vX"`
+    - `docker build -t $tag .`
+    - `docker push $tag`
+* [ ] Merge the release branch back into `main`. This is necessary to propagate
+      the changes to CHANGELOG etc.
+* [ ] After `demo.stackspin.net` is upgraded, create a new backup for the new
+      version that will be used for the nightly reset. See `cli.stackspin.net`, directory
+      `/srv/stackspin/clusters/demo.stackspin.net/custom-scripts`.
+* [ ] Update the version of the stackspin repo on `cli.stackspin.net`:
+  - `git pull` in `/srv/stackspin` to get the latest version of the release branch.
+  - Update python requirements: `pip install -r requirements.txt`. Note
+    that we use a virtualenv on `cli.stackspin.net` that's in the PATH of the
+    root user.
+* [ ] Close released milestone and set start date for the new milestone.
+
+### Update flux
+
+See available [releases](https://github.com/fluxcd/flux2/releases).
+
+* [ ] Update flux CLI version:
 
-## In the new release branch
+      current_flux_version=$(yq eval .flux.version ansible/group_vars/all/stackspin.yml)
+      new_flux_version=$(curl -s https://api.github.com/repos/fluxcd/flux2/releases/latest | jq -r .name | tr -d 'v')
+      sed -i "s/$current_flux_version/$new_flux_version/g" Dockerfile install/flux-version-check.sh docs/installation/install_cli.rst ansible/group_vars/all/stackspin.yml
 
-* [ ] If this is a major release, create a new release branch (i.e. `v0.7`)
+  * [ ] After changing the required flux version in `flux-version-check.sh`,
+    you also need to rebuild the CI image to contain the new binary, otherwise
+    the check will fail in the CI.
 
-Create a MR for the new branch with the following changes:
+* [ ] Update flux version (daemon components running on clusters): see
+  `dev-tools/generate-flux-manifests/README.md`.
 
-* Ensure all dependencies are updated and locked (locking should be done
-    only on the release branch):
-  * [ ] flux chart versions: `find flux2/* -type f -exec yq eval .spec.chart {} \;`
-* [ ] Wait for MR to get merged
+* [ ] After the new version of the CLI in particular has been merged to `main`,
+      manually run a pipeline for `main` (with `CI_PIPELINE_SOURCE: schedule`) so
+      the `main` CI image gets rebuilt with the new flux. Although we don't
+      regularly run pipelines for `main` anymore, this image is used by pipelines for
+      MRs that don't change the CI image.
 
-## After release
+## Celebration
 
-* [ ] Log into <https://readthedocs.org> and update documentation for tagged versions
-* [ ] Cherry-pick all changes from the release branch into main which
-      shouldn't be only in the 0.4.0 branch (i.e. not the commit locking the
-      dependencies)
-* [ ] Create an issue for creating a release blog post
-* [ ] Create an issue fpr cleaning up
-  * old branches/tags
 * [ ] Celebrate :clinking_glass: !!

.gitlab/issue_templates/update_all_components.md

-## Update all applications in this repository
-
-* [ ] Update Flux version
-  
-      current_flux_version=0.20.1
-      new_flux_version=0.24.0
-      sed -i "s/$current_flux_version/$new_flux_version/g" Dockerfile install/flux-version-check.sh docs/installation/install_stackspin.rst ansible/group_vars/all/stackspin.yml
-
-* [ ] Update all apk app versions inside `Dockerfile`
-* [ ] Update mitogen version in `ansible/plugins` (https://github.com/mitogen-hq/mitogen/releases)
-* [ ] Update k3s in `ansible/group_vars/all/stackspin.yml`:
-
-## Update our custom charts:
-
-We get dependeny updates via renovate for our custom helm charts.  However, we
-still don't get [automated version
-bumps](https://open.greenhost.net/stackspin/stackspin/-/issues/1001) with it, so
-make sure to bump the version if needed for:
-
-* [ ] [dashboard](https://open.greenhost.net/stackspin/dashboard)
-* [ ] [nextcloud](https://open.greenhost.net/stackspin/nextcloud)
-* [ ] [single-sign-on](https://open.greenhost.net/stackspin/single-sign-on)
-* [ ] [wordpress](https://open.greenhost.net/stackspin/wordpress-helm)
-
-Pull in upstream changes to:
-
-* [ ] [local-path-provisioner](https://open.greenhost.net/stackspin/local-path-provisioner)

.pre-commit-config.yaml

 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.0.1
+    rev: v5.0.0
     hooks:
       - id: check-added-large-files
       - id: check-ast
@@ -8,12 +8,17 @@ repos:
       - id: check-yaml
         args:
           - --allow-multiple-documents
+        exclude: |
+          (?x)^(
+              .gitlab-ci.yml|
+              .gitlab.*
+          )$
       - id: detect-private-key
       - id: end-of-file-fixer
       - id: trailing-whitespace
       - id: debug-statements
   - repo: https://github.com/jumanjihouse/pre-commit-hooks
-    rev: 2.1.5
+    rev: 3.0.0
     hooks:
       - id: shellcheck
       - id: shfmt
@@ -22,15 +27,24 @@ repos:
           - "2"
           - "-ci"
   - repo: https://github.com/hadolint/hadolint
-    rev: v1.23.0
+    rev: v2.12.0
     hooks:
       - id: hadolint-docker
   - repo: https://github.com/timothycrosley/isort
     # Sorts import statements for you
-    rev: 5.0.9
+    rev: 6.0.1
     hooks:
       - id: isort
   - repo: https://github.com/zricethezav/gitleaks
-    rev: v8.2.0
+    rev: v8.24.2
     hooks:
-      - id: gitleaks
+      - id: gitleaks-docker
+  - repo: https://github.com/python-jsonschema/check-jsonschema
+    rev: 0.31.3
+    hooks:
+      - id: check-gitlab-ci
+        files: |
+          (?x)^(
+              .gitlab-ci.yml|
+              .gitlab.*
+          )$

.readthedocs.yaml

+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+version: 2
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.11"
+# Build documentation in the docs/ directory with Sphinx
+sphinx:
+  configuration: docs/conf.py
+# https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html
+python:
+  install:
+  - requirements: docs/requirements.txt

.renovaterc.json

-{
-  "regexManagers": [
-    {
-      "fileMatch": ["flux2/.+release\\.yaml$"],
-      "matchStrings": [
-        "registryUrl=(?<registryUrl>.*?)\n *chart: (?<depName>.*?)\n *version: (?<currentValue>.*)\n"
-      ],
-      "datasourceTemplate": "helm"
-    }
-  ],
-  "pip-compile": {
-    "fileMatch": ["(^|/)requirements\\.in$"]
-  },
-  "pip_requirements": {
-    "enabled": false
-  }
-}

CONTRIBUTING.md

 # How to contribute
 
-## Code of conduct
-
-We follow the [Contributor Covenant](https://www.contributor-covenant.org) code
-of conduct, please see the [CODE_OF_CONDUCT.md](./CODE_OF_CONDUCT.md) document
-for details.
-
-## Preparing the development environment
-
-Make sure you have development dependencies installed in your development
-environment.
-
-```
-pip install -r requirements-dev.txt
-```
-
-## pre-commit hooks
-
-We use [pre-commit](https://pre-commit.com/) to maintain and install pre-commit
-hooks that should be executed before each commit.
-
-
-Please install these required tools on your system:
-
-* [hadolint](https://github.com/hadolint/hadolint) for linting the `Dockerfile`
-* [shellcheck](https://www.shellcheck.net/) and
-* [shfmt](https://github.com/mvdan/sh) to lint and validate shell scripts
-
-Then install pre-commit hooks:
-
-```
-pre-commit install
-```
-
-Running `git commmit` for the first time after installing the hook usually takes a
-little longer because `pre-commit` pulls it's hooks from upstream repositories.
-You can find all hooks in `.pre-commit-config.yaml`.
-
-In case you need to skip the execution of the pre-commit hooks (please don't!),
-use `git commit --no-verify`.
-
-
-## Adding dependencies
-
-Make sure you update our `requirements.txt` file before you push your changes.
-Whenever you need a new python package, add it to requirements.in and run
-
-`pip-compile`
-
-to generate an new `requirements.txt` which does not only pin the new package
-but also its dependencies.
-
-If the new package you are adding is only used by developers,
-please add it to the `requirements-dev.txt` file.
-
-## CI pipeline image
-
-We use a custom disk image for the VPSs used by the CI pipeline. On this image,
-the `install-kubernetes.yaml` playbook has already been applied, which usually
-saves a few minutes of pipeline running time.
-
-### What to do when I change a part of the `install-kubernetes.yaml` playbook?
-
-Don't worry, the playbook *runs* in the CI (just faster, because usually
-nothing needs to change). So if you make changes, you can test those in the CI
-without problems.
-
-If you want to start with a clean slate, however, you might want to change
-`.gitlab/ci_scripts/create_vps.sh` and temporarily remove the `--disk-image-id`
-argument.
-
-#### Before you merge, make sure your changes are applied to a new custom image:
-
-If you changed the `install-kubernetes.yaml` playbook, for example to upgrade
-the k3s version in use, you'll want to generate a new disk image template and
-use it. This is a manual process for now. Follow these steps:
-
-1. Create a new VPS
-   ```
-   export HOST_NAME=baseimage$(date +'%Y%m%d')
-   # Make sure you use your private ssh key id, 411 is the ssh key used in CI
-   export SSH_ID=411
-   python -m stackspin ${HOST_NAME} create --create-droplet --create-hostname ${HOST_NAME}.stackspin.net --ssh-key-id $SSH_ID --create-domain-records --subdomain ${HOST_NAME} stackspin.net
-   ```
-
-2. Accept ssh host key
-   ```
-   ssh root@${HOST_NAME}.stackspin.net
-   ```
-
-3. Run the following to install *only kubernetes* on the VPS:
-   ```
-   python3 -m stackspin ${HOST_NAME} install
-   ```
-4. Log into your machine and clean up the k3s server, then delete the cluster
-   data:
-   ```
-   ssh root@${HOST_NAME}.stackspin.net
-     # Clean up running containers and firewall
-     /usr/local/bin/k3s-killall.sh
-     # Remove k3s state
-     rm -rf /var/lib/rancher/k3s
-   ```
-5. Log into Cosmos with the Stackspin account
-6. Go to VPS Cloud -> VPS and shut down your VPS
-7. Go to VPS Cloud -> Disk Images and click `Manage` for your VPSs disk image
-   1. Change the Disk Label to something like `k3s-template-DATE`
-   2. Set VPS to `-- not assigned --`
-   3. Click save
-   4. `Manage` the same image again
-   5. Click 'make template'
-   6. Choose "Linux Latest LTS" as the kernel
-   7. Choose OS type Debian 11 (bullseye)
-   8. Remember the disk image ID that you can see in the current URL as `id=...`
-   9. Click save
-8. Change the `--disk-image-id` argument in `.gitlab/ci_scripts/create_vps.sh`
-   to your current disk-image-id **with a minus in front of it**. This is
-   because custom images are negative integers, whereas Greenhost's disk images
-   are positive integers. Don't forget to change the comment above to include
-   the current commit sha and date.
-9. Remove the droplet
-
-You are now ready to merge the changes you made to the `install-kubernetes`
-playbook
+Please refer to the [contributing.rst file](./docs/for_developers/contributing.rst)
+or to the [Contributing Guide in Stackspin docs](https://docs.stackspin.net/en/latest/for_developers/contributing.html)

Dockerfile

 # https://hub.docker.com/_/alpine?tab=tags
-FROM alpine:3.15
+FROM alpine:3.21
 
 LABEL name="Stackspin management"
-LABEL version="4.5"
+LABEL version="4.6"
 LABEL vendor1="Greenhost"
 
-ENV TAIKO_SKIP_CHROMIUM_DOWNLOAD=true
-ENV TAIKO_BROWSER_PATH=/usr/bin/chromium-browser
-ENV TAIKO_BROWSER_ARGS=--no-sandbox,--start-maximized,--disable-dev-shm-usage,--ignore-certificate-errors
+# https://docs.renovatebot.com/modules/datasource/#repology-datasource
 
-ADD https://github.com/fluxcd/flux2/releases/download/v0.25.3/flux_0.25.3_linux_amd64.tar.gz /tmp/
+# renovate: datasource=repology depName=alpine_3_20/bash
+ENV BASH_VERSION="5.2.37-r0"
+# renovate: datasource=repology depName=alpine_3_20/cargo
+ENV CARGO_VERSION="1.83.0-r0"
+# renovate: datasource=repology depName=alpine_3_20/chromium versioning=loose
+ENV CHROMIUM_VERSION="134.0.6998.117-r0"
+# renovate: datasource=repology depName=alpine_3_20/coreutils version=loose
+ENV COREUTILS_VERSION="9.5-r2"
+# renovate: datasource=repology depName=alpine_3_20/curl
+ENV CURL_VERSION="8.12.1-r1"
+# renovate: datasource=repology depName=alpine_3_20/expect
+ENV EXPECT_VERSION="5.45.4-r5"
+# renovate: datasource=repology depName=alpine_3_20/git
+ENV GIT_VERSION="2.47.2-r0"
+# renovate: datasource=repology depName=alpine_3_20/jq versioning=loose
+ENV JQ_VERSION="1.7.1-r0"
+# renovate: datasource=repology depName=alpine_edge/kubectl
+ENV KUBECTL_VERSION="1.32.2-r1"
+# renovate: datasource=repology depName=alpine_3_20/libffi-dev
+ENV LIBFFI_DEV_VERSION="3.4.7-r0"
+# renovate: datasource=repology depName=alpine_3_20/libsodium-dev
+ENV LIBSODIUM_DEV_VERSION="1.0.20-r0"
+# renovate: datasource=repology depName=alpine_3_20/make versioning=loose
+ENV MAKE_VERSION="4.4.1-r2"
+# renovate: datasource=repology depName=alpine_3_20/moreutils versioning=loose
+ENV MOREUTILS_VERSION="0.69-r2"
+# renovate: datasource=repology depName=alpine_3_20/npm
+ENV NPM_VERSION="10.9.1-r0"
+# renovate: datasource=repology depName=alpine_3_20/openssh-client-default versioning=loose
+ENV OPENSSH_CLIENT_DEFAULT_VERSION="9.9_p2-r0"
+# renovate: datasource=repology depName=alpine_3_20/py3-pip
+ENV PY3_PIP_VERSION="24.3.1-r0"
+# renovate: datasource=repology depName=alpine_3_20/python3-dev
+ENV PYTHON3_DEV_VERSION="3.12.9-r0"
+# renovate: datasource=repology depName=alpine_3_20/rsync
+ENV RSYNC_VERSION="3.4.0-r0"
+# renovate: datasource=repology depName=alpine_3_20/yq-go
+ENV YQ_VERSION="4.44.5-r3"
+
+# Makes pynacl use system SODIUM
+ENV SODIUM_INSTALL=system
+
+ADD https://github.com/fluxcd/flux2/releases/download/v2.4.0/flux_2.4.0_linux_amd64.tar.gz /tmp/
 COPY ./requirements.txt /requirements.txt
+# Ignore the hadolint error for the pip install line with a `grep` in it
+# hadolint ignore=DL3013
 RUN \
-  # Install kubectl from alpine edge until alpine 3.16 is released
-  apk --no-cache add kubectl=~1.23.4-r0 \
-    --repository=http://dl-cdn.alpinelinux.org/alpine/edge/testing && \
+  # Install kubectl from alpine edge until it is included in an alpine release.
+  apk --no-cache add kubectl=~$KUBECTL_VERSION \
+    --repository=http://dl-cdn.alpinelinux.org/alpine/edge/community && \
   apk --no-cache add \
-    bash=~5.1.16-r0 \
-    cargo=~1.56.1-r0 \
-    chromium=~93.0.4577.82-r3 \
-    curl=~7.80.0-r0 \
-    # needed for installing pycurl python module
-    curl-dev=~7.80.0-r0 \
+    bash=$BASH_VERSION \
+    cargo=$CARGO_VERSION \
+    # Needed for numfmt, used in create_vps.sh
+    coreutils=$COREUTILS_VERSION \
+    chromium=$CHROMIUM_VERSION \
+    curl=$CURL_VERSION \
+    # Needed for installing pycurl python module
+    curl-dev=$CURL_VERSION \
     # Needed for "unbuffer" to timestamp cmds
-    expect=~5.45.4-r2 \
-    git=~2.34.1-r0 \
-    libffi-dev=~3.4.2-r1 \
-    make=~4.3-r0 \
+    expect=$EXPECT_VERSION \
+    git=$GIT_VERSION \
+    # Needed in create-vps job
+    jq=$JQ_VERSION \
+    libffi-dev=$LIBFFI_DEV_VERSION \
+    # Needed for pynacl
+    libsodium-dev=$LIBSODIUM_DEV_VERSION \
+    make=$MAKE_VERSION \
     # Needed for timestamp cmd "ts"
-    moreutils=~0.66-r1 \
-    npm=~8.1.3-r0 \
-    openssh-client-default=~8.8_p1-r1 \
-    py3-pip=~20.3.4-r1 \
-    python3-dev=~3.9.7-r4 \
-    rsync=~3.2.3-r5 \
-    yq=~4.14.1-r0 && \
+    moreutils=$MOREUTILS_VERSION \
+    npm=$NPM_VERSION \
+    openssh-client-default=$OPENSSH_CLIENT_DEFAULT_VERSION \
+    py3-pip=$PY3_PIP_VERSION \
+    python3-dev=$PYTHON3_DEV_VERSION \
+    rsync=$RSYNC_VERSION \
+    yq-go=$YQ_VERSION && \
   rm -rf /var/cache/* && \
   mkdir /var/cache/apk && \
-  pip install --no-cache-dir --ignore-installed six -r /requirements.txt && \
-  ln -s /usr/bin/python3 /usr/bin/python && \
-  tar -xzf /tmp/flux*.tar.gz && mv ./flux /usr/local/bin && \
-  npm install -g taiko@1.2.6
+  # We need to install wheel before running the pip install,
+  # so pip can use it while installing the other packages
+  pip install --no-cache-dir --break-system-packages "$(grep '^.*wheel==' requirements.txt)" && \
+  pip install --no-cache-dir --break-system-packages --ignore-installed -r /requirements.txt && \
+  tar -xzf /tmp/flux*.tar.gz && mv ./flux /usr/local/bin

Makefile

-.DEFAULT_GOAL := help
-
-help:
-	# See Makefile for available targets.
-
-freeze:
-	# pip wrongly adds `pkg-resources==0.0.0` which will break some systems, see
-	# https://open.greenhost.net/stackspin/stackspin/issues/498#note_15057
-	pip3 freeze | grep -v "pkg-resources" > requirements-stable.txt

VERSION

-0.8.0
+2.16

ansible/group_vars/all/stackspin.yml

@@ -3,22 +3,32 @@
 data_directory: "/var/lib/Stackspin"
 ip_address: "{{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}"
 
-# Use python3 on cluster nodes for ansible
-ansible_python_interpreter: "/usr/bin/env python3"
-
 # Application versions
 flux:
+  # We'd like to update flux using renovate as well, but that's hard right now
+  # because we need to perform a couple of steps when updating flux.
   # https://github.com/fluxcd/flux2/releases
-  version: 0.25.3
+  version: "2.4.0"
 
 k3s:
   # https://github.com/k3s-io/k3s/releases
-  version: 'v1.23.3+k3s1'
+  # renovate: datasource=github-releases depName=k3s-io/k3s
+  version: "v1.32.1+k3s1"
   # args to start the k3s server with
   # https://rancher.com/docs/k3s/latest/en/installation/install-options/server-config/
   # kubelet arguments can be passed with `--kubelet-arg`
   # https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/
-  server_args: "--disable traefik --disable local-storage --disable servicelb --kube-apiserver-arg=event-ttl=48h0m0s --tls-san {{ ip_address }}"
+  #
+  # egress-selector-mode=disabled is needed to prevent random webhook failures, see
+  # https://open.greenhost.net/stackspin/stackspin/-/issues/1407
+  # https://github.com/k3s-io/k3s/issues/5835
+  server_args: >
+    --disable traefik
+    --disable local-storage
+    --disable servicelb
+    --kube-apiserver-arg=event-ttl=48h0m0s
+    --tls-san {{ ip_address }}
+    --egress-selector-mode=disabled
 
 docker_mirror:
   enabled: false

ansible/requirements.txt

-# ansible>=2.7 is needed for using the `k8s` resource
-ansible>=2.7

ansible/roles/configure/files/stackspin-version-info.sh

@@ -3,10 +3,8 @@
 cat /var/lib/Stackspin/version.txt
 
 kubectl version
-kubectl plugin list
 
 k3s --version
 
 helm version
-helm plugin list
-helm ls
+helm ls -A

ansible/roles/pre-configure/templates/nftables.conf

 #!/usr/sbin/nft -f
 
-flush ruleset
+# only flush inet ruleset, specified in this file. This keeps the rulesets 
+# added by kube in place, even on a reload
+flush ruleset inet
+
+# These rules are loaded before the KUBE rules, labeled by the _pre suffix in the chains. 
+# This is achieved by using a higher priority (one lower then the default for each hook)
+# The input_pre chain blocks WAN interfaces only. The control of input of local/
+# virtual interfaces is given to kubernetes.
 
 table inet nat {
-  chain PREROUTING {
+  chain prerouting_pre {
     # set priority to dstnat - 1
     type nat hook prerouting priority -101; policy accept;
     # translate any ! 10.0.0.0/8 source address headed for nodeports to a dummy ip
@@ -12,9 +19,10 @@ table inet nat {
 }
 
 
+# This table we use to load rules for WAN/public interfaces
 table inet filter {
-  chain INPUT {
-    type filter hook input priority 0; policy drop;
+  chain input_pre {
+    type filter hook input priority -1; policy accept;
 
     # accept any localhost traffic
     iifname "lo" counter accept
@@ -22,6 +30,9 @@ table inet filter {
     # accept traffic originated from us
     ct state related,established counter accept
 
+    # include custom rules (if available)
+    include "/etc/nftables.d/*.nft"
+
     # Ports open from the outside
     # 22:    ssh
     # 80:    http
@@ -34,22 +45,33 @@ table inet filter {
     # Ports only open from within the cluster
     ip saddr 10.0.0.0/8 tcp dport 9100 counter accept
     ip saddr 10.0.0.0/8 tcp dport 10250 counter accept
+    # metallb-speaker metrics
+    ip saddr 10.0.0.0/8 tcp dport 7472 counter accept
+
+    # make IPv6 work
+    icmpv6 type { nd-neighbor-solicit, echo-request, nd-router-advert, nd-neighbor-advert } accept
 
     # respond to ping
     icmp type echo-request counter accept
 
+    # drop all other traffic from public/wan interfaces
+    iifname "eth*" drop
+    iifname "en*" drop
+    iifname "wl*" drop
+    iifname "ww*" drop
+
     # Uncomment this line to log dropped packages
-    # log
+    #log
   }
 
-  chain FORWARD {
+  chain forward_pre {
     # set priority to -1 to make sure we are processed before ip filter FORWARD
     type filter hook forward priority -1; policy accept;
     # drop all traffic destined for the dummy ip we set earlier in our inet nat PREROUTING
     ip daddr 172.16.16.16 counter drop
   }
 
-  chain OUTPUT {
-    type filter hook output priority 0; policy accept;
+  chain output_pre {
+    type filter hook output priority -1; policy accept;
   }
 }

bin/upgrade-scripts/to-1.0/upgrade.sh

+#!/bin/bash
+
+kubectl patch -n flux-system gitrepository stackspin -p '{"spec":{"ref":{"branch":"v1.0"}}}' --type="merge"

bin/upgrade-scripts/to-2.0/fix-app.sh

+#!/usr/bin/env bash
+
+set -euo pipefail
+
+if [ "$#" -ne 2 ]; then
+  echo "Usage: ./bin/upgrade/to-2.0/fix-app.sh <cluster_name> <app>"
+  exit 1
+else
+  cluster_name=$1
+  app="$2"
+fi
+
+ls "./clusters/$cluster_name/kube_config_cluster.yml" &>/dev/null || (echo "Kubeconfig for $cluster_name not found. Please make sure you are using the provisioning machine that installed Stackspin" && exit 2)
+
+export KUBECONFIG=./clusters/$cluster_name/kube_config_cluster.yml
+
+echo "Re-installing app '$app' without data loss"
+
+flux suspend hr -n stackspin-apps "$app"
+
+if [ "$app" == "nextcloud" ]; then
+  echo "Nextcloud installed, adding redis secret."
+  # shellcheck source=/dev/null
+  [ "${VIRTUAL_ENV:-}" != "" ] && . "$VIRTUAL_ENV/bin/activate"
+  python="$(which python || which python3 || echo "Python not found!" >&2)"
+  $python "./install/generate_secrets.py" nextcloud
+  helm delete -n stackspin-apps nc --wait
+  # Trigger flux kustomization reconciliation to make sure the new secrets are
+  # propagated to the HelmRelease object.
+  # The reconciliation is expected to fail, because the helmrelease is still
+  # suspended.
+  echo "Reconciling kustomization. Ignore it if this raises an error."
+  flux reconcile kustomization "$app" --timeout 2m || true
+else
+  helm delete -n stackspin-apps "$app" --wait
+fi
+
+flux resume hr -n stackspin-apps "$app" --timeout 20m