Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • varac/stackspin
  • xeruf/stackspin
  • stackspin/stackspin
3 results
Show changes
Hide whitespace changes
Inline Side-by-side
Showing
with 2057 additions and 917 deletions
.gitlab/issue_templates/release.md
View file @ 601a9684
# Release checklist # Release checklist
## Before finalizing release We use a simple two-component version `x.y`, where `y` is updated for
auto-update releases ("minor release"), and `x` is updated for releases that
require manual intervention ("major release").
Make these changes in the main branch before releasing: In the checklist below, we'll use a literal `X.Y` to stand for the new version.
In case of a major release, `Y = 0`.
* [ ] Ensure all applications/dependencies/charts are at their latest versions ## Pre-release chores
see `.gitlab/issue_templates/update_all_components.md`
* Update/review `docs/*` and make sure it matches the current state
* [ ] Update how to checkout the latest stable tag
* [ ] Document how to upgrade in `docs/upgrade.md`
* [ ] Update dependencies in `requirements.txt` by following the
instructions in `requirements.in`
* [ ] Do a manual upgrade test from last release version to `main`
* [ ] Create a release merge request for the main branch
## In the release merge request ### Update our custom charts
If necessary, do a release of the following charts. Don't forget to change the
version used in Stackspin (or wait for renovatebot to do that automatically).
* [ ] [dashboard](https://open.greenhost.net/stackspin/dashboard) (see also dashboard#10)
* [ ] [nextcloud](https://open.greenhost.net/stackspin/nextcloud) (see also nextcloud#1012)
* [ ] [wordpress](https://open.greenhost.net/stackspin/wordpress-helm)
* [ ] [hedgedoc](https://open.greenhost.net/stackspin/hedgedoc-helm-chart)
* [ ] [local-path-provisioner](https://open.greenhost.net/stackspin/local-path-provisioner)
## Release candidate
* [ ] Create a new branch, `release-candidate/vX.Y`, from main:
```sh
git checkout main
git pull
git checkout -b release-candidate/vX.Y
```
### Only for major releases
* [ ] Update how to checkout the latest stable tag, in
`docs/installation/install_cli.rst`.
* [ ] Document how to upgrade in `docs/system_administration/upgrading.rst`
### For all releases
* [ ] update [CHANGELOG.md](https://keepachangelog.com)
* [ ] Include `Known issues`
* [ ] Update the version number in the `VERSION` file * [ ] Update the version number in the `VERSION` file
* [ ] Commit (signed) * Update [CHANGELOG.md](https://keepachangelog.com)
* [ ] Push to MR * [ ] Include all merged MR since last release, i.e. using [lab](https://github.com/zaquestion/lab#installation):
* [ ] Make sure main pipeline is successful before merging MR !
* [ ] Wait for MR to get merged into main ```sh
lab mr list -s merged -a | \
awk '{first = $1; $1=""; print "*" $0, "(" first ")"}'
```
* [ ] Include app charts and versions table. You can use
[tablemark-cli](https://github.com/haltcase/tablemark-cli), or
[an online tool](https://tableconvert.com/json-to-markdown):
```sh
helm ls -A -o json | jq 'map({name, chart, app_version})' | \
jq 'map(.chart |= split("-")[-1])' > /tmp/versions.json
```
Now look for image tag overrides and fix those app versions in the
table:
```sh
find ./flux2 -name '*.yaml' | xargs grep -A4 '^ *image:'
```
Then produce the final table:
```
tablemark /tmp/versions.json
```
* [ ] Include `Known issues`
* [ ] Update app versions in `flux2/core/base/dashboard/dashboard-apps-configmap.yaml`.
* [ ] Only for major versions: prepare a release blog post. You can
use the `CHANGELOG.md` as basis. (It will be published at a later step.)
* [ ] Decide where to link to from the dashboard "release notes" link. Could be
the release blog post, other release notes page, or as fallback the
CHANGELOG.md entry for this version.
* [ ] Update `flux2/cluster/base/stackspin-static-info.yaml` to reflect the new
version and the release notes URL.
* [ ] Commit, push to `release-candidate/vX.Y`.
* [ ] Only for minor versions: create MR to merge `release-candidate/vX.Y` into
`pre-release/vX`
### Manual pre-release upgrade testing (only major releases)
* [ ] Try the proposed upgrade procedure on a fresh cluster installed with the
previous major version. Make sure to document any required actions and
potential problems in the upgrade docs.
### Automatic pre-release upgrade testing (only minor releases)
We have a special upgrade pipeline to test upgrading from the previous release
to this new (candidate) release. This pipeline runs when the target branch of a
MR matches `pre-release/*`. In principle you could make a MR to merge `main`
into `pre-release/vX`, but that has the unfortunate side-effect that the
pipeline will restart whenever the source branch changes, which happens all the
time because of renovatebot automerging minor updates and colleagues that work
too hard. Therefore:
* [ ] Make sure that the resulting upgrade-test pipeline is successful
* [ ] Before merging, notify #general on `stackspin.net` that we're about to
release, and that `stackspin.net` itself can experience short downtime while
the upgrade is in progress.
* [ ] Wait until MR gets reviewed and merged
* [ ] The CI machine created by the upgrade-test pipeline doesn't get destroyed
automatically, so please remove it yourself.
### Check automatic upgrade on `stackspin.net` (only minor releases)
Now that the new code has been merged to `pre-release/vX`, it will be picked
up by the `stackspin.net` cluster which is set to follow that branch. Even
though the upgrade pipeline already tested the upgrade process, it's still good
to check if the upgrade goes well there:
* [ ] do a `flux reconcile source git stackspin` so you don't have to wait
until flux decides it's time to reconcile;
* [ ] `watch flux get kustomization` to see components being upgraded. If
necessary check the status using `kubectl describe hr ...` and debug.
## Quality assurance
Now is a good time to spend some time using the `stackspin.net` instance: do
some basic manual testing of the dashboard, log in to all the apps, check if
the Nextcloud apps are all working, check any major changes or new features,
etc. Typically we keep the release candidate on `stackspin.net` for a week or
so before pushing out the release, but that really depends on the type of
changes.
## Release branch (only minor releases)
* [ ] Merge the just-updated `pre-release/vX` into the release branch `vX`.
## Release branches (only major releases)
* [ ] Create a new branch `vX` from `v(X-1)`.
* [ ] Change all occurrences of the last release to the new release version:
`sed -i 's/v(X-1)/vX/g' .gitlab/ci_pipelines/upgrade_test.yml \
.gitlab/issue_templates/update_all_components.md \
.gitlab/issue_templates/deploy_stackspin.md \
.gitlab/issue_templates/release.md \
docs/conf.py`
* [ ] Create a merge request for merging the release candidate
into the release branch `vX`. Ask for review and merge.
* [ ] Create a new branch `pre-release/vX` from `vX`.
### Upgrade `stackspin.net` (only major releases)
Note that `stackspin.net` doesn't follow a release branch (`vX`) but a
pre-release branch.
* [ ] Apply the upgrade procedure to `stackspin.net`: following
https://docs.stackspin.net/en/latest/system_administration/upgrading.html
and any actions specific to this release.
* [ ] Configure the cluster to follow the new pre-release branch: `pre-release/vX`.
### Upgrade `staging.stackspin.net` (only major releases)
Note that `staging.stackspin.net` doesn't follow a release branch (`vX`) but the
`main` branch.
* [ ] Apply the upgrade procedure to `staging.stackspin.net`, but do not change
the branch followed by flux (keep it at `main`).
## Push a signed tag ## Push a signed tag
* [ ] Make sure the main pipeline succeeds for the last commit before tagging. This * [ ] Create and push signed tag
is important because tags should not get retagged ! (`git tag -s X.Y -m 'Release X.Y'; git push --tags`)
* [ ] Create and push signed tag (`git tag -s 0.7.0 -m 'Release 0.7.0'; git push --tags`) * If this is a major release:
* [ ] Log into <https://readthedocs.org> and update documentation for tagged versions.
If the new branch is not shown, rebuild the docs for any existing branch,
which will as a side-effect fetch all branches from our gitlab.
* [ ] Publish the release blog post.
* [ ] Create an issue for upgrading managed clusters:
* our own production clusters;
* our managed customer clusters.
* [ ] Create a new CI pipeline image:
https://docs.stackspin.net/en/latest/for_developers/contributing.html#ci-pipeline-image
## Post-release chores
* [ ] Announce the release in the public Stackspin matrix room.
* [ ] Notify Greenhost sysops that an upgrade to their cluster will happen
overnight.
* [ ] Update the "stable" CI docker image (used as base for the upgrade pipeline):
- `tag="open.greenhost.net:4567/stackspin/stackspin/stackspin-ci:vX"`
- `docker build -t $tag .`
- `docker push $tag`
* [ ] Merge the release branch back into `main`. This is necessary to propagate
the changes to CHANGELOG etc.
* [ ] After `demo.stackspin.net` is upgraded, create a new backup for the new
version that will be used for the nightly reset. See `cli.stackspin.net`, directory
`/srv/stackspin/clusters/demo.stackspin.net/custom-scripts`.
* [ ] Update the version of the stackspin repo on `cli.stackspin.net`:
- `git pull` in `/srv/stackspin` to get the latest version of the release branch.
- Update python requirements: `pip install -r requirements.txt`. Note
that we use a virtualenv on `cli.stackspin.net` that's in the PATH of the
root user.
* [ ] Close released milestone and set start date for the new milestone.
### Update flux
See available [releases](https://github.com/fluxcd/flux2/releases).
* [ ] Update flux CLI version:
## In the new release branch current_flux_version=$(yq eval .flux.version ansible/group_vars/all/stackspin.yml)
new_flux_version=$(curl -s https://api.github.com/repos/fluxcd/flux2/releases/latest | jq -r .name | tr -d 'v')
sed -i "s/$current_flux_version/$new_flux_version/g" Dockerfile install/flux-version-check.sh docs/installation/install_cli.rst ansible/group_vars/all/stackspin.yml
* [ ] If this is a major release, create a new release branch (i.e. `v0.7`) * [ ] After changing the required flux version in `flux-version-check.sh`,
you also need to rebuild the CI image to contain the new binary, otherwise
the check will fail in the CI.
Create a MR for the new branch with the following changes: * [ ] Update flux version (daemon components running on clusters): see
`dev-tools/generate-flux-manifests/README.md`.
* Ensure all dependencies are updated and locked (locking should be done * [ ] After the new version of the CLI in particular has been merged to `main`,
only on the release branch): manually run a pipeline for `main` (with `CI_PIPELINE_SOURCE: schedule`) so
* [ ] flux chart versions: `find flux2/* -type f -exec yq eval .spec.chart {} \;` the `main` CI image gets rebuilt with the new flux. Although we don't
* [ ] Wait for MR to get merged regularly run pipelines for `main` anymore, this image is used by pipelines for
MRs that don't change the CI image.
## After release ## Celebration
* [ ] Log into https://readthedocs.org and update documentation for tagged versions
* [ ] Cherry-pick all changes from the release branch into main which
shouldn't be only in the 0.4.0 branch (i.e. not the commit locking the
dependencies)
* [ ] Create issue for creating a release blog post
* [ ] Celebrate :clinking_glass: !! * [ ] Celebrate :clinking_glass: !!
.gitlab/issue_templates/update_all_components.md deleted 100644 → 0
View file @ d6090192
## Update all applications in this repository
* [ ] Update Flux version
current_flux_version=0.20.1
new_flux_version=0.24.0
sed -i "s/$current_flux_version/$new_flux_version/g" Dockerfile install/flux-version-check.sh docs/installation/install_stackspin.rst ansible/group_vars/all/stackspin.yml
* [ ] Update all apk app versions inside `Dockerfile`
* [ ] Update mitogen version in `ansible/plugins` (https://github.com/mitogen-hq/mitogen/releases)
* [ ] Update k3s in `ansible/group_vars/all/stackspin.yml`:
## Update our custom charts:
We get dependeny updates via renovate for our custom helm charts. However, we
still don't get [automated version
bumps](https://open.greenhost.net/stackspin/stackspin/-/issues/1001) with it, so
make sure to bump the version if needed for:
* [ ] [dashboard](https://open.greenhost.net/stackspin/dashboard)
* [ ] [nextcloud](https://open.greenhost.net/stackspin/nextcloud)
* [ ] [single-sign-on](https://open.greenhost.net/stackspin/single-sign-on)
* [ ] [wordpress](https://open.greenhost.net/stackspin/wordpress-helm)
Pull in upstream changes to:
* [ ] [local-path-provisioner](https://open.greenhost.net/stackspin/local-path-provisioner)
.pre-commit-config.yaml
View file @ 601a9684
repos: repos:
- repo: https://github.com/pre-commit/pre-commit-hooks - repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.0.1 rev: v5.0.0
hooks: hooks:
- id: check-added-large-files - id: check-added-large-files
- id: check-ast - id: check-ast
- id: check-merge-conflict - id: check-merge-conflict
- id: check-yaml - id: check-yaml
args:
- --allow-multiple-documents
exclude: |
(?x)^(
.gitlab-ci.yml|
.gitlab.*
)$
- id: detect-private-key - id: detect-private-key
- id: end-of-file-fixer - id: end-of-file-fixer
- id: trailing-whitespace - id: trailing-whitespace
- id: debug-statements - id: debug-statements
- repo: https://github.com/jumanjihouse/pre-commit-hooks - repo: https://github.com/jumanjihouse/pre-commit-hooks
rev: 2.1.5 rev: 3.0.0
hooks: hooks:
- id: shellcheck - id: shellcheck
- id: shfmt - id: shfmt
...@@ -20,11 +27,24 @@ repos: ...@@ -20,11 +27,24 @@ repos:
- "2" - "2"
- "-ci" - "-ci"
- repo: https://github.com/hadolint/hadolint - repo: https://github.com/hadolint/hadolint
rev: v1.23.0 rev: v2.12.0
hooks: hooks:
- id: hadolint-docker - id: hadolint-docker
- repo: https://github.com/timothycrosley/isort - repo: https://github.com/timothycrosley/isort
# Sorts import statements for you # Sorts import statements for you
rev: 5.0.9 rev: 6.0.1
hooks: hooks:
- id: isort - id: isort
- repo: https://github.com/zricethezav/gitleaks
rev: v8.24.2
hooks:
- id: gitleaks-docker
- repo: https://github.com/python-jsonschema/check-jsonschema
rev: 0.32.1
hooks:
- id: check-gitlab-ci
files: |
(?x)^(
.gitlab-ci.yml|
.gitlab.*
)$
.readthedocs.yaml 0 → 100644
View file @ 601a9684
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
version: 2
build:
os: ubuntu-22.04
tools:
python: "3.11"
# Build documentation in the docs/ directory with Sphinx
sphinx:
configuration: docs/conf.py
# https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html
python:
install:
- requirements: docs/requirements.txt
.renovaterc.json deleted 100644 → 0
View file @ d6090192
{
"regexManagers": [
{
"fileMatch": ["flux2/.+release\\.yaml$"],
"matchStrings": [
"registryUrl=(?<registryUrl>.*?)\n *chart: (?<depName>.*?)\n *version: (?<currentValue>.*)\n"
],
"datasourceTemplate": "helm"
}
]
}
CHANGELOG.md
View file @ 601a9684
# Changelog # Changelog
All notable changes to this project will be documented in this file. ## 2.17
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased] ### Features
* Changed project name from OpenAppStack to Stackspin * Introduce support for upstream OIDC. By using an external identity provider,
* Update: this allows you to sign in using their single sign-on system, automatically
* Kube-prometheus-stack to helm chart version 22.0.0 creating a corresponding Stackspin user if necessary. At the moment there is
no UI for configuring this, so this requires some detailed knowledge of
Stackspin and Kratos to set up. Let us know if you want to try this and need
assistance.
* Enable exhaustive nextcloud previews (!2428)
### Updates
* Update helm release velero to v8.4.0 (!2515)
- This changes the backend for backing up persistent data from restic to
kopia. This seems to work fine also for existing clusters, and should even
allow you to restore from old backups. However, because the data repositories
are not compatible, kopia cannot do an incremental backup using the restic
backups and will have to do a full backup at first. If you have a lot of
data, this means a long backup job, and higher data usage in your backup
location.
* Remove mariadb stackspin database, as last step of the migration from mariadb
to mysql. (!2320)
* Update helm release nextcloud-onlyoffice to v0.15.33 (!2539)
- This upgrades Nextcloud to 30.0.4.
* Update helm release stackspin-dashboard to v1.14.0 (!2538)
* Update metallb docker tag to v6.4.8 (!2534)
* Update helm release kube-prometheus-stack to v69.4.1 (!2520)
* Update stackspin database mysql docker tag to v12.2.4 (!2518)
* Update helm release kratos to v0.51.0 (!2503)
* Update helm release cert-manager to v1.17.1 (!2500)
* Update dependency k3s-io/k3s to v1.32.1+k3s1 (!2477)
* Update helm release hydra to v0.51.0 (!2494)
* Update helm release prometheus-blackbox-exporter to v9.2.0 (!2489)
* Update helm release kubernetes-secret-generator to v3.4.1 (!2484)
* Update rancher/local-path-provisioner docker tag to v0.0.31 (!2469)
* Update helm release ingress-nginx to v4.12.1 (!2547)
* Update helm release system-upgrade-controller to v0.5.1 (!2432)
* Upgrade flux controllers to v2.4 (!2418)
* Update nextcloud app groupfolders to v18.1.1 (!2536)
* Update nextcloud app calendar to v5.1.3 (!2535)
* Update nextcloud app contacts to v7.0.3 (!2532)
* Update nextcloud app appointments to v2.3.6 (!2531)
* Update nextcloud app nextcloud-oidc-login to v3.2.2 (!2530)
* Update nextcloud app polls to v7.3.2 (!2528)
* Update nextcloud app forms to v4.3.10 (!2543)
### Documentation
* Add coopcloud and slight restructuring (!2154)
### Versions
* Host OS: Debian 12
* Kubernetes: k3s v1.32.1+k3s1
* Flux: v2.4.0
| **name** | **chart** | **app_version** |
|---------------------------|-----------|-----------------------|
| blackbox-exporter | 9.2.0 | v0.25.0 |
| cert-manager | v1.17.1 | v1.17.1 |
| dashboard | 1.14.0 | 0.14.0 |
| hedgedoc | 0.1.6 | 1.10.0 |
| hydra | 0.51.0 | v2.2.0 |
| ingress-nginx | 4.12.1 | 1.12.1 |
| kratos | 0.51.0 | v1.3.0 |
| kube-prometheus-stack | 69.3.2 | v0.80.0 |
| local-path-provisioner | 0.0.27 | v0.0.27 |
| metallb | 6.4.6 | 0.14.9 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud | 0.15.32 | NC-30.0.2-OO-7.2.2.56 |
| secrets-controller | 3.4.1 | v3.4.1 |
| stackspin-database | 12.2.2 | 8.4.4 |
| system-upgrade-controller | 0.5.1 | v0.14.2 |
| velero | 8.4.0 | 1.15.2 |
| wekan | 1.1.1 | 7.01 |
| wordpress | 0.8.6 | 6.6.2 |
| zulip | 0.8.4-gh1 | 8.4-0 |
### Known issues
The new velero backend for persistent data, based on kopia, runs hourly
maintenance jobs on the kopia data repo. This works most of the time but fails
once in a while, resulting in KubeJobFailed alerts. These alerts have a
description like this:
> Job velero/stackspin-default-kopia-2ttfn-maintain-job-1742698393281 failed to
> complete. Removing failed job after investigation should clear this alert.
Actually the job is automatically removed by velero so no action is required.
This particular alert can thus be ignored, unless it fires every hour. We will
investigate the cause of these intermittent failures and/or adjust the alerting
to not fire on infrequent errors in these jobs.
## 2.16
### Features
* Introduce support for theming the Stackspin dashboard. For more info, see the
documentation:
https://docs.stackspin.net/en/latest/system_administration/customizing.html#custom-dashboard-styling
* Only show Stackspin version info to admin users.
### Fixes
* Fix the initial password (re)set email which was not being sent since the
upgrading of the Kratos client library in the previous release.
* Mongodb cannot deal with password starting with dash (!2353)
* Adapt to new flux metrics (!2340)
### Updates
* Update dependency k3s-io/k3s to v1.31.2+k3s1 (!2384)
* Update helm release stackspin-dashboard to v1.13.1 (!2415)
* Update helm release metallb to v6.3.15 (!2393)
* Update helm release system-upgrade-controller to v0.5.0 (!2373)
* Update rancher/local-path-provisioner docker tag to v0.0.30 (!2361)
* Update helm release ingress-nginx to v4.11.3 (!2344)
* Update helm release cert-manager to v1.16.2 (!2409)
* Update helm release kratos to v0.50.0 (!2406)
* Update helm release kube-prometheus-stack to v66 (!2400)
* Update helm release prometheus-blackbox-exporter to v9.1.0 (!2397)
* Update helm release nextcloud-onlyoffice to v0.15.31 (!2352)
- In particular, this upgrades Nextcloud to 30.0.0.
* Update dependency onlyoffice/onlyoffice-nextcloud to v9.5.0 (!2365)
* Update dependency sergeymosin/appointments to v2.2.0 (!2405)
* Update dependency nextcloud-releases/groupfolders to v18.0.5 (!2403)
* Update dependency nextcloud-releases/calendar to v5.0.5 (!2408)
* Update dependency nextcloud-releases/external to v5.5.2 (!2359)
* Update dependency nextcloud-releases/contacts to v6.1.1 (!2398)
* Update dependency nextcloud-releases/forms to v4.3.3 (!2395)
### Other changes
* Switch the Stackspin database from mariadb to mysql, because hydra
(the single-sign on engine) is not compatible with mariadb.
### Versions
* Host OS: Debian 12
* Kubernetes: k3s v1.31.2+k3s1
* Flux: v2.3.0
| **name** | **chart** | **app_version** |
|---------------------------|-----------|-----------------------|
| blackbox-exporter | 9.1.0 | v0.25.0 |
| cert-manager | v1.16.2 | v1.16.2 |
| dashboard | 1.13.1 | 0.13.1 |
| hedgedoc | 0.1.6 | 1.10.0 |
| hydra | 0.25.6 | v1.11.8 |
| ingress-nginx | 4.11.3 | 1.11.3 |
| kratos | 0.50.0 | v1.3.0 |
| kube-prometheus-stack | 66.2.1 | v0.78.1 |
| local-path-provisioner | 0.0.27 | v0.0.27 |
| metallb | 6.3.15 | 0.14.8 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud | 0.15.31 | NC-30.0.0-OO-7.2.2.56 |
| secrets-controller | 3.4.0 | v3.4.0 |
| single-sign-on-database | 11.4.6 | 10.6.12 |
| stackspin-database | 11.1.20 | 8.4.3 |
| system-upgrade-controller | 0.5.0 | v0.14.1 |
| velero | 4.4.1 | 1.11.1 |
| wekan | 1.1.1 | 7.01 |
| wordpress | 0.8.6 | 6.6.2 |
| zulip | 0.8.4-gh1 | 8.4-0 |
## 2.15
### Features
* Introduce support for WebAuthn as alternative second factor authentication in
the dashboard. This includes support for hardware keys. The new functionality
has been tested to work, but the interface requires some more work, which is
planned for a later release. Any feedback on this new feature is most welcome.
* Do not show app versions to regular users, only to admins.
* Make disk space warning configurable (!2326)
* Pre-fill email address in kratos welcome email (!2313)
* Automatically set the Nextcloud instance name for new installs.
### Fixes
* Fix links to `localhost` in Nextcloud-generated emails.
* Fix some of the new Nextcloud setup warnings.
* Do not serve Dockerfile and nginx.conf from dashboard frontend.
### Updates
* Update dependency k3s-io/k3s to v1.31.1+k3s1 (!2324)
* Upgrade flux controllers to v2.3 (!2289)
* Update helm release metallb to v6.3.13 (!2332)
* Update helm release ingress-nginx to v4.11.2 (!2294)
* Update helm release cert-manager to v1.15.3 (!2293)
* Update helm release stackspin-dashboard to v1.12.4
* Update helm release hedgedoc to v0.1.6 (!2306)
- This upgrades Hedgedoc to 0.10.0.
* Update helm release wordpress to v0.8.6 (!2334)
- This upgrades WordPress to 6.6.2.
* Update helm release nextcloud-onlyoffice to v0.15.30 (!2328)
- This upgrades Nextcloud to 29.
* Update dependency nextcloud-releases/groupfolders to v17.0.4 (!2327)
* Update dependency nextcloud/polls to v7.2.3 (!2319)
* Update dependency sergeymosin/appointments to v2.1.10 (!2310)
* Update dependency nextcloud-releases/calendar to v4.7.16 (!2305)
### Versions
* Host OS: Debian 12
* Kubernetes: k3s v1.31.1+k3s1
* Flux: v2.3.0
| **name** | **chart** | **app_version** |
|---------------------------|-----------|-----------------------|
| cert-manager | v1.15.3 | v1.15.3 |
| dashboard | 1.12.4 | 0.12.4 |
| hedgedoc | 0.1.6 | 1.10.0 |
| hydra | 0.25.6 | v1.11.8 |
| ingress-nginx | 4.11.2 | 1.11.2 |
| kratos | 0.45.0 | v1.1.0 |
| kube-prometheus-stack | 52.1.0 | v0.68.0 |
| local-path-provisioner | 0.0.27 | v0.0.27 |
| metallb | 6.3.12 | 0.14.8 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud | 0.15.30 | NC-29.0.4-OO-7.2.2.56 |
| secrets-controller | 3.4.0 | v3.4.0 |
| single-sign-on-database | 11.4.6 | 10.6.12 |
| stackspin-database | 11.1.16 | 8.4.2 |
| system-upgrade-controller | 0.4.1 | v0.13.4 |
| velero | 4.4.1 | 1.11.1 |
| wekan | 1.1.1 | 7.01 |
| wordpress | 0.8.6 | 6.6.2 |
| zulip | 0.8.4-gh1 | 8.4-0 |
## 2.14
### Fixes
* Fix the password reset form, which was broken since 2.13 by the new feature
to pre-fill an email address.
* Fix flash messages after the first one not showing in the dashboard. One
common instance of this was that no message was shown when you provide a
wrong TOTP code during login.
### Updates
* Update dependency k3s-io/k3s to v1.30.3+k3s1 (!2286)
* Upgrade our flux resources to new api versions in flux v2.2 (!2269)
* Update helm release stackspin-dashboard to v1.11.1 (!2287)
* Update helm release ingress-nginx to v4.11.1 (!2280)
* Update helm release metallb to v6.3.8 (!2275)
* Update helm release nextcloud-onlyoffice to v0.15.24 (!2283)
- We do not upgrade Nextcloud to 29 yet because of unresolved configuration
issues.
* Update app onlyoffice/onlyoffice-nextcloud to v9.3.0 (!2277)
* Update app nextcloud/polls to v7.1.4 (!2274)
* Update app nextcloud-releases/calendar to v4.7.12 (!2273)
### Versions
* Host OS: Debian 12
* Kubernetes: k3s v1.30.3+k3s1
* Flux: v2.2.0
| **name** | **chart** | **app_version** |
|---------------------------|-----------|-----------------------|
| cert-manager | v1.15.1 | v1.15.1 |
| dashboard | 1.11.1 | 0.11.1 |
| hedgedoc | 0.1.5 | 1.9.9 |
| hydra | 0.25.6 | v1.11.8 |
| ingress-nginx | 4.11.1 | 1.11.1 |
| kratos | 0.45.0 | v1.1.0 |
| kube-prometheus-stack | 52.1.0 | v0.68.0 |
| local-path-provisioner | 0.0.27 | v0.0.27 |
| metallb | 6.3.8 | 0.14.5 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud | 0.15.24 | NC-28.0.7-OO-7.2.2.56 |
| secrets-controller | 3.4.0 | v3.4.0 |
| single-sign-on-database | 11.4.6 | 10.6.12 |
| system-upgrade-controller | 0.4.1 | v0.13.4 |
| velero | 4.4.1 | 1.11.1 |
| wekan | 1.1.1 | 7.01 |
| wordpress | 0.8.5 | 6.5.4 |
| zulip | gh1 | 8.4-0 |
## 2.13
### Features
* Allow pre-filling user's email address in a link to the password (re)set
form. This is useful when creating new user accounts.
### Fixes
* Compare email addresses case insensitively to deal with Stackspin apps
changing case of email address strings.
* Fix broken user accounts when created via the flask CLI.
* Increase memory limit for nextcloud (!2236)
* Fix user provisioning after installing new apps.
### Updates
* Add a system-upgrade-controller Plan to perform a fully unattended upgrade of
the host OS from Debian 11 to 12. Because of the decentralized nature of
Stackspin, it could be that some are running Stackspin on something other
than Debian 11, or have other important services running on the same host. For
that reason we chose to not enable this host OS upgrade automatically. To
enable it for a Stackspin instance, perform the following command on the VPS:
`kubectl label node $yournode stackspin.net/provisioned-os=debian`. If you're
unsure what value to use as `$yournode`, use `kubectl get node` first. Please
note that the OS upgrade also does a system reboot, so will incur some
downtime of your Stackspin instance. By default it will be performed during a
maintenance window in the night. (!2257)
* Upgrade flux controllers to 2.2 (!2241)
* Upgrade kubernetes to 1.29 (!2246)
* Upgrade nextcloud to 28 (!2249)
* Upgrade wordpress-helm to 0.8.5.
- This upgrades wordpress to 6.5.4.
* Update helm release system-upgrade-controller to v0.4.1 (!2264)
* Update helm release cert-manager to v1.15.1 (!2255)
* Update helm release kratos to v0.45.0 (!2253)
* Update helm release stackspin-dashboard to v1.10.5 (!2247)
* Update Nextcloud app `appointments` to v2.1.6 (!2263)
* Update Nextcloud app `calendar` to v4.7.9 (!2260)
* Update Nextcloud app `polls` to v7.1.3 (!2261)
### Versions
* Host OS: Debian 12
* Kubernetes: k3s v1.29.5+k3s1
* Flux: v2.2.0
| **name** | **chart** | **app_version** |
|---------------------------|-----------|-----------------------|
| cert-manager | v1.15.1 | v1.15.1 |
| dashboard | 1.10.5 | 0.10.5 |
| hedgedoc | 0.1.5 | 1.9.9 |
| hydra | 0.25.6 | v1.11.8 |
| ingress-nginx | 4.10.1 | 1.10.1 |
| kratos | 0.45.0 | v1.1.0 |
| kube-prometheus-stack | 52.1.0 | v0.68.0 |
| local-path-provisioner | 0.0.27 | v0.0.27 |
| metallb | 4.16.1 | 0.14.3 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud | 0.15.23 | NC-28.0.7-OO-7.2.2.56 |
| secrets-controller | 3.4.0 | v3.4.0 |
| single-sign-on-database | 11.4.6 | 10.6.12 |
| system-upgrade-controller | 0.4.0 | v0.13.4 |
| velero | 4.4.1 | 1.11.1 |
| wekan | 1.1.1 | 7.01 |
| wordpress | 0.8.5 | 6.5.4 |
| zulip | 0.8.4-gh1 | 8.4-0 |
## 2.12
### Features
* Include new "System resources" dashboard module with basic statistics.
* We introduced support in Stackspin for SCIM, a protocol for provisioning and
deprovisioning users and user groups. Practically, this means that when you
create a new user account or change a user's app permissions, the dashboard
will instruct the affected applications -- currently supported only by
Nextcloud and Zulip -- to create the new user right away, or change their
in-app permissions. This means that you can start assigning other permissions
or resources to the new account inside the app -- for example, adding them to
a Zulip channel or a Nextcloud group -- right away, instead of having to wait
for the user the log in first.
- Track when a user's full name has been changed, and only include the name
in the SCIM provisioning call when it has changed, or for newly provisioned
users.
* Show custom error for ingress-generated pages (!2208)
* Add a CLI command for deleting older unused accounts.
* Add logo for Gitea in the dashboard.
* Automatically reload the dashboard config when its configmaps with lists of
apps and oauthclients change.
* Fine-tune dashboard backend logging levels, and introduce a new environment
variable `LOG_LEVEL` to set the log level at runtime.
### Fixes
* Set right Zulip role for admins (!2210)
* Make info modals slightly wider, to make sure you can see the full contents
also for slightly larger fonts. In particular, this fixes a partially
invisible reset link.
* Velero OOM during restic maintenance (!2164)
* Start out with system-upgrade-controller scaled down (!2162)
* migration-scripts: fix hedgedoc id issue (!2155)
### Documentation
* Document Hedgedoc system requirements (!1988)
* Document prometheus alert KubeClientCertificateExpiration (!2221)
### Updates
* Upgrade flux resources to v2.0 (!2165)
- After we upgraded flux itself to v2 in the previous release, we could now
upgrade all flux resource manifests to the new API versions.
* Upgrade dashboard frontend to tailwind v3, and update several other
javascript dependencies.
* update helm release cert-manager to v1.15.0 (!2233)
* update helm release hedgedoc to v0.1.5 (!2216)
* update helm release ingress-nginx to v4.10.1 (!2213)
* update helm release wordpress to v0.8.4
* update dependency ansible to v9 (!2200)
* update helm release metallb to v4.16.1 (!2196)
* update helm release kratos to v0.40.1 (!2195)
* update dependency nextcloud/forms to v3.4.7 (!2211)
* update dependency nextcloud-releases/contacts to v5.5.3 (!2184)
* update dependency sergeymosin/appointments to v1.15.5 (!2170)
* update dependency nextcloud/polls to v5.4.3 (!2168)
* update dependency nextcloud-releases/calendar to v4.7.6 (!2229)
* update dependency nextcloud-releases/groupfolders to v15.3.8 (!2228)
* update dependency gino0631/nextcloud-metadata to v0.20.0 (!2226)
### Versions
* Host OS: Debian 11
* Kubernetes: k3s v1.28.2+k3s1
* Flux: v2.1.2
| **Name** | **Chart** | **App version** |
|---------------------------|-----------|-----------------------|
| cert-manager | v1.15.0 | v1.15.0 |
| dashboard | 1.10.4 | 0.10.4 |
| hedgedoc | 0.1.5 | 1.9.9 |
| hydra | 0.25.6 | v1.11.8 |
| ingress-nginx | 4.10.1 | 1.10.1 |
| kratos | 0.40.1 | v1.1.0 |
| kube-prometheus-stack | 52.1.0 | v0.68.0 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| metallb | 4.16.1 | 0.14.3 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud | 0.15.21 | NC-27.1.4-OO-7.2.2.56 |
| secrets-controller | 3.4.0 | v3.4.0 |
| single-sign-on-database | 11.4.6 | 10.6.12 |
| system-upgrade-controller | 0.3.1 | v0.13.1 |
| velero | 4.4.1 | 1.11.1 |
| wekan | 1.1.1 | 7.01 |
| wordpress | 0.8.4 | 6.5.2 |
| zulip | 0.8.4-gh1 | 8.4-0 |
### Known issues
## 2.11
### Features
* Improve dashboard user listing:
- Show label for admin users
- Show last login and password reset times
- Improved layout
* Allow removing user app roles from CLI.
* Do not autocomplete totp input field.
### Fixes
* Password reset fails when 2FA is active (!2152)
* Missing k3s config directory during installation (!2132)
* Hedgedoc users are logged out on pod restart (!2117)
* Grafana fails at startup because flux (re)moved dashboards (!2146)
* ingress-nginx 4.8.4 retracted (!2150)
* Fix rare bug in frontend's idea of admin status in the face of custom apps.
* velero: Improve backup alerts (!2149)
### Documentation
* installation: link flux variable edit instructions (!2148)
### Updates
* Upgrade kubernetes from 1.25 to 1.28 (!2121)
* Upgrade flux to v2.0 (!2129)
* update helm release metallb to v4.7.16 (!2138)
* update helm release ingress-nginx to v4.9.0 (!2151)
* update helm release cert-manager to v1.13.3 (!2143)
* update helm release kratos to v0.37.1 (!2137)
* update helm release kube-prometheus-stack to v52 (!2114)
* Upgrade Nextcloud to 27.1.4.
* update dependency nextcloud/polls to v5.4.2 (!2134)
* update dependency nextcloud-releases/external to v5.3.1 (!2127)
* update dependency nextcloud-releases/calendar to v4.5.3 (!2122)
* Upgrade Hedgedoc to 1.9.9.
* Upgrade Hedgedoc's mariadb to 12.2.9.
### Versions
| **name** | **chart** | **app_version** |
|---------------------------|-----------|-----------------------|
| cert-manager | v1.13.3 | v1.13.3 |
| dashboard | 1.9.1 | 0.9.1 |
| hedgedoc | 0.1.4 | 1.9.9 |
| hydra | 0.25.6 | v1.11.8 |
| ingress-nginx | 4.9.0 | 1.9.5 |
| kratos | 0.37.1 | v1.0.0 |
| kube-prometheus-stack | 52.1.0 | v0.68.0 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| metallb | 4.7.16 | 0.13.12 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud | 0.15.20 | NC-27.1.4-OO-7.2.2.56 |
| secrets-controller | 3.4.0 | v3.4.0 |
| single-sign-on-database | 11.4.6 | 10.6.12 |
| system-upgrade-controller | 0.3.1 | v0.13.1 |
| velero | 4.4.1 | 1.11.1 |
| wekan | 1.1.1 | 7.01 |
| wordpress | 0.7.30 | 6.0.1 |
| zulip | gh1 | 7.5-0 |
### Known issues
#### K3s upgrade fails if dbus not running
After the release, we found that some of our managed clusters do not have the
dbus service enabled for some unknown reason. If your system is like that, the
kubernetes/k3s upgrade will fail. The symptom is a KubeJobFailed prometheus
alert, which if you check `kubectl get job -A` will appear to be a
`apply-k3s-server-on-dashboard-with-*-*` job in the `system-upgrade` namespace.
You can confirm that dbus is the problem by checking the log file on the host
at `/tmp/stackspin-k3s-upgrade-prepare.log`, this contains a message about not
finding the dbus socket at `/host/run/dbus/system_bus_socket`.
The solution is to enable and start dbus on the host, and delete the failed job
by `kubectl delete job -n system-upgrade
apply-k3s-server-on-*-with-*-*`. During the next night the upgrade
controller will be automatically scaled up and re-attempt the upgrade job. If
you're eager to see whether that works, you can do the upgrade right away with
`kubectl scale --replicas=1 -n system-upgrade
deploy/system-upgrade-controller`.
### adopt-secrets fails sometimes for uninstalled apps
In some cases the `adopt-secrets` migration script fail. We've seen this happen
specifically in case you have installed an app in the past, before the time we
had the secrets-controller, and removed the app again afterwards. This results
in a combination of present secret and absent secret template, which the
migration script is not prepared for. The symptom is a KubeJobFailed prometheus
alert due to the `flux-system/adopt-secrets` job failing. The solution is to
delete the lingering secret of the uninstalled app, and then delete the failed
job.
## 2.10.1
### Fixes
* Downgrade the Nextcloud `groupfolders` app back from `15.3.1` to `15.2.0`.
The newer version turned out to be incompatible with our Nextcloud version
27, resulting in disabled group folder functionality.
## 2.10
### Apps
* Upgrade zulip to 7
### Features
* Make it possible to make 2FA mandatory.
* Add a button to the Stackspin dashboard for admin users to reset 2FA of
users. Also improve UX of this and other dangerous operations in the user
edit screen.
* We implemented a Prometheus alert to warn when disk space falls under a
threshold (20%).
### Fixes
* Fix double user accounts in Hedgedoc. Because of a OIDC misconfiguration --
and unhelpful default setting in Hedgedoc -- we accidentally used the
Stackspin username as user identifier for Hedgedoc. This is now rectified by
switching to a stable identifier, and implementing a migration script that
automatically repairs any double user accounts that may have been created in
Hedgedoc.
* Disable Nextcloud bulkupload, to prevent failures with the desktop client.
* Fix some security and setup warnings in Nextcloud.
* Increase memory limits for velero, to allow backing up larger (Nextcloud)
data sets.
* Fix the SSO logout process to include hydra post-logout. This should prevent
various issues we've seen when logging in as multiple users on the same
device.
* Do not show link to start account recovery on the 2FA (TOTP) form.
* Fix css of the sign-up page (demo instance only).
### Documentation
* Document how to uninstall an app from Stackspin.
### Updates
* Update dependency nextcloud/polls to v5.4.1
* Update helm release cert-manager to v1.13.2
* Update helm release system-upgrade-controller to v0.3.1
* Update dependency onlyoffice/onlyoffice-nextcloud to v8.2.4
* Update helm release ingress-nginx to v4.8.3
* Update dependency kubernetes to v28
* Update helm release velero to v4.4.1
* Update helm release metallb to v4.7.11
* Update helm release ingress-nginx to v4.8.2
* Update dependency nextcloud-releases/groupfolders to v15.3.1
* Update dependency ansible to v8.5.0
* Update dependency nextcloud-releases/contacts to v5.4.2
* Update dependency nextcloud-releases/calendar to v4.5.2
* Update helm release kratos to v0.36.0
- This upgrades to kratos 1.0.
* Update Nextcloud's rabbitmq to v12.2.0.
* Upgrade the python version used in the dashboard backend to 3.12.
### Versions
| **name** | **chart** | **app_version** |
|---------------------------|-----------|-----------------------|
| cert-manager | v1.13.2 | v1.13.2 |
| dashboard | 1.8.4 | 0.8.4 |
| hedgedoc | 0.1.3 | 1.9.8 |
| hydra | 0.25.6 | v1.11.8 |
| ingress-nginx | 4.8.3 | 1.9.4 |
| kratos | 0.36.0 | v1.0.0 |
| kube-prometheus-stack | 46.8.0 | v0.65.2 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| metallb | 4.7.11 | 0.13.12 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud | 0.15.18 | NC-27.0.2-OO-7.2.2.56 |
| secrets-controller | 3.4.0 | v3.4.0 |
| single-sign-on-database | 11.4.6 | 10.6.12 |
| system-upgrade-controller | 0.3.1 | v0.13.1 |
| velero | 4.4.1 | 1.11.1 |
| wekan | 1.1.1 | 7.01 |
| wordpress | 0.7.30 | 6.0.1 |
| zulip | 0.7.4-gh1 | 7.4-0 |
## 2.9
### Apps
* Upgrade Nextcloud to 27.
### Fixes
* Prevent Nextcloud from removing users from custom user groups on login.
* Make sure that regular users do not get admin access to Wekan (!2046)
* Fix resetting your password when 2FA is enabled.
* Fix lingering SSO sessions when logging out from some apps (at least
Nextcloud and Wordpress).
* Reduce dropped database connections in the dashboard/SSO by enabling
"pre-ping".
* Do not list Velero in app permissions when batch-creating users.
### Features
* Add possibility in the Stackspin dashboard to quickly edit app permissions
for multiple users at once.
### Improvements
* Skip the Nextcloud login screen and forward to SSO directly.
* Enable all Stackspin-default Nextcloud apps by default.
* Rework management of app secrets (database passwords etc.) (!1991)
### Updates
* Update helm release dashboard to v0.8.1
* Update helm release nextcloud-onlyoffice to v0.15.17
* Update helm release cert-manager to v0.13.0
* Update Nextcloud app `onlyoffice` to v8.2.0
* Update dependency nextcloud-releases/groupfolders to v15.2.0
* Update dependency nextcloud-releases/contacts to v5.4.0
* Update dependency nextcloud-releases/calendar to v4.5.0
### Versions
| **name** | **chart** | **app_version** |
|---------------------------|-----------|-----------------------|
| cert-manager | v1.13.0 | v1.13.0 |
| dashboard | 1.8.2 | 0.8.2 |
| hedgedoc | 0.1.3 | 1.9.8 |
| hydra | 0.25.6 | v1.11.8 |
| ingress-nginx | 4.7.2 | 1.8.2 |
| kratos | 0.33.4 | v0.13.0 |
| kube-prometheus-stack | 46.8.0 | v0.65.2 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| metallb | 4.5.6 | 0.13.10 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud | 0.15.17 | NC-27.0.2-OO-7.2.2.56 |
| secrets-controller | 3.4.0 | v3.4.0 |
| single-sign-on-database | 11.4.6 | 10.6.12 |
| system-upgrade-controller | 0.3.0 | v0.11.0 |
| velero | 4.0.3 | 1.11.0 |
| wekan | 1.1.1 | 7.01 |
| wordpress | 0.7.30 | 6.0.1 |
| zulip | 0.4.0 | 6.0-0 |
## 2.8
### Apps
* Upgrade Wekan to v7.01
### Fixes
* Empty s3 prefix not accepted by new velero (!2024)
* App error pages are replaced by custom error backend (!2023)
### Features
* Do not create demo files for new users (!558)
* Skip Wekan login screen in favour of automatically using single sign-on.
* Add app metadata supporting connecting some more non-Stackspin apps:
Jitsi, Mattermost, Forgejo.
### Improvements
* Install Nextcloud apps on chart install (!544)
- This should greatly reduce Nextcloud downtime during (automatic) updates
and restarts.
### Updates
* update helm release stackspin-dashboard to v1.7.6 (!2044)
* update helm release nextcloud-onlyoffice to v0.15.15 (!2043)
* update helm release metallb to v4.5.6 (!2039)
* update helm release ingress-nginx to v4.7.1 (!2038)
* update helm release kratos to v0.33.4 (!2027)
* update dependency ansible to v8.1.0 (!2030)
* update dependency nextcloud-releases/contacts to v5.3.2 (!2026)
### Documentation
* Improve documentation on email settings (!2042)
* removed second "exec" from command (!2037)
### Versions
| **name** | **chart** | **app_version** |
|---------------------------|-----------|-----------------------|
| cert-manager | v1.12.2 | v1.12.2 |
| dashboard | 1.7.6 | 0.7.6 |
| hedgedoc | 0.1.3 | 1.9.8 |
| hydra | 0.25.6 | v1.11.8 |
| ingress-nginx | 4.7.1 | 1.8.1 |
| kratos | 0.33.4 | v0.13.0 |
| kube-prometheus-stack | 46.8.0 | v0.65.2 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| metallb | 4.5.6 | 0.13.10 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud | 0.15.15 | NC-26.0.2-OO-7.2.2.56 |
| single-sign-on-database | 11.4.6 | 10.6.12 |
| system-upgrade-controller | 0.3.0 | v0.11.0 |
| velero | 4.0.3 | 1.11.0 |
| wekan | 1.1.1 | 7.01 |
| wordpress | 0.7.30 | 6.0.1 |
| zulip | 0.4.0 | 6.0-0 |
## 2.7
### Apps
* Upgrade HedgeDoc to v1.9.8
* Upgrade Nextcloud to v26.0.2
* Upgrade onlyoffice app to v7.8.0
### Fixes
* Correct firewall rules to only block wan interfaces (!2004)
* allow loading of extra firewall rules (!1998)
* OOM for nextcloud database (!2009)
* Increase memory limit for Nextcloud pod (!1931)
* Automatically reset failing cache (!1978)
* Wordpress logout feature broken (!1987)
* Helm upgrade of nextcloud's redis fails (!1984)
### Features
* Improve the UX of the dashboard tiles: adding help texts in modals, add a status dropdown with version info, add alerts before and after automatic upgrades, show greeting, show tag when logged in as admin user.
* Remember the active tab in the authentication settings page when saving.
* No longer send password recovery emails to addresses that do not match an existing account.
* Improve Experience upon "503 Service Temporarily Unavailable" (!1913)
* Activate FreeURL mode for HedgeDoc (!1924)
* Alert on velero warnings (!1973)
* Consider from address to be no-reply@ (!1956)
* Make it easier to customize set of Nextcloud apps (!1952). By dividing `apps` value to `default` and `custom`. This implies that if you have an override to Nextcloud helm values in Stackspin, you need to adapt to this new structure. You can see an example in `values-local.yaml.example` of the Nextcloud helm chart.
### Updates
* update helm release nextcloud-onlyoffice to v0.15.14 (!2020)
* update helm release hedgedoc to v0.1.3 (!2019)
* update helm release stackspin-dashboard to v1.7.4
* update helm release kube-prometheus-stack to v46.8.0 (!2015)
* update helm release kratos to v0.33.3 (!2013)
* update helm release cert-manager to v1.12.2 (!2011)
* update helm release velero to v4.0.3 (!2001)
* update helm release metallb to v4.5.3 (!2000)
* update helm release system-upgrade-controller to v0.3.0 (!1996)
* update helm release ingress-nginx to v4.7.0 (!1992)
* update dependency ansible to v7.6.0 (!1976)
* update dependency nextcloud-releases/calendar to v4.4.2 (!2010)
* update dependency nextcloud-releases/contacts to v5.3.0 (!2006)
* update dependency nextcloud/forms to v3.3.0 (!1982)
* update dependency nextcloud-releases/external to v5.2.0 (!1964)
* update dependency nextcloud/polls to v5 (!1946)
* update dependency zorn-v/nextcloud-social-login to v5.4.3 (!1933)
* update dependency nextcloud-releases/groupfolders to v14.0.2 (!1932)
### Versions
| **name** | **chart** | **app_version** |
|---------------------------|-----------|-----------------------|
| cert-manager | v1.12.2 | v1.12.2 |
| dashboard | 1.7.1 | 0.7.1 |
| hedgedoc | 0.1.3 | 1.9.8 |
| hydra | 0.25.6 | v1.11.8 |
| ingress-nginx | 4.7.0 | 1.8.0 |
| kratos | 0.33.3 | v0.13.0 |
| kube-prometheus-stack | 46.8.0 | v0.65.2 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| metallb | 4.5.3 | 0.13.10 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud | 0.15.13 | NC-26.0.2-OO-7.8.0 |
| single-sign-on-database | 11.4.6 | 10.6.12 |
| system-upgrade-controller | 0.3.0 | v0.9.1 |
| velero | 4.0.3 | 1.11.0 |
| wekan | 1.1.1 | 5.93 |
| wordpress | 0.7.30 | 6.0.1 |
| zulip | 0.4.0 | 6.0-0 |
## 2.6
### Fixes
* Resolve "Hedgedoc database not backed up" (!1920, #1532)
* Upgrade flux before any others (!1919, #1300)
### Updates
* Update dependency nextcloud-releases/groupfolders to v14.0.1 (!1922)
### Versions
| **name** | **chart** | **app_version** |
|---------------------------|-----------|-----------------------|
| cert-manager | v1.11.0 | v1.11.0 |
| dashboard | 1.6.5 | 0.6.5 |
| hedgedoc | 0.1.2 | 1.9.7 |
| hydra | 0.25.6 | v1.11.8 |
| ingress-nginx | 4.6.0 | 1.7.0 |
| kratos | 0.30.0 | v0.11.1 |
| kube-prometheus-stack | 45.9.1 | 0.63.0 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| metallb | 4.2.0 | 0.13.9 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud | 0.15.12 | NC-25.0.4-OO-7.2.2.56 |
| single-sign-on-database | 11.4.6 | 10.6.12 |
| system-upgrade-controller | 0.2.0 | v0.9.1 |
| velero | 2.32.6 | 1.9.4 |
| wekan | 1.1.1 | 5.93 |
| wordpress | 0.7.30 | 6.0.1 |
| zulip | 0.4.0 | 6.0-0 |
## 2.5
### Fixes
* Resolve "Hedgedoc - pasting content breaks app" (!1916, #1530)
### Features
* Display cluster name in alertmanager subject (!1910, #831)
### Updates
* Update dependency ansible to v7.4.0 (!1903)
* Update dependency gino0631/nextcloud-metadata to v0.18.0 (!1904)
* Update dependency nextcloud-releases/calendar to v4.3.2 (!1914)
* Update helm release ingress-nginx to v4.6.0 (!1907)
* Update helm release kratos to v0.30.0 (!1908)
* Update helm release kube-prometheus-stack to v45 (!1912)
* Update helm release metallb to v4.2.0 (!1911)
* Update flux to 0.41.2 (!1898, #1524)
### Versions
| **name** | **chart** | **app_version** |
|---------------------------|-----------|-----------------------|
| cert-manager | v1.11.0 | v1.11.0 |
| dashboard | 1.6.5 | 0.6.5 |
| hedgedoc | 0.1.1 | 1.9.7 |
| hydra | 0.25.6 | v1.11.8 |
| ingress-nginx | 4.6.0 | 1.7.0 |
| kratos | 0.30.0 | v0.11.1 |
| kube-prometheus-stack | 45.9.1 | 0.63.0 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| metallb | 4.2.0 | 0.13.9 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud | 0.15.12 | NC-25.0.4-OO-7.2.2.56 |
| single-sign-on-database | 11.4.6 | 10.6.12 |
| system-upgrade-controller | 0.2.0 | v0.9.1 |
| velero | 2.32.6 | 1.9.4 |
| wekan | 1.1.1 | 5.93 |
| wordpress | 0.7.30 | 6.0.1 |
| zulip | 0.4.0 | 6.0-0 |
## 2.4
### Apps
* Add Hedgedoc (!1728)
### Fixes
* Do not allow changing SSO-provided display name in Nextcloud (!1263)
* Reduce memory requirements for flux (!1838)
* Adjust memory request and limit for prometheus (!1845)
* Fix deletion of apps via the CLI.
* Only show installed apps when configuring roles.
* Disable changing email address in settings, which is not supported.
### Features
* Initial support for 2FA (!1815)
* Add button to dashboard for creating recovery links to send via method other
than email.
* Automatically log in to dashboard if already authenticated.
* Redirect to the dashboard on login if no redirect is set.
* Show the user UUID on the dashboard user modal.
* Add special features (sign-up form) for a demo instance.
### Updates
* Update helm release system-upgrade-controller to v0.2.1 (!1895)
* Update helm release ingress-nginx to v4.5.2 (!1846)
* Update helm release metallb to v4.1.21 (!1885)
* Update helm release stackspin-dashboard to v1.6.5 (!1897)
* Update helm release kratos to v0.28.2 (!1876)
* Update nextcloud onlyoffice app to v7.8.0 (!1888)
* Update helm release nextcloud-onlyoffice to v0.15.12 (!1882)
* Update helm release mariadb to v11.4.6 (!1837)
* Update dependency nextcloud-releases/calendar to v4.3.1 (!1887)
* Update dependency nextcloud-releases/groupfolders to v14 (!1883)
* Update dependency nextcloud-releases/contacts to v5.2.0 (!1880)
* Update dependency zorn-v/nextcloud-social-login to v5.4.2 (!1875)
* Update dependency nextcloud-releases/external to v5.1.0 (!1867)
* Update dependency nextcloud/forms to v3.2.0 (!1864)
* Update dependency nextcloud/polls to v4.1.8 (!1863)
### Versions
| **name** | **chart** | **app_version** |
|---------------------------|-----------|-----------------------|
| cert-manager | v1.11.0 | v1.11.0 |
| dashboard | 1.6.5 | 0.6.5 |
| hedgedoc | 0.1.0 | 1.9.7 |
| hydra | 0.25.6 | v1.11.8 |
| ingress-nginx | 4.5.2 | 1.6.4 |
| kratos | 0.28.2 | v0.11.1 |
| kube-prometheus-stack | 43.3.1 | 0.61.1 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| metallb | 4.1.21 | 0.13.9 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud | 0.15.12 | NC-25.0.4-OO-7.2.2.56 |
| single-sign-on-database | 11.4.6 | 10.6.12 |
| system-upgrade-controller | 0.2.0 | v0.9.1 |
| velero | 2.32.6 | 1.9.4 |
| wekan | 1.1.1 | 5.93 |
| wordpress | 0.7.30 | 6.0.1 |
| zulip | 0.4.0 | 6.0-0 |
## 2.3
### Fixes
* Fix the broken account recovery page (!1831)
### Updates
* update dependency nextcloud-releases/calendar to v4.2.2 (!1828)
* Update flux to 0.38.3 (!1829)
### Versions
| **name** | **chart** | **app_version** |
|---------------------------|-----------|-----------------|
| cert-manager | v1.11.0 | v1.11.0 |
| dashboard | 1.6.0 | 0.6.0 |
| hydra | 0.25.6 | v1.11.8 |
| ingress-nginx | 4.4.2 | 1.5.1 |
| kratos | 0.27.0 | v0.11.1 |
| kube-prometheus-stack | 43.3.1 | 0.61.1 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| metallb | 4.1.14 | 0.13.7 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| single-sign-on-database | 11.4.4 | 10.6.11 |
| system-upgrade-controller | 0.2.0 | v0.9.1 |
| wekan | 1.1.1 | 5.93 |
| wordpress | 0.7.30 | 6.0.1 |
| zulip | 0.4.0 | 6.0-0 |
## 2.2
### Features
* Make it easier to add an app (!1819)
* Pin and renovate Nextcloud apps (!1727)
### Documentation
* Error during upgrading: upgrade failed: another operation (install/upgrade/rollback) is in progress (!1767)
### Updates
* update dependency nextcloud-releases/contacts to v5.0.3 (!1823)
* update helm release nextcloud-onlyoffice to v0.15.9 (!1822)
* update dependency zorn-v/nextcloud-social-login to v5.3.0 (!1821)
* update dependency nextcloud/polls to v4.1.2 (!1818)
* update helm release kube-prometheus-stack to v43.3.1 (!1814)
* update helm release kratos to v0.27.0 (!1813)
* update helm release metallb to v4.1.14 (!1809)
* update helm release mariadb to v11.4.4 (!1808)
* update helm release ingress-nginx to v4.4.2 (!1803)
* update dependency nextcloud/forms to v3.0.3 (!1802)
* update helm release cert-manager to v1.11.0 (!1800)
* update dependency nextcloud-releases/calendar to v4.2.1 (!1795)
* update helm release velero to v2.32.6 (!1791)
* Upgrade Zulip to 6.0 (!1752)
### Versions
| **name** | **chart** | **app_version** |
|---------------------------|-----------|-----------------|
| cert-manager | v1.11.0 | v1.11.0 |
| dashboard | 1.6.0 | 0.5.2 |
| hedgedoc | 1.1.0 | 1.9.0 |
| hydra | 0.25.6 | v1.11.8 |
| ingress-nginx | 4.4.2 | 1.5.1 |
| kratos | 0.27.0 | v0.11.1 |
| kube-prometheus-stack | 43.3.1 | 0.61.1 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| metallb | 4.1.14 | 0.13.7 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| single-sign-on-database | 11.4.4 | 10.6.11 |
| system-upgrade-controller | 0.2.0 | v0.9.1 |
| wekan | 1.1.1 | 5.93 |
| wordpress | 0.7.30 | 6.0.1 |
| zulip | 0.4.0 | 6.0-0 |
## 2.1
### Features
* Alert on stale backups (!1599)
* Auto-update k3s with flux and renovate (!1569)
* Replace install-app.sh script with dashboard-backend install command (!1546)
* Set an upgrade window for production clusters (!1517)
### Bugfixes
* Wordpress OOM (!1601)
### Documentation
* Document advanced DNS config (!1589)
* Add admin user documentation about Stackspin limitations (!1573)
* Increase disk space recommendations (!1559)
* Documentation improvements (!1533)
### Updates
* Update Helm release stackspin-dashboard to v1.5.2 (!1657)
* Update Helm release kube-prometheus-stack to v41.6.1 (!1585)
* Update Helm release metallb to v4.1.10 (!1586)
* Upgrade to flux 0.36.0 (!1579)
* Update Helm release nextcloud-onlyoffice to v0.14.1 (!1580)
* Update Helm release wordpress to v0.7.30 (!1575)
* Update Helm release cert-manager to v1.10.0 (!1565)
* Update Helm release velero to v2.32.1 (!1561)
* Update Helm release zulip to v0.4.0 (!1556)
* Update Helm release kratos to v0.25.6 (!1552)
* Update Helm release hydra to v0.25.6 (!1551)
### Versions
| **name** | **chart** | **app_version** |
|---------------------------|-----------|------------------------|
| cert-manager | v1.10.0 | v1.10.0 |
| dashboard | 1.5.2 | 0.5.2 |
| hydra | 0.25.6 | v1.11.8 |
| ingress-nginx | 4.3.0 | 1.4.0 |
| kratos | 0.25.6 | v0.10.1 |
| kube-prometheus-stack | 41.6.1 | 0.60.1 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| metallb | 4.1.10 | 0.13.7 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud-onlyoffice | 0.15.2 | NC-24.0.5-OO-7.2.0.204 |
| single-sign-on-database | 11.3.3 | 10.6.10 |
| system-upgrade-controller | 0.1.0 | v0.8.1 |
| velero | 2.32.1 | 1.9.2 |
| wekan | 1.1.1 | 5.93 |
| wordpress | 0.7.30 | 6.0.1 |
| zulip | 0.4.0 | 5.3-0 |
## 2.0
### Breaking / relevant to major release
* **New versioning scheme**
We stepped away from semver because the patch version number does not make
sense in the Stackspin context. We have two release types:
Minor release, updates `y` in a version number formatted `x.y`. These releases
occur regularly (we aim for every 2 weeks). Clusters should auto-update when
we do minor releases
Major release, updates `x` in a version number formatted `x.y`. These releases
indicate that you need to manually run an upgrade script, because
auto-upgrade is not possible. We aim for a major release around every 6
months, but we could do one earlier if it includes an important security or
other big bug fix that we cannot apply through auto-updates.
* Add Nextcloud redis password in upgrade script
* Perform Zulip postgresql upgrade in upgrade script
* Removed Loki, Eventrouter and Promtail (!1500, !1509)
### Features
* Resolve "Allow adjusting the subdomains applications run on" (!1270)
* Enable Redis caching for Nextcloud (!1407)
### Bugfixes
* Fix random webhook failures (!1483)
* Fix "Cannot log in after logging out" (!1476)
* Resolve "Could not find a tag or branch when installing oas-behave" (!260)
* Resolve "Creating files and folders in nextcloud gives errors" (!1473)
* Resolve "Fix flux branch in 1.0 upgrade script" (!1503)
* Resolve "install-stackspin.sh fails with `python: command not found`" (!1518)
* Resolve "`kubectl exec` fails" (!1469)
* Resolve "Use same velero backup label for dashboard and single sign on" (!1495)
* Resolve "Velero: No volume named wordpress-wp-uploads" (!1508)
### Documentation
* Resolve "Improve velero restore docs for PVCs that are shared between helm releases" (!1499)
* Resolve "Update gitlab issue release template" (!1458)
### Updates
* Update Helm release nextcloud-onlyoffice to v0.15.4 (!1705)
* Refresh pip-compile outputs (!1430, !1451, !1463)
* Resolve "Upgrade dashboard helm chart to version 1.4.0" (!1524)
* Resolve "Upgrade to k3s v1.25.2+k3s1" (!1505)
* Resolve "Upgrade Zulip helm chart to 0.3.0" (!1156)
* Update dependency alpine_3_16/chromium to v102.0.5005.182-r0 (!1498)
* Update dependency alpine_3_16/curl to v7.83.1-r3 (!1420)
* Update dependency alpine_3_16/yq to v4.25.1-r4 (!1444)
* Update dependency ansible to v6.4.0 (!1456)
* Update dependency psutil to v5.9.2 (!1428)
* Update dependency pyopenssl to v22.1.0 (!1491)
* Update dependency sphinx to v5.2.3 (!1514)
* Update Helm release hydra to v0.25.4 (!1484)
* Update Helm release ingress-nginx to v4.3.0 (!1515)
* Update Helm release kratos to v0.25.3 (!1446)
* Update Helm release kratos to v0.25.4 (!1485)
* Update Helm release kube-prometheus-stack to v40.3.1 (!1516)
* Update Helm release mariadb to v11.3.2 (!1510)
* Update Helm release metallb to v4.1.5 (!1520)
* Update Helm release promtail to v6.4.0 (!1450)
* Update Helm release velero to v2.31.8 (!1493)
* Update Helm release wordpress to v0.7.26 (!1521)
* Update pre-commit hook zricethezav/gitleaks to v8.13.0 (!1480)
* Update to flux 0.34.0 (!1468)
### Features and Fixes relevant for Stackspin developers
* Enable CI upgrade test on demand (!1502)
* Resolve "Allow install-stackspin to get retried" (!1435)
* Resolve "Failed nextcloud install in pipeline" (!1464)
* Resolve "Tune probes for nextcloud" (!1507)
* Use same version for curl + curl_dev (!1434)
### Versions
| name | chart | app_version |
|---------------------------|--------|------------------------|
| cert-manager | v1.9.1 | v1.9.1 |
| dashboard | 1.4.0 | 0.4.0 |
| hydra | 0.25.4 | v1.11.8 |
| ingress-nginx | 4.3.0 | 1.4.0 |
| kratos | 0.25.4 | v0.10.1 |
| kube-prometheus-stack | 40.3.1 | 0.59.2 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| metallb | 4.1.5 | 0.13.5 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud | 0.15.4 | NC-24.0.5-OO-7.2.0.204 |
| single-sign-on-database | 11.3.2 | 10.6.10 |
| wekan | 1.1.1 | 5.93 |
| wordpress | 0.7.26 | 6.0.1 |
| zulip | 0.3.0 | 5.3-0 |
## 1.0.0
### Features
* Allow adjusting the subdomains applications run on (!1270)
### Bug fixes
* Fix NC warnings in settings (!1313)
* Automigrate jobs are spawned too fast (!1386)
### Documentation
* Make documentation production ready (!1346)
### Updates
* update k3s to v1.24.4+k3s1 (!1439)
* update flux to 0.33.0 (!1438)
* update helm release wordpress to v0.7.18 (!1432)
* update helm release hydra to v0.25.2 (!1416)
* update helm release metallb to v4.1.2 (!1427)
* update helm release ingress-nginx to v4.2.5 (!1431)
* update helm release kube-prometheus-stack to v39.11.0 (!1424)
* update helm release mariadb to v11.2.2 (!1422)
* update helm release loki to v2.16.0 (!1417)
* update helm release nextcloud-onlyoffice to v0.10.25 (!1410)
* update helm release stackspin-dashboard to v1.2.1 (!1397)
* update dependency ansible to v6.3.0 (!1402)
* update helm release promtail to v6.3.0 (!1389)
* update helm release velero to v2.31.3 (!1390)
* update helm release kratos to v0.24.5 (!1336)
### Current application versions:
| name | chart | app_version |
|---------------------------|-----------|-----------------------|
| cert-manager | v1.9.1 | v1.9.1 |
| dashboard | 1.2.2 | 0.2.8 |
| eventrouter | 0.4.0 | 0.3 |
| hydra | 0.25.1 | v1.11.8 |
| ingress-nginx | 4.2.3 | 1.3.0 |
| kratos | 0.24.5 | v0.10.1 |
| kube-prometheus-stack | 39.11.0 | 0.58.0 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| loki | 2.16.0 | v2.6.1 |
| metallb | 4.1.1 | 0.13.4 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud & onlyoffice | 0.10.25 | NC-24.0.2-OO-7.1.1.23 |
| promtail | 6.3.0 | 2.6.1 |
| single-sign-on-database | 11.2.2 | 10.6.9 |
| velero | 2.31.3 | 1.9.1 |
| wekan | 1.1.1 | 5.93 |
| wordpress | 0.7.16 | 6.0.1 |
| zulip | 0.2.1 | 5.3-0 |
## 0.8.5
### Features
* Override Kratos' default account recovery email (!1293)
* Migrate grafana taiko tests to cypress (!1300)
* Batch user import in the Dashboard
* A new "User" in the dashboard now gets "User" roles for all apps by default
### Known issues
* Although it is now possible to host applications on custom URLs, the
dashboard's home page still points to the default URLs.
### Bug fixes
* Add wekan and zulip to stackspin secrets cli subcmd (!1311)
* Headers / reverse proxy settings do not read the IP address (in at least Zulip) (!1299)
* Investigate why Wekan test fails some times for main CI pipeline (!1287)
### Documentation
* docs: Move relevant SSO documentation to stackspin/stackspin (!1294)
### Updates
* chore(deps): update dependency alpine_3_16/chromium to v102.0.5005.167-r0 (!1309)
* chore(deps): update dependency alpine_3_16/yq to v4.25.1-r2 (!1304)
* chore(deps): update dependency sphinx to v5.1.1 (!1317)
* chore(deps): update helm release cert-manager to v1.9.0 (!1306)
* chore(deps): update helm release cert-manager to v1.9.1 (!1315)
* chore(deps): update helm release kube-prometheus-stack to v38.0.3 (!1322)
* chore(deps): update helm release kube-prometheus-stack to v38 (!1307)
* chore(deps): update helm release loki to v2.13.3 (!1318)
* chore(deps): update helm release mariadb to v11.1.1 (!1316)
* chore(deps): update helm release nextcloud-onlyoffice to v0.10.4 (!1308)
* chore(deps): update helm release nextcloud-onlyoffice to v0.10.6 (!1319)
* chore(deps): update helm release promtail to v6.2.2 (!1297)
* chore(deps): update helm release stackspin-dashboard to v1.2.0 (!1323)
* chore(deps): update helm release wordpress to v0.7.2 (!1320)
* chore(deps): update pre-commit hook zricethezav/gitleaks to v8.9.0 (!1321)
### Current application versions:
| name | chart | app_version |
|---------------------------|--------|-----------------------|
| cert-manager | v1.9.1 | v1.9.1 |
| dashboard | 1.2.0 | 0.2.8 |
| eventrouter | 0.4.0 | 0.3 |
| hydra | 0.23.3 | v1.10.5 |
| ingress-nginx | 4.2.0 | 1.3.0 |
| kratos | 0.23.3 | v0.9.0-alpha.2 |
| kube-prometheus-stack | 38.0.3 | 0.57.0 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| loki | 2.13.3 | v2.6.1 |
| metallb | 3.0.11 | 0.12.1 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud & onlyoffice | 0.10.6 | NC-23.0.3-OO-7.1.1.23 |
| promtail | 6.2.2 | 2.6.1 |
| single-sign-on-database | 11.1.1 | 10.6.8 |
| velero | 2.30.1 | 1.9.0 |
| wekan | 1.1.1 | 5.93 |
| wordpress | 0.7.2 | 6.0.1 |
| zulip | 0.2.1 | 5.3-0 |
## 0.8.4
### Features
* Add some nextcloud apps (!1258) (thank you @xeruf! :fire:)
* Alert on low node memory (!1275)
* Allow adjusting the subdomains applications run on (!1270)
- Known issue: this does not change the URLs in the dashboard yet (dashboard#75)
* Configure SSO-specific nextcloud settings (!1263)
* SSO updates now trigger all apps in CI pipeline (!1256)
* Logging out of the Dashboard ends your SSO session
* User rights can be set from the dashboard.
### Bug fixes
* Grafana does not use the sso email as username anymore (!1281)
* Can't upload (big?) image in Zulip (!1246)
* Grafana metrics are publicly exposed (!1255)
* Ignore `HostHighLoad` alert in CI (!1285)
* Ignore KubePersistentVolumeFillingUp in CI (!1273)
* Nextcloud OOM killed (!1245)
* Protect the dashboard against forged post requests to /users endpoint
### Documentation
* Clean up unneeded docs, fix LogQL queries (!1198)
* Document how to add a new app to Stackspin (!1236)
* Update CI image of current release branch after each release (!1274)
### Updates
* Upgrade flux (!1257)
* Update dependency alpine\_3\_16/chromium to v102.0.5005.158-r0 (!1280)
* Update dependency alpine\_3\_16/curl to v7.83.1-r2 (!1243)
* Update dependency ansible to v6.1.0 (!1277)
* Update helm release Dashboard to 1.1.0 (!1283)
* Update helm release kube-prometheus-stack to v37.2.0 (!1282)
* Update Helm release ingress-nginx to v4.2.0 (!1276)
* Update Helm release kube-prometheus-stack to v37 (!1279)
* Update Helm release loki to v2.13.1 (!1268)
* Update Helm release mariadb to v11.0.14 (!1247)
* Update Helm release metallb to v3.0.10 (!1249)
* Update Helm release nextcloud-onlyoffice to v0.10.1 (!1266)
* Update Helm release promtail to v6.2.1 (!1278)
* Update Helm release wordpress to v0.6.53 (!1254)
* Update pre-commit hook zricethezav/gitleaks to v8.8.12 (!1272)
Current application versions:
| Name | Chart | App Version |
| :------------------------ | :------ | :-------------------- |
| cert-manager | v1.8.2 | v1.8.2 |
| dashboard | 1.1.0 | 0.2.6 |
| eventrouter | 0.4.0 | 0.3 |
| hydra | 0.23.3 | v1.10.5 |
| ingress-nginx | 4.2.0 | 1.3.0 |
| kratos | 0.23.3 | v0.9.0-alpha.2 |
| kube-prometheus-stack | 37.2.0 | 0.57.0 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| loki | 2.13.1 | v2.6.0 |
| metallb | 3.0.10 | 0.12.1 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud & onlyoffice | 0.10.1 | NC-23.0.3-OO-7.0.1.37 |
| promtail | 6.2.1 | 2.6.0 |
| single-sign-on-database | 11.0.14 | 10.6.8 |
| velero | 2.30.1 | 1.9.0 |
| wekan | 1.1.1 | 5.93 |
| wordpress | 0.6.53 | 5.9.3 |
| zulip | 0.2.1 | 5.3-0 |
## 0.8.3
### Bug fixes
* Disable NC helm test (!1201, !1224)
* Have to login almost daily into Nextcloud (!1192)
### Features
* Monitor backup software (!1052)
* Monitor ingress logs for http errors (i.e. 499) (!1166)
* Tune new nginx prometheus alerts (!1203)
* Disable nginx alerts (!1227)
### Updates
* Update Helm release cert-manager to v1.8.2 (!1228)
* Update Helm release hydra to v0.23.3 (!1229)
* Update Helm release kratos to v0.23.3 (!1230)
* Update Helm release stackspin-dashboard to v1.0.5 (!1238)
* Update Helm release velero to v2.30.1 (!1239)
* Update Helm release wordpress to v0.6.51 (!1233)
* Update Zulip to 5.3 (!1222)
* Update dependency ansible to v6 (!1217)
* Update dependency kubernetes to v24 (!1225)
* Update dependency pytest-testinfra to v6.8.0 (!1206)
* Update dependency sphinx to v5 (!1218)
* Update helm release kube-prometheus-stack to v36.0.3 (!1209)
* Update helm release loki to v2.12.2 (!1205)
* Update helm release nextcloud-onlyoffice to v0.9.13 (!1197)
* Update helm release promtail to v6 (!1211)
* Update helm release wordpress to v0.6.49 (!1196)
* Update kubectl to 1.24.2-r1 (!1220)
* Update pre-commit hook zricethezav/gitleaks to v8.8.11 (!1240)
* Update release template (!1221)
Current application versions:
| Name | Chart | App version |
| :------------------------ | :------ | :-------------------- |
| cert-manager | v1.8.2 | v1.8.2 |
| dashboard | 1.0.5 | 0.2.6 |
| eventrouter | 0.4.0 | 0.3 |
| hydra | 0.23.3 | v1.10.5 |
| ingress-nginx | 4.1.4 | 1.2.1 |
| kratos | 0.23.3 | v0.9.0-alpha.2 |
| kube-prometheus-stack | 36.2.0 | 0.57.0 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| loki | 2.12.2 | v2.5.0 |
| metallb | 3.0.9 | 0.12.1 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nc | 0.9.13 | NC-23.0.3-OO-7.0.1.37 |
| promtail | 6.0.0 | 2.5.0 |
| single-sign-on-database | 11.0.13 | 10.6.8 |
| velero | 2.30.0 | 1.8.1 |
| wekan | 1.1.1 | 5.93 |
| wordpress | 0.6.51 | 5.9.3 |
| zulip | 0.2.1 | 4.7-0 |
## 0.8.2
### Features
* Enable metallb prometheus monitoring (!1151)
* Monitor ingress logs for http errors (i.e. 499) (!1166)
* Monitor backup software (!1052)
* set logout endpoints for WordPress, Nextcloud and Grafana (!1082)
* Increase disk size of droplet in upgrade tests (!1088)
* test Nextcloud with helm test (!1014)
* Remove single-sign-on release and upgrade Dashboard to a version that includes the login code (!946)
* Let flux auto-upgrade itself (!964)
* fine-tune container memory requests (!927)
* use upstream zulip-postgres:14 image (!996)
* Add missing-container-metrics (!951)
### Bug fixes
* Grafana dashboard fails to install (!1152)
* Increase health check timeouts (!1106)
* v0.8: Increase disk size of droplet in upgrade tests (!1091)
* resolve "clusters/main/.cluster.env: No such file or directory" (!1086)
* Hydra and Kratos serviceMonitors are failing (!1048)
* additional nextcloud configs (!1042)
* Simplify redundant outgoing_mail config (!983)
* Auto-upgrade of kube-prometheus-stack with flux often fails (!982)
### Upgrades
* update helm release kube-prometheus-stack to v36 (!1185)
* update helm release loki to v2.12.1 (!1190)
* update helm release nextcloud-onlyoffice to v0.9.12 (!1186)
* update helm release wordpress to v0.6.48 (!1184)
* update helm release metallb to v3.0.9 (!1182)
* update helm release mariadb to v11.0.13 (!1181)
* update helm release ingress-nginx to v4.1.4 (!1179)
* update helm release velero to v2.29.8 (!1178)
* Update Zulip image to 5.2-0 (!1105)
* update helm release missing-container-metrics to v0.25.0 (!1168)
* update dependency ansible to v5.9.0 (!1167)
* update helm release promtail to v5.1.0 (!1164)
* Upgrade to Flux v0.30.2 (!1155)
* update dependency alpine_edge/kubectl to v1.24.0-r3 (!1153)
* update dependency alpine to v3.16 (!1132)
* Upgrade dashboard to chart version 0.2.3 (!1124)
* update dependency paramiko to v2.11.0 (!1099)
* update dependency alpine_3_15/yq to v4.14.1-r2 (!1083)
* update dependency kubernetes to v23.6.0 (!1061)
* update helm release kratos to v0.23.2 (!1058)
* update helm release hydra to v0.23.2 (!1057)
* update alpine packages to v7.80.0-r1 (!1030)
* Update dependency pytest-testinfra to v6.7.0 (!981)
Current application versions:
| Name | Chart | App version |
|---------------------------|---------|-----------------------|
| cert-manager | v1.8.0 | v1.8.0 |
| dashboard | 0.2.3 | 0.2.3 |
| eventrouter | 0.4.0 | 0.3 |
| hydra | 0.23.2 | v1.10.5 |
| ingress-nginx | 4.1.4 | 1.2.1 |
| kratos | 0.23.2 | v0.9.0-alpha.2 |
| kube-prometheus-stack | 35.5.3 | 0.56.3 |
| local-path-provisioner | 0.0.22 | v0.0.22-gh1 |
| loki | 2.12.0 | v2.5.0 |
| metallb | 3.0.9 | 0.12.1 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nc | 0.8.15 | NC-23.0.3-OO-7.0.1.37 |
| promtail | 5.1.0 | 2.5.0 |
| single-sign-on-database | 11.0.13 | 10.6.8 |
| velero | 2.29.8 | 1.8.1 |
| wekan | 1.1.1 | 5.93 |
| wordpress | 0.6.48 | 5.9.3 |
| zulip | 0.2.1 | 5.2-0 |
### Documentation
* make copy-pasting installation docs easier (!1137)
* document RAM and CPU requirements for each app (!1123)
* Document how to run `occ` nextcloud CLI commands (!1100)
* Add instructions for Importing Nextcloud, Zulip, Wekan and Wordpress (!1090)
### Known issues
* During development of 0.8.2, we sometimes encountered issues with the
automatic upgrade process, specifically
- https://open.greenhost.net/stackspin/stackspin/-/issues/1228
- https://open.greenhost.net/stackspin/stackspin/-/issues/1300
In both cases, the solution was to `flux resume hr -n $NAMESPACE $HELM_RELEASE`
the helm release that was stuck or failing.
## 0.8.1
### Changed
* Accept python3 and fix nftables.conf for IPv6 (!916)
* Documentation amendments and adjustments (!903, !890, !870)
* Enable "Merge when pipeline succeeds" for renovate patch PRs (!888)
* Fix: "Clusters out of CPU" (!934)
* Include default upstream renovate presets as well (!891)
* Remove all cpu limits (!843)
* Alert on out of memory situations (!612)
* Allow nextcloud plugins bruteforcesettings files_markdown groupfolders (!639)
* Automate per-release upgrade testing (!684)
* Do not fire ContainerMemoryUsage alert for unlabelled metrics (!865)
* Nextcloud cronjobs are not running (!912)
* PVC deletion in restore instructions (!905)
* Renovate: Auto-merge patch releases (!880)
* Update kubectl in container image (!898)
* Update taiko (!878)
* Upgrade zulip to 5.0rc1 (!900)
* Update Helm release cert-manager to v1.7.2 (!911)
* Update Helm release ingress-nginx to v4.0.19 (!937)
* Update Helm release kube-prometheus-stack to v34.8.0 (!941)
* Update Helm release local-path-provisioner to v0.0.22 (!932)
* Update Helm release loki to v2.10.3 (!947)
* Update Helm release mariadb to v10.4.5 (!938)
* Update Helm release metallb to v2.6.11 (!939)
* Update Helm release nextcloud-onlyoffice to v0.7.23 (!948)
* Update Helm release single-sign-on to v0.8.2 (!879)
* Update Helm release velero to v2.29.4 (!922)
* Update Helm release wordpress to v0.6.17 (!940)
* Update dependency ansible to v5.5.0 (!883)
* Update dependency dnspython to v2.2.1 (!863)
* Update dependency kubernetes to v23 (!859)
* Update dependency paramiko to v2.10.3 (!896)
* Update dependency sphinx to v4.5.0 (!920)
* Validate passwords in .flux.env (!931)
Current application versions:
| Name | Chart | App version |
| ------------------------- | ------ | --------------------- |
| cert-manager | v1.7.2 | v1.7.2 |
| dashboard | 0.1.7 | 0.1.6 |
| eventrouter | 0.4.0 | 0.3 |
| ingress-nginx | 4.0.19 | 1.1.3 |
| kube-prometheus-stack | 34.8.0 | 0.55.0 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| loki | 2.10.3 | v2.4.2 |
| metallb | 2.6.11 | 0.12.1 |
| missing-container-metrics | 0.24.0 | 0.24.0 |
| nc | 0.7.23 | NC-23.0.2-OO-7.0.1.37 |
| promtail | 3.11.0 | 2.4.2 |
| single-sign-on | 0.8.2 | |
| single-sign-on-database | 10.4.5 | 10.5.15 |
| velero | 2.29.4 | 1.8.1 |
| wekan | 1.1.1 | 5.93 |
| wordpress | 0.6.17 | 5.9.1 |
| zulip | 0.2.1 | 4.7-0 |
### Removed
* Remove unneeded Makefile (!882)
## 0.8
### Changed
* Improve documentation based on beta feedback (!860)
* Make sure that re-running Ansible does not break kubeconfig (!861)
### Added
* Document frequent alertmanager emails (!864)
* Add memory limits for Zulip redis and rabbitmq pods (!853)
* Add Stackspin theme for documentation (!857)
## 0.8beta1
### Removed
* Rocket.Chat in favor of Zulip (!656)
* Remove coredns from system resolvers (!510)
### Changed
* Rename OpenAppStack to Stackspin (!596)
* Tuned application CPU limits (!839)
* Switch ONLYOFFICE database to MariaDB (!835)
* Switch SSO database to MariaDB (!834)
* Make Wekans MongoDB use a statefulset (!813)
* Use helmRepository instead of gitRepository for Zulip (!764)
* Improve installation documentation, make SMTP settings mandatory (!751)
* Increase alertmanager email interval (!714)
* Restructure and improve CI pipeline (!663)
* Increase grafana retention time (!409)
* Use helmRepository instead of gitRepository for single-sign-on (!705)
* Update secrets with new values (!693)
* Use helm package registry to publish our helm charts (!672)
* Use SSL.com instead of ZeroSSL (!689)
* Improve logging docs (!667)
* Revisit flux dependencies (!653)
* Reduce kustomization healthCheck timeouts (!652)
* Use production certs in CI (!640)
* Install helm and kubectl with snap (!613)
* Increase application mem limits (!606, !589, !587)
* Map wordpress admin role with SSO admin login (!559)
* Close nodeports using nft inet (!552)
* Migrate eventrouter chart repo from helm stable to bitnami (!509)
* Upgrade to debian bullseye (!488)
* Configure SMTP and disable 2fa for new users (!493)
* Make all monitoring optional (!491)
Updated applications:
| Name | Chart | App version |
| ----------------------- | ------ | ---------------------- |
| cert-manager | v1.7.1 | v1.7.1 |
| dashboard | 0.1.7 | 0.1.6 |
| eventrouter | 0.4.0 | 0.3 |
| ingress-nginx | 4.0.17 | 1.1.1 |
| kube-prometheus-stack | 33.0.0 | 0.54.1 |
| local-path-provisioner | 0.0.21 | v0.0.21 |
| loki | 2.10.1 | v2.4.2 |
| metallb | 2.6.2 | 0.11.0 |
| nc | 0.7.4 | NC-22.2.3-OO-7.0.1.37 |
| promtail | 3.11.0 | 2.4.2 |
| single-sign-on | 0.8.0 | |
| single-sign-on-database | 10.3.7 | 10.5.15 |
| velero | 2.27.4 | 1.7.1 |
| wekan | 1.1.1 | 5.93 |
| wordpress | 0.6.3 | 5.9.1 |
| zulip | 0.2.1 | 4.7-0 |
Also:
* Update k3s to v1.23.1+k3s2 (!760)
* Update flux to 0.25.3 (!634)
### Added
* [Zulip](https://zulip.com) as a replacement for [Rocket.Chat](https://rocket.chat)
(!594)
* [Velero](https://velero.io) as backup solution
* Restore instructions (!463)
* Add NC calendar app (!467)
* A complete rewrite of the [Stackspin Dashboard](https://open.greenhost.net/stackspin/dashboard)
is replacing the old user- and admin panel
* Documentation for Dashboard (!808)
* Add helm package CI template (!678)
* Use renovate to update Dockerfile and helmReleases (!516)
* Add documentation about the available APIs (!574)
* Alert on high load (!798)
* Configure wordpress to send out email notifications (!560)
* Document wordpress login limitations (!716)
* Enable and use Hydra Maester (!700)
* Testing single sign on in CI (!686)
* Ensure upgrading CRDs with flux (!658)
* Add container name to ContainerMemoryUsage alert (!657)
* Allow nextcloud plugins bruteforcesettings files_markdown groupfolders (!639)
* Alert on out of memory situations (!612)
* Add upgrade instructions and scripts for upgrading OAS clusters to Stackspin
clusters (!614)
* Monitor flux components (!583)
* Add basic ingress test (!546)
* Use kustomization health-assessments (!511)
## [0.7.0] - 2021-08-19 ## 0.7.0 - 2021-08-19
### Removed ### Removed
...@@ -49,7 +1739,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ...@@ -49,7 +1739,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* Helm chart values overrdides (#873) * Helm chart values overrdides (#873)
* Prometheus CI test (#766) * Prometheus CI test (#766)
## [0.6.0] - 2021-07-14 ## 0.6.0 - 2021-07-14
### Added ### Added
...@@ -103,7 +1793,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ...@@ -103,7 +1793,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* remove old and now unused variables from compatibility-checks role * remove old and now unused variables from compatibility-checks role
## [0.5.0] - 2020-09-17 ## 0.5.0 - 2020-09-17
* Added automatic smtp config for nextcloud * Added automatic smtp config for nextcloud
* Updated k3s version to 1.18.6+k3s1 * Updated k3s version to 1.18.6+k3s1
...@@ -123,7 +1813,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ...@@ -123,7 +1813,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* Separated ansible playbooks for Kubernetes and OpenAppStack (re-)installation * Separated ansible playbooks for Kubernetes and OpenAppStack (re-)installation
* Increased installation and general stability * Increased installation and general stability
## [0.4.0] - 2020-06-11 ## 0.4.0 - 2020-06-11
* Replaced [local-storage * Replaced [local-storage
provisioner](https://open.greenhost.net/openappstack/local-storage) with provisioner](https://open.greenhost.net/openappstack/local-storage) with
...@@ -159,7 +1849,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ...@@ -159,7 +1849,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* Loki stack chart 0.36.2 * Loki stack chart 0.36.2
* Prometheus chart 11.4.0 * Prometheus chart 11.4.0
## [0.3.1] - 2020-03-18 ## 0.3.1 - 2020-03-18
* Repair rocket chat installation * Repair rocket chat installation
* Rocket Chat supports e2e encryption by default * Rocket Chat supports e2e encryption by default
...@@ -170,7 +1860,7 @@ Known issues: ...@@ -170,7 +1860,7 @@ Known issues:
* Same as 0.3.0 * Same as 0.3.0
## [0.3.0] - 2020-01-20 ## 0.3.0 - 2020-01-20
* Release candidate for beta tests * Release candidate for beta tests
* Added features: * Added features:
...@@ -206,7 +1896,7 @@ Known issues: ...@@ -206,7 +1896,7 @@ Known issues:
These issues will be addressed in the next version. These issues will be addressed in the next version.
## [0.2.1] - 2019-10-29 ## 0.2.1 - 2019-10-29
This is the release for the second user test. This is the release for the second user test.
...@@ -220,7 +1910,7 @@ Fixes: ...@@ -220,7 +1910,7 @@ Fixes:
* [Creating cluster fails if the hostname is oas-dev](https://open.greenhost.net/openappstack/openappstack/issues/346) * [Creating cluster fails if the hostname is oas-dev](https://open.greenhost.net/openappstack/openappstack/issues/346)
* [kube_config_cluster.yml should be copied to local machine earlier](https://open.greenhost.net/openappstack/openappstack/issues/319) * [kube_config_cluster.yml should be copied to local machine earlier](https://open.greenhost.net/openappstack/openappstack/issues/319)
## [0.2.0] - 2019-10-22 ## 0.2.0 - 2019-10-22
* Release candidate for second user test * Release candidate for second user test
* Added features: * Added features:
...@@ -243,11 +1933,15 @@ Known issues: ...@@ -243,11 +1933,15 @@ Known issues:
after a fresh OAS installation after a fresh OAS installation
* [Prometheus web interface is publicly available](https://open.greenhost.net/openappstack/openappstack/issues/294) * [Prometheus web interface is publicly available](https://open.greenhost.net/openappstack/openappstack/issues/294)
## [0.1.0] - 2019-07-31 ## 0.1.0 - 2019-07-31
This is the version for the alpha test. This is the version for the alpha test.
[Unreleased]: https://open.greenhost.net/openappstack/openappstack/-/compare/0.7.0...master [Unreleased]: https://open.greenhost.net/openappstack/openappstack/-/compare/0.8.3...main
[0.8.3]: https://open.greenhost.net/openappstack/openappstack/-/compare/0.8.2...0.8.3
[0.8.2]: https://open.greenhost.net/openappstack/openappstack/-/compare/0.8.1...0.8.2
[0.8.1]: https://open.greenhost.net/openappstack/openappstack/-/compare/0.8.0...0.8.1
[0.8.0]: https://open.greenhost.net/openappstack/openappstack/-/compare/0.7.0...0.8.0
[0.7.0]: https://open.greenhost.net/openappstack/openappstack/-/compare/0.6.0...0.7.0 [0.7.0]: https://open.greenhost.net/openappstack/openappstack/-/compare/0.6.0...0.7.0
[0.6.0]: https://open.greenhost.net/openappstack/openappstack/-/compare/0.5.0...0.6.0 [0.6.0]: https://open.greenhost.net/openappstack/openappstack/-/compare/0.5.0...0.6.0
[0.5.0]: https://open.greenhost.net/openappstack/openappstack/-/compare/0.4.0...0.5.0 [0.5.0]: https://open.greenhost.net/openappstack/openappstack/-/compare/0.4.0...0.5.0
......
CONTRIBUTING.md
View file @ 601a9684
# How to contribute # How to contribute
## Code of conduct Please refer to the [contributing.rst file](./docs/for_developers/contributing.rst)
or to the [Contributing Guide in Stackspin docs](https://docs.stackspin.net/en/latest/for_developers/contributing.html)
We follow the [Contributor Covenant](https://www.contributor-covenant.org) code
of conduct, please see the [CODE_OF_CONDUCT.md](./CODE_OF_CONDUCT.md) document
for details.
## Preparing the development environment
Make sure you have development dependencies installed in your development
environment.
```
pip install -r requirements-dev.txt
```
## pre-commit hooks
We use [pre-commit](https://pre-commit.com/) to maintain and install pre-commit
hooks that should be executed before each commit.
Please install these required tools on your system:
* [hadolint](https://github.com/hadolint/hadolint) for linting the `Dockerfile`
* [shellcheck](https://www.shellcheck.net/) and
* [shfmt](https://github.com/mvdan/sh) to lint and validate shell scripts
Then install pre-commit hooks:
```
pre-commit install
```
Running `git commmit` for the first time after installing the hook usually takes a
little longer because `pre-commit` pulls it's hooks from upstream repositories.
You can find all hooks in `.pre-commit-config.yaml`.
In case you need to skip the execution of the pre-commit hooks (please don't!),
use `git commit --no-verify`.
## Adding dependencies
Make sure you update our `requirements.txt` file before you push your changes.
Whenever you need a new python package, add it to requirements.in and run
`pip-compile`
to generate an new `requirements.txt` which does not only pin the new package
but also its dependencies.
If the new package you are adding is only used by developers,
please add it to the `requirements-dev.txt` file.
## CI pipeline image
We use a custom disk image for the VPSs used by the CI pipeline. On this image,
the `install-kubernetes.yaml` playbook has already been applied, which usually
saves a few minutes of pipeline running time.
### What to do when I change a part of the `install-kubernetes.yaml` playbook?
Don't worry, the playbook *runs* in the CI (just faster, because usually
nothing needs to change). So if you make changes, you can test those in the CI
without problems.
If you want to start with a clean slate, however, you might want to change
`.gitlab/ci_scripts/create_vps.sh` and temporarily remove the `--disk-image-id`
argument.
#### Before you merge, make sure your changes are applied to a new custom image:
If you changed the `install-kubernetes.yaml` playbook, for example to upgrade
the k3s version in use, you'll want to generate a new disk image template and
use it. This is a manual process for now. Follow these steps:
1. Create a new VPS
```
export HOST_NAME=baseimage$(date +'%Y%m%d')
# Make sure you use your private ssh key id, 411 is the ssh key used in CI
export SSH_ID=411
python -m stackspin ${HOST_NAME} create --create-droplet --create-hostname ${HOST_NAME}.stackspin.net --ssh-key-id $SSH_ID --create-domain-records --subdomain ${HOST_NAME} stackspin.net
```
2. Accept ssh host key
```
ssh root@${HOST_NAME}.stackspin.net
```
3. Run the following to install *only kubernetes* on the VPS:
```
python3 -m stackspin ${HOST_NAME} install
```
4. Log into your machine and clean up the k3s server, then delete the cluster
data:
```
ssh root@${HOST_NAME}.stackspin.net
# Clean up running containers and firewall
/usr/local/bin/k3s-killall.sh
# Remove k3s state
rm -rf /var/lib/rancher/k3s
```
5. Log into Cosmos with the Stackspin account
6. Go to VPS Cloud -> VPS and shut down your VPS
7. Go to VPS Cloud -> Disk Images and click `Manage` for your VPSs disk image
1. Change the Disk Label to something like `k3s-template-DATE`
2. Set VPS to `-- not assigned --`
3. Click save
4. `Manage` the same image again
5. Click 'make template'
6. Choose "Linux Latest LTS" as the kernel
7. Choose OS type Debian 11 (bullseye)
8. Remember the disk image ID that you can see in the current URL as `id=...`
9. Click save
8. Change the `--disk-image-id` argument in `.gitlab/ci_scripts/create_vps.sh`
to your current disk-image-id **with a minus in front of it**. This is
because custom images are negative integers, whereas Greenhost's disk images
are positive integers
9. Remove the droplet
You are now ready to merge the changes you made to the `install-kubernetes`
playbook
Dockerfile
View file @ 601a9684
# https://hub.docker.com/_/alpine?tab=tags # https://hub.docker.com/_/alpine?tab=tags
FROM alpine:3.15 FROM alpine:3.21
LABEL name="Stackspin management" LABEL name="Stackspin management"
LABEL version="4.5" LABEL version="4.6"
LABEL vendor1="Greenhost" LABEL vendor1="Greenhost"
ENV TAIKO_SKIP_CHROMIUM_DOWNLOAD=true # https://docs.renovatebot.com/modules/datasource/#repology-datasource
ENV TAIKO_BROWSER_PATH=/usr/bin/chromium-browser
ENV TAIKO_BROWSER_ARGS=--no-sandbox,--start-maximized,--disable-dev-shm-usage,--ignore-certificate-errors
ADD https://github.com/fluxcd/flux2/releases/download/v0.22.0/flux_0.22.0_linux_amd64.tar.gz /tmp/ # renovate: datasource=repology depName=alpine_3_20/bash
ENV BASH_VERSION="5.2.37-r0"
# renovate: datasource=repology depName=alpine_3_20/cargo
ENV CARGO_VERSION="1.83.0-r0"
# renovate: datasource=repology depName=alpine_3_20/chromium versioning=loose
ENV CHROMIUM_VERSION="134.0.6998.117-r0"
# renovate: datasource=repology depName=alpine_3_20/coreutils version=loose
ENV COREUTILS_VERSION="9.5-r2"
# renovate: datasource=repology depName=alpine_3_20/curl
ENV CURL_VERSION="8.12.1-r1"
# renovate: datasource=repology depName=alpine_3_20/expect
ENV EXPECT_VERSION="5.45.4-r5"
# renovate: datasource=repology depName=alpine_3_20/git
ENV GIT_VERSION="2.47.2-r0"
# renovate: datasource=repology depName=alpine_3_20/jq versioning=loose
ENV JQ_VERSION="1.7.1-r0"
# renovate: datasource=repology depName=alpine_edge/kubectl
ENV KUBECTL_VERSION="1.32.2-r1"
# renovate: datasource=repology depName=alpine_3_20/libffi-dev
ENV LIBFFI_DEV_VERSION="3.4.7-r0"
# renovate: datasource=repology depName=alpine_3_20/libsodium-dev
ENV LIBSODIUM_DEV_VERSION="1.0.20-r0"
# renovate: datasource=repology depName=alpine_3_20/make versioning=loose
ENV MAKE_VERSION="4.4.1-r2"
# renovate: datasource=repology depName=alpine_3_20/moreutils versioning=loose
ENV MOREUTILS_VERSION="0.69-r2"
# renovate: datasource=repology depName=alpine_3_20/npm
ENV NPM_VERSION="10.9.1-r0"
# renovate: datasource=repology depName=alpine_3_20/openssh-client-default versioning=loose
ENV OPENSSH_CLIENT_DEFAULT_VERSION="9.9_p2-r0"
# renovate: datasource=repology depName=alpine_3_20/py3-pip
ENV PY3_PIP_VERSION="24.3.1-r0"
# renovate: datasource=repology depName=alpine_3_20/python3-dev
ENV PYTHON3_DEV_VERSION="3.12.9-r0"
# renovate: datasource=repology depName=alpine_3_20/rsync
ENV RSYNC_VERSION="3.4.0-r0"
# renovate: datasource=repology depName=alpine_3_20/yq-go
ENV YQ_VERSION="4.44.5-r3"
# Makes pynacl use system SODIUM
ENV SODIUM_INSTALL=system
ADD https://github.com/fluxcd/flux2/releases/download/v2.5.1/flux_2.5.1_linux_amd64.tar.gz /tmp/
COPY ./requirements.txt /requirements.txt COPY ./requirements.txt /requirements.txt
# Ignore the hadolint error for the pip install line with a `grep` in it
# hadolint ignore=DL3013
RUN \ RUN \
# Install kubectl from alpine edge until alpine 3.16 is released # Install kubectl from alpine edge until it is included in an alpine release.
apk --no-cache add kubectl=~1.23.1-r0 \ apk --no-cache add kubectl=~$KUBECTL_VERSION \
--repository=http://dl-cdn.alpinelinux.org/alpine/edge/testing && \ --repository=http://dl-cdn.alpinelinux.org/alpine/edge/community && \
apk --no-cache add \ apk --no-cache add \
bash=~5.1.8-r0 \ bash=$BASH_VERSION \
cargo=~1.56.1-r0 \ cargo=$CARGO_VERSION \
chromium=~93.0.4577.82-r2 \ # Needed for numfmt, used in create_vps.sh
curl=~7.80.0-r0 \ coreutils=$COREUTILS_VERSION \
# needed for installing pycurl python module chromium=$CHROMIUM_VERSION \
curl-dev=~7.80.0-r0 \ curl=$CURL_VERSION \
# Needed for installing pycurl python module
curl-dev=$CURL_VERSION \
# Needed for "unbuffer" to timestamp cmds # Needed for "unbuffer" to timestamp cmds
expect=~5.45.4-r2 \ expect=$EXPECT_VERSION \
git=~2.34.1-r0 \ git=$GIT_VERSION \
libffi-dev=~3.4.2-r1 \ # Needed in create-vps job
make=~4.3-r0 \ jq=$JQ_VERSION \
libffi-dev=$LIBFFI_DEV_VERSION \
# Needed for pynacl
libsodium-dev=$LIBSODIUM_DEV_VERSION \
make=$MAKE_VERSION \
# Needed for timestamp cmd "ts" # Needed for timestamp cmd "ts"
moreutils=~0.66-r1 \ moreutils=$MOREUTILS_VERSION \
npm=~8.1.3-r0 \ npm=$NPM_VERSION \
openssh-client-default=~8.8_p1-r1 \ openssh-client-default=$OPENSSH_CLIENT_DEFAULT_VERSION \
py3-pip=~20.3.4-r1 \ py3-pip=$PY3_PIP_VERSION \
python3-dev=~3.9.7-r4 \ python3-dev=$PYTHON3_DEV_VERSION \
rsync=~3.2.3-r5 \ rsync=$RSYNC_VERSION \
yq=~4.14.1-r0 && \ yq-go=$YQ_VERSION && \
rm -rf /var/cache/* && \ rm -rf /var/cache/* && \
mkdir /var/cache/apk && \ mkdir /var/cache/apk && \
pip install --no-cache-dir --ignore-installed six -r /requirements.txt && \ # We need to install wheel before running the pip install,
ln -s /usr/bin/python3 /usr/bin/python && \ # so pip can use it while installing the other packages
tar -xzf /tmp/flux*.tar.gz && mv ./flux /usr/local/bin && \ pip install --no-cache-dir --break-system-packages "$(grep '^.*wheel==' requirements.txt)" && \
npm install -g taiko@1.2.6 pip install --no-cache-dir --break-system-packages --ignore-installed -r /requirements.txt && \
tar -xzf /tmp/flux*.tar.gz && mv ./flux /usr/local/bin
Makefile deleted 100644 → 0
View file @ d6090192
.DEFAULT_GOAL := help
help:
# See Makefile for available targets.
freeze:
# pip wrongly adds `pkg-resources==0.0.0` which will break some systems, see
# https://open.greenhost.net/stackspin/stackspin/issues/498#note_15057
pip3 freeze | grep -v "pkg-resources" > requirements-stable.txt
VERSION
View file @ 601a9684
0.7.0 2.17
ansible/ansible.cfg
View file @ 601a9684
...@@ -3,5 +3,3 @@ callback_whitelist = profile_tasks, timer ...@@ -3,5 +3,3 @@ callback_whitelist = profile_tasks, timer
inventory = inventory.yml inventory = inventory.yml
nocows = 1 nocows = 1
stdout_callback = yaml stdout_callback = yaml
strategy_plugins = plugins/mitogen-0.2.9/ansible_mitogen/plugins/strategy
strategy = mitogen_linear
ansible/group_vars/all/stackspin.yml
View file @ 601a9684
...@@ -3,22 +3,32 @@ ...@@ -3,22 +3,32 @@
data_directory: "/var/lib/Stackspin" data_directory: "/var/lib/Stackspin"
ip_address: "{{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}" ip_address: "{{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}"
# Use python3 on cluster nodes for ansible
ansible_python_interpreter: "/usr/bin/env python3"
# Application versions # Application versions
flux: flux:
# We'd like to update flux using renovate as well, but that's hard right now
# because we need to perform a couple of steps when updating flux.
# https://github.com/fluxcd/flux2/releases # https://github.com/fluxcd/flux2/releases
version: 0.22.0 version: "2.5.1"
k3s: k3s:
# https://github.com/k3s-io/k3s/releases # https://github.com/k3s-io/k3s/releases
version: 'v1.23.1+k3s2' # renovate: datasource=github-releases depName=k3s-io/k3s
version: "v1.32.3+k3s1"
# args to start the k3s server with # args to start the k3s server with
# https://rancher.com/docs/k3s/latest/en/installation/install-options/server-config/ # https://rancher.com/docs/k3s/latest/en/installation/install-options/server-config/
# kubelet arguments can be passed with `--kubelet-arg` # kubelet arguments can be passed with `--kubelet-arg`
# https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ # https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/
server_args: "--disable traefik --disable local-storage --disable servicelb --kube-apiserver-arg=event-ttl=48h0m0s --tls-san {{ ip_address }}" #
# egress-selector-mode=disabled is needed to prevent random webhook failures, see
# https://open.greenhost.net/stackspin/stackspin/-/issues/1407
# https://github.com/k3s-io/k3s/issues/5835
server_args: >
--disable traefik
--disable local-storage
--disable servicelb
--kube-apiserver-arg=event-ttl=48h0m0s
--tls-san {{ ip_address }}
--egress-selector-mode=disabled
docker_mirror: docker_mirror:
enabled: false enabled: false
ansible/plugins/mitogen-0.2.9/.lgtm.yml deleted 100644 → 0
View file @ d6090192
path_classifiers:
library:
- "mitogen/compat"
- "ansible_mitogen/compat"
queries:
# Mitogen 2.4 compatibility trips this query everywhere, so just disable it
- exclude: py/unreachable-statement
- exclude: py/should-use-with
# mitogen.core.b() trips this query everywhere, so just disable it
- exclude: py/import-and-import-from
ansible/plugins/mitogen-0.2.9/LICENSE deleted 100644 → 0
View file @ d6090192
Copyright 2019, David Wilson
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors
may be used to endorse or promote products derived from this software without
specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
ansible/plugins/mitogen-0.2.9/MANIFEST.in deleted 100644 → 0
View file @ d6090192
include LICENSE
ansible/plugins/mitogen-0.2.9/README.md deleted 100644 → 0
View file @ d6090192
# Mitogen
<!-- [![Build Status](https://travis-ci.org/dw/mitogen.png?branch=master)](https://travis-ci.org/dw/mitogen}) -->
<a href="https://mitogen.networkgenomics.com/">Please see the documentation</a>.
![](https://i.imgur.com/eBM6LhJ.gif)
[![Total alerts](https://img.shields.io/lgtm/alerts/g/dw/mitogen.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/dw/mitogen/alerts/)
[![Build Status](https://travis-ci.org/dw/mitogen.svg?branch=master)](https://travis-ci.org/dw/mitogen)
[![Pipelines Status](https://dev.azure.com/dw-mitogen/Mitogen/_apis/build/status/dw.mitogen?branchName=master)](https://dev.azure.com/dw-mitogen/Mitogen/_build/latest?definitionId=1?branchName=master)
ansible/plugins/mitogen-0.2.9/ansible_mitogen/affinity.py deleted 100644 → 0
View file @ d6090192
# Copyright 2019, David Wilson
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# 3. Neither the name of the copyright holder nor the names of its contributors
# may be used to endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
"""
As Mitogen separates asynchronous IO out to a broker thread, communication
necessarily involves context switching and waking that thread. When application
threads and the broker share a CPU, this can be almost invisibly fast - around
25 microseconds for a full A->B->A round-trip.
However when threads are scheduled on different CPUs, round-trip delays
regularly vary wildly, and easily into milliseconds. Many contributing factors
exist, not least scenarios like:
1. A is preempted immediately after waking B, but before releasing the GIL.
2. B wakes from IO wait only to immediately enter futex wait.
3. A may wait 10ms or more for another timeslice, as the scheduler on its CPU
runs threads unrelated to its transaction (i.e. not B), wake only to release
its GIL, before entering IO sleep waiting for a reply from B, which cannot
exist yet.
4. B wakes, acquires GIL, performs work, and sends reply to A, causing it to
wake. B is preempted before releasing GIL.
5. A wakes from IO wait only to immediately enter futex wait.
6. B may wait 10ms or more for another timeslice, wake only to release its GIL,
before sleeping again.
7. A wakes, acquires GIL, finally receives reply.
Per above if we are unlucky, on an even moderately busy machine it is possible
to lose milliseconds just in scheduling delay, and the effect is compounded
when pairs of threads in process A are communicating with pairs of threads in
process B using the same scheme, such as when Ansible WorkerProcess is
communicating with ContextService in the connection multiplexer. In the worst
case it could involve 4 threads working in lockstep spread across 4 busy CPUs.
Since multithreading in Python is essentially useless except for waiting on IO
due to the presence of the GIL, at least in Ansible there is no good reason for
threads in the same process to run on distinct CPUs - they always operate in
lockstep due to the GIL, and are thus vulnerable to issues like above.
Linux lacks any natural API to describe what we want, it only permits
individual threads to be constrained to run on specific CPUs, and for that
constraint to be inherited by new threads and forks of the constrained thread.
This module therefore implements a CPU pinning policy for Ansible processes,
providing methods that should be called early in any new process, either to
rebalance which CPU it is pinned to, or in the case of subprocesses, to remove
the pinning entirely. It is likely to require ongoing tweaking, since pinning
necessarily involves preventing the scheduler from making load balancing
decisions.
"""
from __future__ import absolute_import
import ctypes
import logging
import mmap
import multiprocessing
import os
import struct
import mitogen.core
import mitogen.parent
LOG = logging.getLogger(__name__)
try:
_libc = ctypes.CDLL(None, use_errno=True)
_strerror = _libc.strerror
_strerror.restype = ctypes.c_char_p
_sem_init = _libc.sem_init
_sem_wait = _libc.sem_wait
_sem_post = _libc.sem_post
_sched_setaffinity = _libc.sched_setaffinity
except (OSError, AttributeError):
_libc = None
_strerror = None
_sem_init = None
_sem_wait = None
_sem_post = None
_sched_setaffinity = None
class sem_t(ctypes.Structure):
"""
Wrap sem_t to allow storing a lock in shared memory.
"""
_fields_ = [
('data', ctypes.c_uint8 * 128),
]
def init(self):
if _sem_init(self.data, 1, 1):
raise Exception(_strerror(ctypes.get_errno()))
def acquire(self):
if _sem_wait(self.data):
raise Exception(_strerror(ctypes.get_errno()))
def release(self):
if _sem_post(self.data):
raise Exception(_strerror(ctypes.get_errno()))
class State(ctypes.Structure):
"""
Contents of shared memory segment. This allows :meth:`Manager.assign` to be
called from any child, since affinity assignment must happen from within
the context of the new child process.
"""
_fields_ = [
('lock', sem_t),
('counter', ctypes.c_uint8),
]
class Policy(object):
"""
Process affinity policy.
"""
def assign_controller(self):
"""
Assign the Ansible top-level policy to this process.
"""
def assign_muxprocess(self, index):
"""
Assign the MuxProcess policy to this process.
"""
def assign_worker(self):
"""
Assign the WorkerProcess policy to this process.
"""
def assign_subprocess(self):
"""
Assign the helper subprocess policy to this process.
"""
class FixedPolicy(Policy):
"""
:class:`Policy` for machines where the only control method available is
fixed CPU placement. The scheme here was tested on an otherwise idle 16
thread machine.
- The connection multiplexer is pinned to CPU 0.
- The Ansible top-level (strategy) is pinned to CPU 1.
- WorkerProcesses are pinned sequentually to 2..N, wrapping around when no
more CPUs exist.
- Children such as SSH may be scheduled on any CPU except 0/1.
If the machine has less than 4 cores available, the top-level and workers
are pinned between CPU 2..N, i.e. no CPU is reserved for the top-level
process.
This could at least be improved by having workers pinned to independent
cores, before reusing the second hyperthread of an existing core.
A hook is installed that causes :meth:`reset` to run in the child of any
process created with :func:`mitogen.parent.popen`, ensuring CPU-intensive
children like SSH are not forced to share the same core as the (otherwise
potentially very busy) parent.
"""
def __init__(self, cpu_count=None):
#: For tests.
self.cpu_count = cpu_count or multiprocessing.cpu_count()
self.mem = mmap.mmap(-1, 4096)
self.state = State.from_buffer(self.mem)
self.state.lock.init()
if self.cpu_count < 2:
# uniprocessor
self._reserve_mux = False
self._reserve_controller = False
self._reserve_mask = 0
self._reserve_shift = 0
elif self.cpu_count < 4:
# small SMP
self._reserve_mux = True
self._reserve_controller = False
self._reserve_mask = 1
self._reserve_shift = 1
else:
# big SMP
self._reserve_mux = True
self._reserve_controller = True
self._reserve_mask = 3
self._reserve_shift = 2
def _set_affinity(self, descr, mask):
if descr:
LOG.debug('CPU mask for %s: %#08x', descr, mask)
mitogen.parent._preexec_hook = self._clear
self._set_cpu_mask(mask)
def _balance(self, descr):
self.state.lock.acquire()
try:
n = self.state.counter
self.state.counter += 1
finally:
self.state.lock.release()
self._set_cpu(descr, self._reserve_shift + (
(n % (self.cpu_count - self._reserve_shift))
))
def _set_cpu(self, descr, cpu):
self._set_affinity(descr, 1 << (cpu % self.cpu_count))
def _clear(self):
all_cpus = (1 << self.cpu_count) - 1
self._set_affinity(None, all_cpus & ~self._reserve_mask)
def assign_controller(self):
if self._reserve_controller:
self._set_cpu('Ansible top-level process', 1)
else:
self._balance('Ansible top-level process')
def assign_muxprocess(self, index):
self._set_cpu('MuxProcess %d' % (index,), index)
def assign_worker(self):
self._balance('WorkerProcess')
def assign_subprocess(self):
self._clear()
class LinuxPolicy(FixedPolicy):
def _mask_to_bytes(self, mask):
"""
Convert the (type long) mask to a cpu_set_t.
"""
chunks = []
shiftmask = (2 ** 64) - 1
for x in range(16):
chunks.append(struct.pack('<Q', mask & shiftmask))
mask >>= 64
return mitogen.core.b('').join(chunks)
def _get_thread_ids(self):
try:
ents = os.listdir('/proc/self/task')
except OSError:
LOG.debug('cannot fetch thread IDs for current process')
return [os.getpid()]
return [int(s) for s in ents if s.isdigit()]
def _set_cpu_mask(self, mask):
s = self._mask_to_bytes(mask)
for tid in self._get_thread_ids():
_sched_setaffinity(tid, len(s), s)
if _sched_setaffinity is not None:
policy = LinuxPolicy()
else:
policy = Policy()
ansible/plugins/mitogen-0.2.9/ansible_mitogen/compat/simplejson/__init__.py deleted 100644 → 0
View file @ d6090192
r"""JSON (JavaScript Object Notation) <http://json.org> is a subset of
JavaScript syntax (ECMA-262 3rd edition) used as a lightweight data
interchange format.
:mod:`simplejson` exposes an API familiar to users of the standard library
:mod:`marshal` and :mod:`pickle` modules. It is the externally maintained
version of the :mod:`json` library contained in Python 2.6, but maintains
compatibility with Python 2.4 and Python 2.5 and (currently) has
significant performance advantages, even without using the optional C
extension for speedups.
Encoding basic Python object hierarchies::
>>> import simplejson as json
>>> json.dumps(['foo', {'bar': ('baz', None, 1.0, 2)}])
'["foo", {"bar": ["baz", null, 1.0, 2]}]'
>>> print json.dumps("\"foo\bar")
"\"foo\bar"
>>> print json.dumps(u'\u1234')
"\u1234"
>>> print json.dumps('\\')
"\\"
>>> print json.dumps({"c": 0, "b": 0, "a": 0}, sort_keys=True)
{"a": 0, "b": 0, "c": 0}
>>> from StringIO import StringIO
>>> io = StringIO()
>>> json.dump(['streaming API'], io)
>>> io.getvalue()
'["streaming API"]'
Compact encoding::
>>> import simplejson as json
>>> json.dumps([1,2,3,{'4': 5, '6': 7}], separators=(',',':'))
'[1,2,3,{"4":5,"6":7}]'
Pretty printing::
>>> import simplejson as json
>>> s = json.dumps({'4': 5, '6': 7}, sort_keys=True, indent=4)
>>> print '\n'.join([l.rstrip() for l in s.splitlines()])
{
"4": 5,
"6": 7
}
Decoding JSON::
>>> import simplejson as json
>>> obj = [u'foo', {u'bar': [u'baz', None, 1.0, 2]}]
>>> json.loads('["foo", {"bar":["baz", null, 1.0, 2]}]') == obj
True
>>> json.loads('"\\"foo\\bar"') == u'"foo\x08ar'
True
>>> from StringIO import StringIO
>>> io = StringIO('["streaming API"]')
>>> json.load(io)[0] == 'streaming API'
True
Specializing JSON object decoding::
>>> import simplejson as json
>>> def as_complex(dct):
... if '__complex__' in dct:
... return complex(dct['real'], dct['imag'])
... return dct
...
>>> json.loads('{"__complex__": true, "real": 1, "imag": 2}',
... object_hook=as_complex)
(1+2j)
>>> import decimal
>>> json.loads('1.1', parse_float=decimal.Decimal) == decimal.Decimal('1.1')
True
Specializing JSON object encoding::
>>> import simplejson as json
>>> def encode_complex(obj):
... if isinstance(obj, complex):
... return [obj.real, obj.imag]
... raise TypeError(repr(o) + " is not JSON serializable")
...
>>> json.dumps(2 + 1j, default=encode_complex)
'[2.0, 1.0]'
>>> json.JSONEncoder(default=encode_complex).encode(2 + 1j)
'[2.0, 1.0]'
>>> ''.join(json.JSONEncoder(default=encode_complex).iterencode(2 + 1j))
'[2.0, 1.0]'
Using simplejson.tool from the shell to validate and pretty-print::
$ echo '{"json":"obj"}' | python -m simplejson.tool
{
"json": "obj"
}
$ echo '{ 1.2:3.4}' | python -m simplejson.tool
Expecting property name: line 1 column 2 (char 2)
"""
__version__ = '2.0.9'
__all__ = [
'dump', 'dumps', 'load', 'loads',
'JSONDecoder', 'JSONEncoder',
]
__author__ = 'Bob Ippolito <bob@redivi.com>'
from decoder import JSONDecoder
from encoder import JSONEncoder
_default_encoder = JSONEncoder(
skipkeys=False,
ensure_ascii=True,
check_circular=True,
allow_nan=True,
indent=None,
separators=None,
encoding='utf-8',
default=None,
)
def dump(obj, fp, skipkeys=False, ensure_ascii=True, check_circular=True,
allow_nan=True, cls=None, indent=None, separators=None,
encoding='utf-8', default=None, **kw):
"""Serialize ``obj`` as a JSON formatted stream to ``fp`` (a
``.write()``-supporting file-like object).
If ``skipkeys`` is true then ``dict`` keys that are not basic types
(``str``, ``unicode``, ``int``, ``long``, ``float``, ``bool``, ``None``)
will be skipped instead of raising a ``TypeError``.
If ``ensure_ascii`` is false, then the some chunks written to ``fp``
may be ``unicode`` instances, subject to normal Python ``str`` to
``unicode`` coercion rules. Unless ``fp.write()`` explicitly
understands ``unicode`` (as in ``codecs.getwriter()``) this is likely
to cause an error.
If ``check_circular`` is false, then the circular reference check
for container types will be skipped and a circular reference will
result in an ``OverflowError`` (or worse).
If ``allow_nan`` is false, then it will be a ``ValueError`` to
serialize out of range ``float`` values (``nan``, ``inf``, ``-inf``)
in strict compliance of the JSON specification, instead of using the
JavaScript equivalents (``NaN``, ``Infinity``, ``-Infinity``).
If ``indent`` is a non-negative integer, then JSON array elements and object
members will be pretty-printed with that indent level. An indent level
of 0 will only insert newlines. ``None`` is the most compact representation.
If ``separators`` is an ``(item_separator, dict_separator)`` tuple
then it will be used instead of the default ``(', ', ': ')`` separators.
``(',', ':')`` is the most compact JSON representation.
``encoding`` is the character encoding for str instances, default is UTF-8.
``default(obj)`` is a function that should return a serializable version
of obj or raise TypeError. The default simply raises TypeError.
To use a custom ``JSONEncoder`` subclass (e.g. one that overrides the
``.default()`` method to serialize additional types), specify it with
the ``cls`` kwarg.
"""
# cached encoder
if (not skipkeys and ensure_ascii and
check_circular and allow_nan and
cls is None and indent is None and separators is None and
encoding == 'utf-8' and default is None and not kw):
iterable = _default_encoder.iterencode(obj)
else:
if cls is None:
cls = JSONEncoder
iterable = cls(skipkeys=skipkeys, ensure_ascii=ensure_ascii,
check_circular=check_circular, allow_nan=allow_nan, indent=indent,
separators=separators, encoding=encoding,
default=default, **kw).iterencode(obj)
# could accelerate with writelines in some versions of Python, at
# a debuggability cost
for chunk in iterable:
fp.write(chunk)
def dumps(obj, skipkeys=False, ensure_ascii=True, check_circular=True,
allow_nan=True, cls=None, indent=None, separators=None,
encoding='utf-8', default=None, **kw):
"""Serialize ``obj`` to a JSON formatted ``str``.
If ``skipkeys`` is false then ``dict`` keys that are not basic types
(``str``, ``unicode``, ``int``, ``long``, ``float``, ``bool``, ``None``)
will be skipped instead of raising a ``TypeError``.
If ``ensure_ascii`` is false, then the return value will be a
``unicode`` instance subject to normal Python ``str`` to ``unicode``
coercion rules instead of being escaped to an ASCII ``str``.
If ``check_circular`` is false, then the circular reference check
for container types will be skipped and a circular reference will
result in an ``OverflowError`` (or worse).
If ``allow_nan`` is false, then it will be a ``ValueError`` to
serialize out of range ``float`` values (``nan``, ``inf``, ``-inf``) in
strict compliance of the JSON specification, instead of using the
JavaScript equivalents (``NaN``, ``Infinity``, ``-Infinity``).
If ``indent`` is a non-negative integer, then JSON array elements and
object members will be pretty-printed with that indent level. An indent
level of 0 will only insert newlines. ``None`` is the most compact
representation.
If ``separators`` is an ``(item_separator, dict_separator)`` tuple
then it will be used instead of the default ``(', ', ': ')`` separators.
``(',', ':')`` is the most compact JSON representation.
``encoding`` is the character encoding for str instances, default is UTF-8.
``default(obj)`` is a function that should return a serializable version
of obj or raise TypeError. The default simply raises TypeError.
To use a custom ``JSONEncoder`` subclass (e.g. one that overrides the
``.default()`` method to serialize additional types), specify it with
the ``cls`` kwarg.
"""
# cached encoder
if (not skipkeys and ensure_ascii and
check_circular and allow_nan and
cls is None and indent is None and separators is None and
encoding == 'utf-8' and default is None and not kw):
return _default_encoder.encode(obj)
if cls is None:
cls = JSONEncoder
return cls(
skipkeys=skipkeys, ensure_ascii=ensure_ascii,
check_circular=check_circular, allow_nan=allow_nan, indent=indent,
separators=separators, encoding=encoding, default=default,
**kw).encode(obj)
_default_decoder = JSONDecoder(encoding=None, object_hook=None)
def load(fp, encoding=None, cls=None, object_hook=None, parse_float=None,
parse_int=None, parse_constant=None, **kw):
"""Deserialize ``fp`` (a ``.read()``-supporting file-like object containing
a JSON document) to a Python object.
If the contents of ``fp`` is encoded with an ASCII based encoding other
than utf-8 (e.g. latin-1), then an appropriate ``encoding`` name must
be specified. Encodings that are not ASCII based (such as UCS-2) are
not allowed, and should be wrapped with
``codecs.getreader(fp)(encoding)``, or simply decoded to a ``unicode``
object and passed to ``loads()``
``object_hook`` is an optional function that will be called with the
result of any object literal decode (a ``dict``). The return value of
``object_hook`` will be used instead of the ``dict``. This feature
can be used to implement custom decoders (e.g. JSON-RPC class hinting).
To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``
kwarg.
"""
return loads(fp.read(),
encoding=encoding, cls=cls, object_hook=object_hook,
parse_float=parse_float, parse_int=parse_int,
parse_constant=parse_constant, **kw)
def loads(s, encoding=None, cls=None, object_hook=None, parse_float=None,
parse_int=None, parse_constant=None, **kw):
"""Deserialize ``s`` (a ``str`` or ``unicode`` instance containing a JSON
document) to a Python object.
If ``s`` is a ``str`` instance and is encoded with an ASCII based encoding
other than utf-8 (e.g. latin-1) then an appropriate ``encoding`` name
must be specified. Encodings that are not ASCII based (such as UCS-2)
are not allowed and should be decoded to ``unicode`` first.
``object_hook`` is an optional function that will be called with the
result of any object literal decode (a ``dict``). The return value of
``object_hook`` will be used instead of the ``dict``. This feature
can be used to implement custom decoders (e.g. JSON-RPC class hinting).
``parse_float``, if specified, will be called with the string
of every JSON float to be decoded. By default this is equivalent to
float(num_str). This can be used to use another datatype or parser
for JSON floats (e.g. decimal.Decimal).
``parse_int``, if specified, will be called with the string
of every JSON int to be decoded. By default this is equivalent to
int(num_str). This can be used to use another datatype or parser
for JSON integers (e.g. float).
``parse_constant``, if specified, will be called with one of the
following strings: -Infinity, Infinity, NaN, null, true, false.
This can be used to raise an exception if invalid JSON numbers
are encountered.
To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``
kwarg.
"""
if (cls is None and encoding is None and object_hook is None and
parse_int is None and parse_float is None and
parse_constant is None and not kw):
return _default_decoder.decode(s)
if cls is None:
cls = JSONDecoder
if object_hook is not None:
kw['object_hook'] = object_hook
if parse_float is not None:
kw['parse_float'] = parse_float
if parse_int is not None:
kw['parse_int'] = parse_int
if parse_constant is not None:
kw['parse_constant'] = parse_constant
return cls(encoding=encoding, **kw).decode(s)