Commit c294e927 authored by Arie Peterson's avatar Arie Peterson
Browse files

Merge branch 'release-candidate/v1.0.0' into 'v1.0'

Releasing 1.0.0

See merge request !1442
parents 2ad9a335 df62574b
......@@ -12,9 +12,12 @@
/.direnv
.envrc
# Ignore files created during CI
# Ignore files created during CI or development
/test/group_vars/all/
/test/inventory*
/test/cypress/videos/
/test/package-lock.json
/test/package.json
/clusters
/install/installation-kustomization/*.txt
......@@ -31,3 +34,6 @@ Taskfile.yaml
# Documentation files
/docs/_build
# node modules for taiko or cypress
node_modules
......@@ -59,6 +59,9 @@ taiko:
when: on_failure
interruptible: true
rules:
- if: $RESOURCE == "wekan"
when: always
allow_failure: true
- if: '$SKIP_TAIKO == "true"'
when: never
- when: always
......@@ -94,8 +97,3 @@ helm-test:
- if: '$HELM_TEST == "true"'
when: always
- when: never
# TODO: REMOVE THIS
# Had to allow failure again because the new onlyoffice plugin makes helm test
# fail. This has been fixed in a newer chart version, but not in the v0.8
# branch
allow_failure: true
......@@ -88,6 +88,17 @@ enable-monitoring:
enable-nextcloud:
variables:
RESOURCE: "nextcloud"
before_script:
# Add Cypress record key so screenshots/videos of helm test are uploaded to
# the Cypress dashboard
- |
echo -e " tests:\n cypress:\n projectId: \"$CYPRESS_PROJECT_ID\"\n recordKey: \"$CYPRESS_RECORD_KEY\"" >> \
./install/overrides/ci/stackspin-nextcloud-override.yaml
echo -e " commitInfo:\n branch: \"$CI_COMMIT_REF_NAME\"\n message: \"$CI_COMMIT_TITLE\"" >> \
./install/overrides/ci/stackspin-nextcloud-override.yaml
echo -e " author: \"$CI_COMMIT_AUTHOR\"\n sha: \"$CI_COMMIT_SHORT_SHA\"" >> \
./install/overrides/ci/stackspin-nextcloud-override.yaml
- cat ./install/overrides/ci/stackspin-nextcloud-override.yaml
extends:
- .enable_app_template
- .nextcloud_rules
......
## Preparation
1. [ ] Received necessary information
- [ ] SMTP login credentials
- [ ] Domain
- [ ] List of requested applications
- [ ] Customization (custom subdomains, etc.)
- [ ] IP address of VPS
- [ ] Access to VPS
- [ ] Email address of the first admin
2. [ ] DNS is configured, one of:
- [ ] Wildcard CNAME `*.{domain}` & A record for `{domain}`, or
- [ ] Domains for all the requested apps
## Installation
3. [ ] Steps from [create
cluster](https://docs.stackspin.net/en/v0.8/installation/create_cluster.html)
finished
4. [ ] Edited `.flux.env`
5. [ ] Steps from [Install
Stackspin](https://docs.stackspin.net/en/v0.8/installation/install_stackspin.html)
finished
6. [ ] Installed apps
7. [ ] Copied cluster folder from `./clusters/<client>` into `cluster-configs` repository
## Test
8. [ ] Confirmed all `kustomizations` and `helmreleases` are `Ready`.
9. [ ] Confirmed apps are reachable through requested domains
## Communication
10. [ ] Communicated admin user credentials?
11. [ ] Sent the following email:
- [ ] Replaced `{customer}` with customer name
- [ ] Replaced `{me}` with my name
- [ ] Replaced `{domain}` with right domain (and changed any custom (sub)domains)
- [ ] Removed apps that were not installed
> Dear `{customer}`,
>
> I have installed Stackspin and created an admin account for you.
> You can set a password for your account by going to the following link:
>
> https://dashboard.{domain}/web/recovery
>
> There, you have to enter your email address
> (the one I'm sending this email to)
> and then you'll be emailed another link
> which allows you to set the password you use to log in.
>
> After setting that password you can use it to log in to:
>
> - The dashboard at: https://dashboard.{domain}
>
> - The Zulip hosted at: https://talk.prototypefund.de
>
> - Nextcloud (file storage and sharing, password storage and a calendar)
> at https://files.{domain}
>
> - WordPress (Website CMS) at https://www.{domain}
>
> - Wekan ("Kanban" planning board) at https://wekan.{domain}
>
> Feel free to contact us if you have any questions.
>
> Kind regards,
> `{me}`
> Stackspin
# Release checklist
## Before finalizing release
## Release candidate
Make these changes in the main branch before releasing:
* [ ] Ensure all applications/dependencies/charts are at their latest versions
see `.gitlab/issue_templates/update_all_components.md`
## Do pre-release upgrade testing
We have a special upgrade pipeline to test upgrading from the previous release
to this new (candidate) release. This pipeline runs when the target branch of a
MR matches `pre-release/*`. In principle you could make a MR to merge `main`
into `pre-release/v0.8`, but that has the unfortunate side-effect that the
pipeline will restart whenever the source branch changes, which happens all the
time because of renovatebot automerging minor updates and colleagues that work
too hard. Therefore:
* [ ] Create a new branch, for example `release-candidate/v0.8.3`, from `main`.
* [ ] Create a MR to merge `release-candidate/v0.8.3` into `pre-release/v0.8`.
* [ ] Make sure that the resulting upgrade-test pipeline is successful
* [ ] Before merging, maybe notify #general on `stackspin.net` that we're
about to release, and that `stackspin.net` itself can experience short
downtime while the upgrade is in progress.
### Check upgrade on `stackspin.net`
Now that the new code has been merged to `pre-release/v0.8`, it will be picked
up by the `stackspin.net` cluster which is set to follow that branch. Even
though the upgrade pipeline already tested the upgrade process, it's still good
to check if the upgrade goes well there:
* [ ] do a `flux reconcile source git stackspin` so you don't have to wait
until flux decides it's time to reconcile;
* [ ] `watch flux get kustomization` to see components being upgraded. If
necessary check the status using `kubectl describe hr ...` and debug.
## Release metadata
* [ ] Create a new merge request for merging the just-updated
`pre-release/v0.8` into the release branch `v0.8`.
Add the below items to this MR.
* [ ] Create a new branch, `release-candidate/v0.8.x`, from `main`.
- `git checkout main`
- `git pull`
- `git checkout -b release-candidate/v0.8.x`
### Only for major releases
......@@ -52,7 +16,6 @@ Add the below items to this MR.
### For all releases
* Update [CHANGELOG.md](https://keepachangelog.com)
* [ ] Include any `Known issues`
* [ ] Include all merged MR since last release, i.e. using [lab](https://github.com/zaquestion/lab#installation):
```sh
......@@ -84,8 +47,40 @@ Add the below items to this MR.
* [ ] Include `Known issues`
* [ ] Update the version number in the `VERSION` file
* [ ] Commit, push to MR (of `pre-release/*` into `v0.8`)
* [ ] Commit, push and create MR of `release-candidate/v0.8.x` into `pre-release/v0.8.x`
### Pre-release upgrade testing
We have a special upgrade pipeline to test upgrading from the previous release
to this new (candidate) release. This pipeline runs when the target branch of a
MR matches `pre-release/*`. In principle you could make a MR to merge `main`
into `pre-release/v0.8`, but that has the unfortunate side-effect that the
pipeline will restart whenever the source branch changes, which happens all the
time because of renovatebot automerging minor updates and colleagues that work
too hard. Therefore:
* [ ] Make sure that the resulting upgrade-test pipeline is successful
* [ ] Wait until MR gets reviewed and merged
* [ ] Before merging, notify #general on `stackspin.net` that we're about to
release, and that `stackspin.net` itself can experience short downtime while
the upgrade is in progress.
### Check upgrade on `stackspin.net`
Now that the new code has been merged to `pre-release/v0.8`, it will be picked
up by the `stackspin.net` cluster which is set to follow that branch. Even
though the upgrade pipeline already tested the upgrade process, it's still good
to check if the upgrade goes well there:
* [ ] do a `flux reconcile source git stackspin` so you don't have to wait
until flux decides it's time to reconcile;
* [ ] `watch flux get kustomization` to see components being upgraded. If
necessary check the status using `kubectl describe hr ...` and debug.
## Release metadata
* [ ] Create a new merge request for merging the just-updated
`pre-release/v0.8` into the release branch `v0.8`.
## Push a signed tag
......@@ -103,10 +98,4 @@ Add the below items to this MR.
* [ ] Create a new MR to merge the release branch back into `main`. This is
necessary to propagate the changes to CHANGELOG etc.
* [ ] Close released Milestone and set start date for the new Milestone
* [ ] Upgrade `init.stackspin.net`
* Create issues for
* [ ] Scoping the next Milestone
* [ ] Cleaning up
* old branches
* old container images
* [ ] Celebrate :clinking_glass: !!
......@@ -13,7 +13,7 @@ repos:
- id: trailing-whitespace
- id: debug-statements
- repo: https://github.com/jumanjihouse/pre-commit-hooks
rev: 2.1.6
rev: 3.0.0
hooks:
- id: shellcheck
- id: shfmt
......@@ -31,7 +31,7 @@ repos:
hooks:
- id: isort
- repo: https://github.com/zricethezav/gitleaks
rev: v8.9.0
rev: v8.12.0
hooks:
- id: gitleaks-docker
# Enable if you want to lint your commit msgs according to
......
# Changelog
## [1.0.0]
### Features
* Allow adjusting the subdomains applications run on (!1270)
### Bug fixes
* Fix NC warnings in settings (!1313)
* Automigrate jobs are spawned too fast (!1386)
### Documentation
* Make documentation production ready (!1346)
### Updates
* update k3s to v1.24.4+k3s1 (!1439)
* update flux to 0.33.0 (!1438)
* update helm release wordpress to v0.7.18 (!1432)
* update helm release hydra to v0.25.2 (!1416)
* update helm release metallb to v4.1.2 (!1427)
* update helm release ingress-nginx to v4.2.5 (!1431)
* update helm release kube-prometheus-stack to v39.11.0 (!1424)
* update helm release mariadb to v11.2.2 (!1422)
* update helm release loki to v2.16.0 (!1417)
* update helm release nextcloud-onlyoffice to v0.10.25 (!1410)
* update helm release stackspin-dashboard to v1.2.1 (!1397)
* update dependency ansible to v6.3.0 (!1402)
* update helm release promtail to v6.3.0 (!1389)
* update helm release velero to v2.31.3 (!1390)
* update helm release kratos to v0.24.5 (!1336)
### Current application versions:
| name | chart | app_version |
|---------------------------|-----------|-----------------------|
| cert-manager | v1.9.1 | v1.9.1 |
| dashboard | 1.2.2 | 0.2.8 |
| eventrouter | 0.4.0 | 0.3 |
| hydra | 0.25.1 | v1.11.8 |
| ingress-nginx | 4.2.3 | 1.3.0 |
| kratos | 0.24.5 | v0.10.1 |
| kube-prometheus-stack | 39.11.0 | 0.58.0 |
| local-path-provisioner | 0.0.22 | v0.0.22 |
| loki | 2.16.0 | v2.6.1 |
| metallb | 4.1.1 | 0.13.4 |
| missing-container-metrics | 0.25.0 | 0.25.0 |
| nextcloud & onlyoffice | 0.10.25 | NC-24.0.2-OO-7.1.1.23 |
| promtail | 6.3.0 | 2.6.1 |
| single-sign-on-database | 11.2.2 | 10.6.9 |
| velero | 2.31.3 | 1.9.1 |
| wekan | 1.1.1 | 5.93 |
| wordpress | 0.7.16 | 6.0.1 |
| zulip | 0.2.1 | 5.3-0 |
## [0.8.5]
### Features
......
......@@ -16,13 +16,11 @@ ENV BASH_VERSION="5.1.16-r2"
# renovate: datasource=repology depName=alpine_3_16/cargo
ENV CARGO_VERSION="1.60.0-r2"
# renovate: datasource=repology depName=alpine_3_16/chromium versioning=loose
ENV CHROMIUM_VERSION="102.0.5005.167-r0"
ENV CHROMIUM_VERSION="102.0.5005.173-r0"
# renovate: datasource=repology depName=alpine_3_16/coreutils version=loose
ENV COREUTILS_VERSION="9.1-r0"
# renovate: datasource=repology depName=alpine_3_16/curl
ENV CURL_VERSION="7.83.1-r2"
# renovate: datasource=repology depName=alpine_3_16/curl-dev
ENV CURL_DEV_VERSION="7.83.1-r2"
ENV CURL_VERSION="7.83.1-r3"
# renovate: datasource=repology depName=alpine_3_16/expect
ENV EXPECT_VERSION="5.45.4-r2"
# renovate: datasource=repology depName=alpine_3_16/git
......@@ -30,7 +28,7 @@ ENV GIT_VERSION="2.36.2-r0"
# renovate: datasource=repology depName=alpine_3_16/jq versioning=loose
ENV JQ_VERSION="1.6-r1"
# renovate: datasource=repology depName=alpine_edge/kubectl
ENV KUBECTL_VERSION="1.24.3-r1"
ENV KUBECTL_VERSION="1.24.4-r0"
# renovate: datasource=repology depName=alpine_3_16/libffi-dev
ENV LIBFFI_DEV_VERSION="3.4.2-r1"
# renovate: datasource=repology depName=alpine_3_16/libsodium-dev
......@@ -48,14 +46,14 @@ ENV PY3_PIP_VERSION="22.1.1-r0"
# renovate: datasource=repology depName=alpine_3_16/python3-dev
ENV PYTHON3_DEV_VERSION="3.10.5-r0"
# renovate: datasource=repology depName=alpine_3_16/rsync
ENV RSYNC_VERSION="3.2.4-r1"
ENV RSYNC_VERSION="3.2.4-r2"
# renovate: datasource=repology depName=alpine_3_16/yq
ENV YQ_VERSION="4.25.1-r2"
ENV YQ_VERSION="4.25.1-r3"
# Makes pynacl use system SODIUM
ENV SODIUM_INSTALL=system
ADD https://github.com/fluxcd/flux2/releases/download/v0.31.3/flux_0.31.3_linux_amd64.tar.gz /tmp/
ADD https://github.com/fluxcd/flux2/releases/download/v0.33.0/flux_0.33.0_linux_amd64.tar.gz /tmp/
COPY ./requirements.txt /requirements.txt
# Ignore the hadolint error for the pip install line with a `grep` in it
# hadolint ignore=DL3013
......@@ -71,7 +69,7 @@ RUN \
chromium=$CHROMIUM_VERSION \
curl=$CURL_VERSION \
# Needed for installing pycurl python module
curl-dev=$CURL_DEV_VERSION \
curl-dev=$CURL_VERSION \
# Needed for "unbuffer" to timestamp cmds
expect=$EXPECT_VERSION \
git=$GIT_VERSION \
......
......@@ -9,16 +9,28 @@ ansible_python_interpreter: "/usr/bin/env python3"
# Application versions
flux:
# https://github.com/fluxcd/flux2/releases
version: 0.31.3
version: 0.33.0
k3s:
# https://github.com/k3s-io/k3s/releases
version: 'v1.23.3+k3s1'
version: 'v1.24.4+k3s1'
# args to start the k3s server with
# https://rancher.com/docs/k3s/latest/en/installation/install-options/server-config/
# kubelet arguments can be passed with `--kubelet-arg`
# https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/
server_args: "--disable traefik --disable local-storage --disable servicelb --kube-apiserver-arg=event-ttl=48h0m0s --tls-san {{ ip_address }}"
#
# The --kube-apiserver-arg=feature-gates=JobTrackingWithFinalizers=false
# and --kube-controller-manager-arg=feature-gates=JobTrackingWithFinalizers=false
# are needed without this, jobs (i.e. helm install hooks) are spawned too fast
# after each other (see https://open.greenhost.net/stackspin/stackspin/-/issues/1228)
server_args: >
--disable traefik
--disable local-storage
--disable servicelb
--kube-apiserver-arg=event-ttl=48h0m0s
--kube-apiserver-arg=feature-gates=JobTrackingWithFinalizers=false
--kube-controller-manager-arg=feature-gates=JobTrackingWithFinalizers=false
--tls-san {{ ip_address }}
docker_mirror:
enabled: false
......@@ -34,6 +34,8 @@ table inet filter {
# Ports only open from within the cluster
ip saddr 10.0.0.0/8 tcp dport 9100 counter accept
ip saddr 10.0.0.0/8 tcp dport 10250 counter accept
# metallb-speaker metrics
ip saddr 10.0.0.0/8 tcp dport 7472 counter accept
# make IPv6 work
icmpv6 type { nd-neighbor-solicit, echo-request, nd-router-advert, nd-neighbor-advert } accept
......
#!/bin/bash
# kubectl patch -n flux-system gitrepository stackspin -p '{"spec":{"ref":{"branch":"v1.0"}}}' --type="merge"
kubectl patch -n flux-system gitrepository stackspin -p '{"spec":{"ref":{"branch":"release-candidate/v1.0.0"}}}' --type="merge"
......@@ -44,6 +44,7 @@ For more information, go to `the Stackspin website`_.
system_administration/upgrading
system_administration/customizing
system_administration/troubleshooting
system_administration/testing
system_administration/security
.. toctree::
......
......@@ -5,14 +5,9 @@ Installation overview
=====================
.. warning::
- Stackspin is still under heavy development and is not ready for production use!
We anticipate major changes
and do not guarantee a data-preserving upgrade path from current installations.
However, we encourage you to try Stackspin
and ask you to `report all issues you encounter <https://stackspin.net/contact.html>`__.
- When you install Stackspin on a server,
the installation process will make some substantial changes to the server's configuration,
so please do not use a server that functions as anything other than a testing ground.
When you install Stackspin on a server, the installation process will make
substantial changes to the server's configuration, so please do not use a
server that performs any other functions that should not be disrupted.
This page will guide you through the installation of the "Stackspin CLI".
The CLI can be used to set up Stackspin on a different machine that is connected to the internet.
......@@ -39,17 +34,17 @@ The following software setup is required **on your provisioning machine**:
If you run into problems during ``pip install``
you might need to run ``pip install --upgrade pip``.
- ``kubectl`` (`installation instructions <https://kubernetes.io/docs/tasks/tools/#kubectl>`__)
- ``flux`` version 0.31.3 (`Download flux_0.31.3_linux_amd64.tar.gz
<https://github.com/fluxcd/flux2/releases/download/v0.31.3/flux_0.31.3_linux_amd64.tar.gz>`_
- ``flux`` version 0.33.0 (`Download flux_0.33.0_linux_amd64.tar.gz
<https://github.com/fluxcd/flux2/releases/download/v0.33.0/flux_0.33.0_linux_amd64.tar.gz>`_
or use the official installation script:
``curl -s https://fluxcd.io/install.sh | sudo FLUX_VERSION=0.31.3 bash``)
``curl -s https://fluxcd.io/install.sh | sudo FLUX_VERSION=0.33.0 bash``)
Then you can clone the Stackspin git repository
and checkout the latest release branch (currently ``v0.8``):
and checkout the latest release branch (currently ``v1.0``):
.. code:: console
$ git clone -b v0.8 https://open.greenhost.net/stackspin/stackspin.git
$ git clone -b v1.0 https://open.greenhost.net/stackspin/stackspin.git
$ cd stackspin
Apart from the previously installed software,
......
......@@ -26,7 +26,7 @@ you complete all steps.
Command line tests
------------------
Please :ref:`system_administration/troubleshooting:Run the CLI tests` which checks
Please refer to :ref:`system_administration/testing:Testing` which checks
the overall functionality of your cluster and include the output in your
feedback.
......
......@@ -7,7 +7,7 @@
#
recommonmark==0.7.1
sphinx==5.1.1
sphinx-design==0.2.0
sphinx-design==0.3.0
sphinx-rtd-theme==1.0.0
sphinx-version-warning==1.1.2
sphinxcontrib-mermaid==0.7.1
......@@ -6,11 +6,11 @@
#
alabaster==0.7.12
# via sphinx
babel==2.10.1
babel==2.10.3
# via sphinx
certifi==2022.5.18.1
certifi==2022.6.15
# via requests
charset-normalizer==2.0.12
charset-normalizer==2.1.1
# via requests
commonmark==0.9.1
# via recommonmark
......@@ -21,7 +21,7 @@ docutils==0.17.1
# sphinx-rtd-theme
idna==3.3
# via requests
imagesize==1.3.0
imagesize==1.4.1
# via sphinx
jinja2==3.1.2
# via sphinx
......@@ -29,15 +29,15 @@ markupsafe==2.1.1
# via jinja2
packaging==21.3
# via sphinx
pygments==2.12.0
pygments==2.13.0
# via sphinx
pyparsing==3.0.9
# via packaging
pytz==2022.1
pytz==2022.2.1
# via babel
recommonmark==0.7.1
# via -r requirements.in
requests==2.27.1
requests==2.28.1
# via sphinx
snowballstemmer==2.2.0
# via sphinx
......@@ -48,7 +48,7 @@ sphinx==5.1.1
# sphinx-design
# sphinx-rtd-theme
# sphinx-version-warning
sphinx-design==0.2.0
sphinx-design==0.3.0
# via -r requirements.in
sphinx-rtd-theme==1.0.0
# via -r requirements.in
......@@ -68,5 +68,5 @@ sphinxcontrib-qthelp==1.0.3
# via sphinx
sphinxcontrib-serializinghtml==1.1.5
# via sphinx
urllib3==1.26.9
urllib3==1.26.12
# via requests
=======
Testing
=======
To get an overall status of your cluster,
run the tests from the command line.
There are two types of tests:
`testinfra <https://testinfra.readthedocs.io/>`__ tests verify the setup,
and end-to-end tests that check the webservice availability.
Testinfra tests
===============
Testinfra tests are split into two groups, lets call them *blackbox* and
*clearbox* tests. The blackbox tests run on your provisioning machine and test
the Stackspin cluster from the outside. For example, the certificate check will check
if the Stackspin returns valid certificates for the provided services.
The clearbox tests run on the Stackspin host and check i.e. if docker is installed
in the right version etc. Our testinfra tests are a combination of blackbox and
clearbox tests.
If you have installed the Stackspin python requirements inside a :ref:`virtual environment <installation/install_cli:Create a python virtual environment (Optional)>`
then you need to activate it before testing:
.. code:: console
$ . env/bin/activate
Then enter the *test* directory in the Git repository
**on your provisioning machine**.
.. code:: console
$ cd test
To run the test against your cluster, first export the *CLUSTER_DIR*
environment variable with the location of your cluster config directory
(replace *stackspin.example.org* with your cluster name):
Then export the *KUBECONFIG* variable that lets ``kubectl`` talk to your cluster.
Thirdly, export a variable that contains the password for the Prometheus HTTP endpoint.
.. code:: console
$ export CLUSTER_DIR="${PWD}/../clusters/stackspin.example.org"
$ export KUBECONFIG=${CLUSTER_DIR}/kube_config_cluster.yml
$ export BASIC_AUTH_PW=$(kubectl get secret -n stackspin stackspin-prometheus-basic-auth --template '{{ .data.pass }}' | base64 -d)
$
Run all tests
-------------
.. code:: console
$ py.test -s --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*'
Test all applications
---------------------
This will check for:
* The applications return proper certificates
* All helm releases are successfully installed
* All app pods are running and healthy (this test includes all optional
applications)
These tests includes all optional applications and will fail for optional
applications that are not installed.
.. code:: console
$ pytest -s -m 'app' --connection=ansible --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*'
Tests a specific application
----------------------------
.. code:: console
$ pytest -s -m 'app' --app="wordpress" --connection=ansible --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*'
Advanced usage of Testinfra tests
---------------------------------