diff --git a/docs/README.md b/docs/README.md
index 9513698911ce73af869761d22def493e63ec490a..3b9cc2e9280cd72600bfa79f7cefbf65dec6fa30 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -1,8 +1,8 @@
## Documentation
-This folder contains the documentation. You can find the documentation at
-https://stackspin.readthedocs.io/. It is also possible to build the
-documentation, using Sphinx:
+This folder contains the documentation.
+You can find the documentation at https://stackspin.readthedocs.io/.
+It is also possible to build the documentation locally using Sphinx:
pip install -r ./requirements.txt
make html
diff --git a/docs/comparable_projects.md b/docs/comparable_projects.md
index 55c8bb13297c70393ffbd23b1a2edfd313568054..5adab2e5c022bdb635923dae9d3cef03b67b3013 100644
--- a/docs/comparable_projects.md
+++ b/docs/comparable_projects.md
@@ -1,40 +1,76 @@
# Comparable projects
-Other open source projects similar to Stackspin exist. Each of the platforms
-listed here, like Stackspin, provide a suite of open source cloud
-applications. Each of the platforms, like Stackspin, include their own user
-management dashboard and all of them offer single sign-on (SSO) features.
-
-However there are changes in implementation that make Stackspin different
-from these alternatives. As far as we found, none of the projects listed
-below will automatically update your applications without you having to push a
-button, for example.
-
-**[Sandstorm](https://sandstorm.io)** allows applications to be installed as so
-called Grains. Each grain is a copy of the complete application made for a
-specific purpose. For example, a grain for document editor "Etherpad" contains
-not only the data written in the notepad, but also the Etherpad and database
-software. If you have two notepads, you also have two copies of the software.
-With many users, this approach can run into limits.
-
-**[YunoHost](https://yunohost.org)** is mostly based on Debian and its package
-management and user management system. As a result, Yunohost is relatively lightweight.
-However, another result of this is that it is likely that if one of
-the applications on your Yunohost contain a security hole, the data of the other
-applications is compromised as well. This is less likely in Stackspin because
-it separates the environments of applications from each other to seal them off.
-
-**[Cloudron](https://cloudron.io)** offers a similar application suite. In
-contrast to Stackspin, Cloudron [requires a paid
-account](https://cloudron.io/pricing.html) if you want to use more than two
-applications or more than five users.
-
-All mentioned platforms require applications running on it to be changed in some
-way, for example to make use of the authentication system. Stackspin tries to
-steer clear of changing the applications it includes. As long as they support
-OpenID Connect, so you can sign in to it, we can usually run the application
-as-is. In most cases, Stackspin will only need one intervention before an
-update can be pushed to users: it needs to be tested. We want to make sure that
-an application works together with the platform and other applications before we
-let you use it. We work towards a fully automated test suite, so even this would
-not require human intervention.
+Other open source projects similar to Stackspin exist.
+Each of the platforms listed here, like Stackspin,
+provide a suite of open source cloud applications.
+Most of the platforms, like Stackspin,
+include their own user management dashboard
+and single sign-on (SSO) features.
+
+## What makes Stackspin unique
+
+However there are changes in implementation
+that make Stackspin different from these alternatives.
+As far as we found, none of the projects listed
+below will automatically update your applications
+without you having to push a button.
+Also, all mentioned platforms require application modifications,
+for example to make use of the authentication system.
+
+At Stackspin, we avoid changing the included applications
+to prevent maintenance overhead
+and enable seamless native updates.
+It only requires OpenID Connect support,
+so the application can be integrated with single sign-on.
+
+So far, Stackspin will need one intervention
+before an update can be pushed to users:
+It needs to be tested.
+We want to make sure that an application works together with the platform
+and other applications before we let you use it.
+We work towards a fully automated test suite,
+so in the future Stackspin will keep all applications up-to-date
+while minimizing failure.
+
+## Platforms
+
+### Fully Open
+
+**[Sandstorm](https://sandstorm.io)** allows applications to be installed as so called Grains.
+Each grain is a copy of the complete application made for a specific purpose.
+For example, a grain for document editor "Etherpad"
+contains not only the data written in the notepad,
+but also the Etherpad and database software.
+If you have two notepads, you also have two copies of the software.
+With many users, this approach runs into limits.
+
+**[YunoHost](https://yunohost.org)** is mostly based on Debian
+and its package management and user management system.
+As a result, Yunohost is relatively lightweight.
+However, another result of this is that it is likely that
+if one of the applications on your Yunohost contain a security hole,
+the data of the other applications is compromised as well.
+This is less likely in Stackspin
+as it separates the environments of applications from each other to seal them off.
+
+**[SelfPrivacy](https://selfprivacy.org/en)** automatically sets up an open stack
+focused on personal use from your phone on an established cloud provider.
+From the documentation it seems to not add any integration
+(such as SSO) between the services
+and the only dashboard is the app itself.
+
+**[Mail-in-a-Box](https://mailinabox.email)** is a narrowly focused project
+that tightly integrates Nextcloud and various applications
+to create a full-featured mailserver with a control panel.
+As such there is no SSO.
+
+### Open Core
+
+**[Cloudron](https://cloudron.io)** offers a similar application suite.
+In contrast to Stackspin,
+Cloudron [requires a paid account](https://cloudron.io/pricing.html)
+if you want to use more than two applications or more than five users.
+
+**[Univention Corporate Server](https://www.univention.com/products/ucs)**
+is focused on business and does not make reproducible builds easily available
+or facilitate community contribution.
diff --git a/docs/installation/install_cli.rst b/docs/installation/install_cli.rst
index 2acbf021c594e3ee2bf886ebd730442d6ff4b015..87742d61f13f3448ec8aeba95e660a4746185211 100644
--- a/docs/installation/install_cli.rst
+++ b/docs/installation/install_cli.rst
@@ -5,56 +5,57 @@ Installation overview
=====================
.. warning::
- - Stackspin is still under heavy development and is not ready for
- production use! We anticipate major changes and do not guarantee a
- data-preserving upgrade path from current installations. However, we
- encourage you to try Stackspin and ask you to `report all issues
- you encounter <https://stackspin.net/contact.html>`__.
- - When you install Stackspin on a server, the installation process
- will make some substantial changes to the server's configuration, so
- please do not use a server that functions as anything other than a
- testing ground.
-
-This page will guide you through the installation of the "Stackspin CLI". The
-CLI can be used to set up Stackspin on a different machine that is connected to
-the internet. It also functions as a starting point for running tests.
-
-Requirements
+ - Stackspin is still under heavy development and is not ready for production use!
+ We anticipate major changes
+ and do not guarantee a data-preserving upgrade path from current installations.
+ However, we encourage you to try Stackspin
+ and ask you to `report all issues you encounter <https://stackspin.net/contact.html>`__.
+ - When you install Stackspin on a server,
+ the installation process will make some substantial changes to the server's configuration,
+ so please do not use a server that functions as anything other than a testing ground.
+
+This page will guide you through the installation of the "Stackspin CLI".
+The CLI can be used to set up Stackspin on a different machine that is connected to the internet.
+It also functions as a starting point for running tests.
+
+Preparing the provisioning machine
------------
-You should have a trusted machine to run the installer on, this can be your
-laptop or pc. In this guide, we call it the *provisioning machine*. All
-commands in these installation instructions need to be run the provisioning
-machine, except when a command starts with ``cluster$``
-
-Before you can start, you need this software installed **on your provisioning
-machine**:
-
-- You need Python 3 with its development files, Pip and Git installed
- (``apt install python3-pip python3-dev git`` on Debian/Ubuntu)
-- We recommend using a `python
- virtualenv <https://docs.python.org/3/tutorial/venv.html>`__ to make
- sure we do not change any of your other projects. Install virtualenv
- by running ``pip3 install --user venv`` or ``apt install python3-venv``.
-- You need a recent version of Pip. If you run into problems during ``pip
- install`` you might need to run ``pip install --upgrade pip``.
-- ``kubectl`` (`installation instructions
- <https://kubernetes.io/docs/tasks/tools/#kubectl>`__)
+You should have a trusted machine to run the installer on,
+this can be your laptop or pc.
+In this guide, we call it the *provisioning machine*.
+All commands in these installation instructions need to be run the provisioning machine,
+except when a command starts with ``cluster$``
+
+The following software setup is required **on your provisioning machine**:
+
+- You need Python 3 with its development files, Pip and Git installed.
+ On Debian/Ubuntu: ``apt install python3-pip python3-dev git``
+- We recommend using a `python virtualenv <https://docs.python.org/3/tutorial/venv.html>`__
+ to make sure Stackspin does not change any of your other python projects.
+ Install virtualenv by running ``pip3 install --user venv``
+ or ``apt install python3-venv``.
+- You need a recent version of Pip.
+ If you run into problems during ``pip install``
+ you might need to run ``pip install --upgrade pip``.
+- ``kubectl`` (`installation instructions <https://kubernetes.io/docs/tasks/tools/#kubectl>`__)
- ``flux version 0.25.3`` (`Download flux_0.25.3_linux_amd64.tar.gz
<https://github.com/fluxcd/flux2/releases/download/v0.25.3/flux_0.25.3_linux_amd64.tar.gz>`_)
+ ``curl -s https://fluxcd.io/install.sh | sudo FLUX_VERSION=0.25.3 bash``
-
-Clone the Stackspin git repository
--------------------------------------
-
-On your **provisioning machine**, clone the Stackspin git repository
+Then you can clone the Stackspin git repository
and checkout the latest release branch (currently ``v0.8``):
-::
+:: code:: sh
$ git clone -b v0.8 https://open.greenhost.net/stackspin/stackspin.git
$ cd stackspin
+Apart from the previously installed software,
+Stackspin will only modify in the ``clusters`` subdirectory of the repository.
+So multiple provisioning machines can administer the same instance
+so long as that subdirectory is synced between them.
+
Create a python virtual environment (Optional)
----------------------------------------------
@@ -67,7 +68,7 @@ second command "activates" the virtualenv.
run python programs instead of your global environment. If you close your
terminal or open a new one, you need to activate the virtualenv again.
-::
+:: code:: sh
$ python3 -m venv env
$ . env/bin/activate
@@ -78,7 +79,7 @@ Install requirements
Next, install the Stackspin CLI client by running the following
commands:
-::
+:: code:: sh
$ pip3 install -r requirements.txt
@@ -87,7 +88,7 @@ Stackspin CLI client usage
Now you can run the Stackspin CLI as follows:
-::
+:: code:: sh
$ python -m stackspin CLUSTER_NAME <command>
@@ -96,10 +97,11 @@ subcommand help messages. Be sure to run this command in the root
directory of the git repository. In this tutorial, we're using
``stackspin.example.org`` as the cluster name. Try it out by running
-::
+:: code:: sh
$ python -m stackspin stackspin.example.org --help
-The next chapter, :ref:`create_kubernetes_cluster`, will explain setting up
-and/or configuring a Kubernetes cluster to install Stackspin on.
+The next chapter, :ref:`create_kubernetes_cluster`,
+will explain setting up and/or configuring a Kubernetes cluster
+to install Stackspin on.
diff --git a/docs/installation/install_stackspin.rst b/docs/installation/install_stackspin.rst
index 27343f509f91aad8da0576db66954cda1de0e2e3..7d7211a8a75f0191282785f26f804c45ae0e685f 100644
--- a/docs/installation/install_stackspin.rst
+++ b/docs/installation/install_stackspin.rst
@@ -13,8 +13,8 @@ Step 1: Flux configuration
==========================
Flux will run inside your Stackspin cluster to install and upgrade applications.
-It needs to be configured once, using the ``flux`` command line tool and scripts
-provided by us in the Stackspin repository.
+It needs to be configured once, using the ``flux`` command line tool
+and scripts provided by us in the Stackspin repository.
Configuration
-------------
@@ -39,16 +39,17 @@ Cluster information
Outgoing email
~~~~~~~~~~~~~~
-Stackspin uses SMTP to send emails. This is essential for finishing account
-setups with password recovery links. Additionally, apps like Nextcloud, Zulip
-and Wordpress will be able to send email notifications from the email address
-configured here.
-You also may receive alert notification emails from Stackspin's
-monitoring system. See :ref:`monitoring:Email alerts` for more information about
-those alerts, especially during installation.
+Stackspin uses SMTP to send emails.
+This is essential for finishing account setups with password recovery links.
+Additionally, apps like Nextcloud, Zulip and Wordpress
+will be able to send email notifications from the email address configured here.
+You also may receive alert notification emails from Stackspin's monitoring system.
+See :ref:`monitoring:Email alerts` for more information about those alerts,
+especially during installation.
-Because Stackspin does not include an email server, you need to search your
-(external) email provider's helpdesk for SMTP configuration details.
+Because Stackspin does not include an email server,
+you need to search your (external) email provider's helpdesk
+for SMTP configuration details.
Edit the ``clusters/stackspin.example.org/.flux.env`` file as follows:
@@ -117,49 +118,49 @@ directory **on your provisioning machine**. Don't forget to replace
We will use this variable in the following commands, set it to your cluster
directory.
-.. code:: bash
+.. code:: sh
export CLUSTER_DIR=$PWD/clusters/stackspin.example.org
Make sure your ``virtualenv`` is activated.
-.. code:: bash
+.. code:: sh
- . venv/bin/activate
+ . env/bin/activate
Copy the installation kustomization to your cluster directory.
-.. code:: bash
+.. code:: sh
cp install/kustomization.yaml $CLUSTER_DIR/
Tell kubectl to use your cluster's kube_config.
-.. code:: bash
+.. code:: sh
export KUBECONFIG=$CLUSTER_DIR/kube_config_cluster.yml
Ensure flux-system namespace is created.
-.. code:: bash
+.. code:: sh
kubectl get namespace flux-system 2>/dev/null || kubectl create namespace flux-system
This inserts the configuration from .flux.env into your cluster as a "secret".
-.. code:: bash
+.. code:: sh
kubectl apply -k $CLUSTER_DIR
After you have executed that code, your terminal should show:
-.. code:: bash
+.. code::
secret/stackspin-cluster-variables created
Next, run:
-.. code::
+.. code:: sh
./install/install-stackspin.sh
@@ -181,8 +182,8 @@ to install:
- Zulip chat with ``./install/install-app.sh zulip``
- Wekan with ``./install/install-app.sh wekan``
- WordPress with ``./install/install-app.sh wordpress``
-- Velero with ``./install/install-app.sh velero`` (only if you have configured it in
- :ref:`backups-with-velero`).
+- Velero with ``./install/install-app.sh velero``
+ (only if you have configured it in :ref:`backups-with-velero`).
When the installation scripts complete, the application installation may still
be running on the Stackspin cluster. You can monitor the progress by running
@@ -200,7 +201,7 @@ Once the installation has been completed, you can log in on
https://dashboard.stackspin.example.org (as always: replace
*stackspin.example.org* with your domain). To get your login details, run:
-.. code:: bash
+.. code:: sh
$ python -m stackspin stackspin.example.org admin-credentials
diff --git a/docs/troubleshooting.rst b/docs/troubleshooting.rst
index e1e02c09fc05871e74aa918f999c42b70aad2f11..58bf7f3aea1e1fa368008a2d3ca27489b33c367c 100644
--- a/docs/troubleshooting.rst
+++ b/docs/troubleshooting.rst
@@ -5,16 +5,15 @@ If you run into problems, there are a few things you can do to research the
problem. This document describes what you can do.
.. note::
- ``cluster$`` indicates that the commands should be run as root on your
- Stackspin machine. All other commands need to be run on your *provisioning
- machine*.
+ **cluster$** indicates that the commands should be run as root on your Stackspin machine.
+ All other commands need to be run on your *provisioning machine*.
-**We would love to hear from you!** If you have problems, please create an issue
-in our `issue tracker
-<https://open.greenhost.net/groups/stackspin/-/issues>`__ or reach out as
-described on our `contact page <https://stackspin.net/contact.html>`__. We
-want to be in communication with our users, and we want to help you if you run
-into problems.
+**We would love to hear from you!**
+If you have problems, please create an issue
+in our `issue tracker <https://open.greenhost.net/groups/stackspin/-/issues>`__
+or reach out as described on our `contact page <https://stackspin.net/contact.html>`__.
+We want to be in communication with our users,
+and we want to help you if you run into problems.
Known issues
------------
@@ -28,11 +27,11 @@ issue, please open a new one so we can solve the problems you encounter.
Run the CLI tests
-----------------
-To get an overall status of your cluster you can run the tests from the
-command line.
-
-There are two types of tests: `testinfra <https://testinfra.readthedocs.io/en/latest/>`__
-tests, and `Taiko <https://taiko.dev>`__ tests.
+To get an overall status of your cluster,
+run the tests from the command line.
+There are two types of tests:
+`testinfra <https://testinfra.readthedocs.io/en/latest/>`__ tests verify the setup,
+`Taiko <https://taiko.dev>`__ tests check the webservice availability.
Testinfra tests
~~~~~~~~~~~~~~~
@@ -45,32 +44,31 @@ The clearbox tests run on the Stackspin host and check i.e. if docker is install
in the right version etc. Our testinfra tests are a combination of blackbox and
clearbox tests.
-First, enter the `test` directory in the Git repository **on your provisioning
-machine**.
+First, enter the *test* directory in the Git repository
+**on your provisioning machine**.
-.. code:: bash
+.. code:: sh
- cd test
+ cd test
-To run the test against your cluster, first export the ``CLUSTER_DIR``
-environment variable with the location of your cluster config directory (replace
-``stackspin.example.org`` with your cluster name). Then export the
-``KUBECONFIG`` variable that lets ``kubectl`` talk to your cluster. Thirdly,
-we'll export a variable that contains the password for the Prometheus HTTP
-endpoint.
+To run the test against your cluster, first export the *CLUSTER_DIR*
+environment variable with the location of your cluster config directory
+(replace *stackspin.example.org* with your cluster name):
+Then export the *KUBECONFIG* variable that lets ``kubectl`` talk to your cluster.
+Thirdly, export a variable that contains the password for the Prometheus HTTP endpoint.
-.. code:: bash
+.. code:: sh
- export CLUSTER_DIR="${PWD}/../clusters/stackspin.example.org"
- export KUBECONFIG=${CLUSTER_DIR}/kube_config_cluster.yml
- export BASIC_AUTH_PW=$(kubectl get secret -n stackspin stackspin-prometheus-basic-auth --template '{{ .data.pass }}' | base64 -d)
+ export CLUSTER_DIR="${PWD}/../clusters/stackspin.example.org"
+ export KUBECONFIG=${CLUSTER_DIR}/kube_config_cluster.yml
+ export BASIC_AUTH_PW=$(kubectl get secret -n stackspin stackspin-prometheus-basic-auth --template '{{ .data.pass }}' | base64 -d)
Run all tests
'''''''''''''
-.. code:: bash
+.. code:: sh
- py.test -s --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*'
+ py.test -s --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*'
Test all applications
'''''''''''''''''''''
@@ -85,15 +83,15 @@ This will check for:
These tests includes all optional applications and will fail for optional
applications that are not installed.
-.. code:: bash
+.. code:: sh
- pytest -s -m 'app' --connection=ansible --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*'
+ pytest -s -m 'app' --connection=ansible --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*'
Tests a specific application
''''''''''''''''''''''''''''
-.. code:: bash
+.. code:: sh
pytest -s -m 'app' --app="wordpress" --connection=ansible --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*'
@@ -101,47 +99,47 @@ Tests a specific application
Known Issues
''''''''''''
-The default ssh backend for testinfra tests is ``paramiko``, which doesn't work
-out of the box. It fails to connect to the host because the ``ed25519`` hostkey
-was not verified. Therefore we need to force plain ``ssh://`` with either
-``connection=ssh`` or ``--hosts=ssh://…``
+The default ssh backend for testinfra tests is *paramiko*,
+which does not work out of the box.
+It fails to connect to the host because the *ed25519* hostkey is not verified.
+Therefore we need to force plain *ssh://* with either ``connection=ssh`` or ``--hosts=ssh://…``
Taiko tests
~~~~~~~~~~~
-Taiko tests run in a browser and test if all the interfaces are up
-and running and correctly connected to each other. They are integrated in the
-`stackspin` CLI command suite.
+Taiko tests run in a browser to test if all the interfaces are up and running
+and correctly connected to each other.
+They are integrated in the ``stackspin`` CLI command suite.
Prerequisites
'''''''''''''
Install `Taiko <https://taiko.dev>`__ on your provisioning machine:
-.. code:: bash
+.. code:: sh
- npm install -g taiko
+ npm install -g taiko
Run Taiko tests
'''''''''''''''
To run all Taiko tests, run the following command in this repository:
-.. code:: bash
+.. code:: sh
- python -m stackspin CLUSTERNAME test
+ python -m stackspin CLUSTERNAME test
-To learn more about the `test` subcommand, run:
+To learn more about the ``test`` subcommand, run:
-.. code:: bash
+.. code:: sh
- python -m stackspin CLUSTERNAME test --help
+ python -m stackspin CLUSTERNAME test --help
You can also only run a Taiko test for a specific application, i.e.:
-.. code:: bash
+.. code:: sh
- python -m stackspin CLUSTERNAME test --taiko-tags nextcloud
+ python -m stackspin CLUSTERNAME test --taiko-tags nextcloud
Advanced usage
~~~~~~~~~~~~~~
@@ -151,40 +149,40 @@ Testinfra tests
Specify host manually:
-.. code:: bash
+.. code:: sh
- py.test -s --hosts='ssh://root@example.stackspin.net'
+ py.test -s --hosts='ssh://root@example.stackspin.net'
-Run only tests tagged with `prometheus`:
+Run only tests tagged with *prometheus*:
-.. code:: bash
+.. code:: sh
- py.test -s --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*' -m prometheus
+ py.test -s --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*' -m prometheus
Run cert test manually using the ansible inventory file:
-.. code:: bash
+.. code:: sh
- py.test -s --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*' -m certs
+ py.test -s --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*' -m certs
Run cert test manually against a different cluster, not configured in any
ansible inventory file, either by using pytest:
-.. code:: bash
+.. code:: sh
- FQDN='example.stackspin.net' py.test -sv -m 'certs'
+ FQDN='example.stackspin.net' py.test -sv -m 'certs'
or directly:
-.. code:: bash
+.. code:: sh
- FQDN='example.stackspin.net' pytest/test_certs.py
+ FQDN='example.stackspin.net' pytest/test_certs.py
Running Testinfra tests with local gitlab-runner docker executor
Export the following environment variables like this:
-.. code:: bash
+.. code:: sh
export CI_REGISTRY_IMAGE='open.greenhost.net:4567/stackspin/stackspin'
export SSH_PRIVATE_KEY="$(cat ~/.ssh/id_ed25519_stackspin_ci)"
@@ -192,19 +190,19 @@ Export the following environment variables like this:
then:
-.. code:: bash
+.. code:: sh
gitlab-runner exec docker --env CI_REGISTRY_IMAGE="$CI_REGISTRY_IMAGE" --env SSH_PRIVATE_KEY="$SSH_PRIVATE_KEY" --env COSMOS_API_TOKEN="$COSMOS_API_TOKEN" bootstrap
Advanced Taiko tests
''''''''''''''''''''
-If you want to use Taiko without invoking the stackspin CLI, go to the
-``test/taiko`` directory and run:
+If you want to use Taiko without invoking the stackspin CLI,
+go to the *test/taiko* directory and run:
For nextcloud & onlyoffice tests:
-.. code:: bash
+.. code:: sh
export DOMAIN='stackspin.example.net'
export SSO_USERNAME='user1'
@@ -212,8 +210,9 @@ For nextcloud & onlyoffice tests:
export TAIKO_TESTS='nextcloud'
taiko --observe taiko-tests.js
-You can replace ``nextcloud`` with ``grafana`` or ``wordpress`` to test the
-other applications, or with ``all`` to test all applications.
+You can replace *nextcloud* with *grafana* or *wordpress*
+to test the other applications,
+or with *all* to test all applications.
SSH access
----------
@@ -233,11 +232,12 @@ on the VPS:
Using kubectl to debug your cluster
-----------------------------------
-You can use ``kubectl``, the Kubernetes control program, to find and manipulate
-your Kubernetes cluster. Once you have installed ``kubectl``, to get access to
-your cluster with the Stackspin CLI:
+You can use ``kubectl``, the Kubernetes control program,
+to find and manipulate your Kubernetes cluster.
+Once you have installed ``kubectl``,
+to get access to your cluster with the Stackspin CLI:
-.. code:: bash
+.. code:: sh
$ python -m stackspin stackspin.example.org info
@@ -245,39 +245,41 @@ Look for these lines:
.. code::
- To use kubectl with this cluster, copy-paste this in your terminal:
- export KUBECONFIG=/home/you/projects/stackspin/clusters/stackspin.example.org/kube_config_cluster.yml
+ To use kubectl with this cluster, copy-paste this in your terminal:
+ export KUBECONFIG=/home/you/projects/stackspin/clusters/stackspin.example.org/kube_config_cluster.yml
-Copy the whole ``export`` line into your terminal. In *the same terminal
-window*, ``kubectl`` will connect to your cluster.
+Copy the whole ``export`` line into your terminal.
+**In the same terminal window**, ``kubectl`` will from now on connect to your cluster.
HTTPS certificates
------------------
-Stackspin uses `cert-manager <https://docs.cert-manager.io/en/latest/>`__ to
-automatically fetch `Let's Encrypt <https://letsencrypt.org/>`__ certificates
-for all deployed services. If you experience invalid SSL certificates, i.e. your
-browser warns you when visiting Zulip (https://zulip.stackspin.example.org),
+Stackspin uses `cert-manager <https://docs.cert-manager.io/en/latest/>`__
+to automatically fetch `Let's Encrypt <https://letsencrypt.org/>`__ certificates
+for all deployed services.
+If you experience invalid SSL certificates,
+i.e. your browser warns you when visiting Zulip (https://zulip.stackspin.example.org),
a useful resource for troubleshooting is the official cert-manager
-`Troubleshooting Issuing ACME Certificates
-<https://cert-manager.io/docs/faq/acme/>`__ documentation. First, try this:
+`Troubleshooting Issuing ACME Certificates <https://cert-manager.io/docs/faq/acme/>`__ documentation.
+First, try this:
-In this example we fix a failed certificate request for
-*https://chat.stackspin.example.org*. We will start by checking if ``cert-manager``
-is set up correctly.
+In this example we fix a failed certificate request for *https://chat.stackspin.example.org*.
+We will start by checking if ``cert-manager`` is set up correctly.
Is your cluster using the live ACME server?
+~~~~~~~
-.. code:: bash
+.. code:: sh
$ kubectl get clusterissuers -o yaml | grep 'server:'
-Should return ``server: https://acme-v02.api.letsencrypt.org/directory`` and not
-something with the word *staging* in it.
+Should return `server: https://acme-v02.api.letsencrypt.org/directory`
+and not something with the word *staging* in it.
-Are all cert-manager pods in the `stackspin` namespace in the `READY` state ?
+Are all cert-manager pods in the *stackspin* namespace in the *READY* state?
+~~~~~~~
-.. code:: bash
+.. code:: sh
$ kubectl -n cert-manager get pods
@@ -287,30 +289,31 @@ can also check the status of your certificates by running:
This returns all the certificates for all applications on your system. The
command includes example output of healthy certificates.
-.. code:: bash
+.. code:: sh
$ kubectl get certificates -A
- NAMESPACE NAME READY SECRET AGE
- stackspin hydra-public.tls True hydra-public.tls 14d
- stackspin single-sign-on-userpanel.tls True single-sign-on-userpanel.tls 14d
- stackspin-apps stackspin-nextcloud-files True stackspin-nextcloud-files 14d
- stackspin-apps stackspin-nextcloud-office True stackspin-nextcloud-office 14d
- stackspin grafana-tls True grafana-tls 13d
- stackspin alertmanager-tls True alertmanager-tls 13d
- stackspin prometheus-tls True prometheus-tls 13d
+ NAMESPACE NAME READY SECRET AGE
+ stackspin hydra-public.tls True hydra-public.tls 14d
+ stackspin single-sign-on-userpanel.tls True single-sign-on-userpanel.tls 14d
+ stackspin-apps stackspin-nextcloud-files True stackspin-nextcloud-files 14d
+ stackspin-apps stackspin-nextcloud-office True stackspin-nextcloud-office 14d
+ stackspin grafana-tls True grafana-tls 13d
+ stackspin alertmanager-tls True alertmanager-tls 13d
+ stackspin prometheus-tls True prometheus-tls 13d
-If there are problems, you can check for the specific ``certificaterequests``:
+If there are problems, you can check for the specific *certificaterequests*:
-.. code:: bash
+.. code:: sh
$ kubectl get certificaterequests -A
-If you still need more information, you can dig into the logs of the
-``cert-manager`` pod:
+For even more information, inspect the logs of the *cert-manager* pod:
+
+.. code:: sh
$ kubectl -n stackspin logs -l "app.kubernetes.io/name=cert-manager"
-You can `grep` for your cluster domain or for any specific subdomain to narrow
+You can ``grep`` for your cluster domain or for any specific subdomain to narrow
down results.
Example
@@ -318,7 +321,7 @@ Example
Query for failed certificates, -requests, challenges or orders:
-.. code:: bash
+.. code:: sh
$ kubectl get --all-namespaces certificate,certificaterequest,challenge,order | grep -iE '(false|pending)'
stackspin-apps certificate.cert-manager.io/stackspin-zulip False stackspin-zulip 15h
@@ -326,32 +329,32 @@ Query for failed certificates, -requests, challenges or orders:
stackspin-apps challenge.acme.cert-manager.io/stackspin-zulip-2045852889-1775447563-837515681 pending chat.stackspin.example.org 15h
stackspin-apps order.acme.cert-manager.io/stackspin-zulip-2045852889-1775447563 pending 15h
-We see that the zulip certificate resources are in a bad state since 15h.
+We see that the zulip certificate resources have been in a bad state for 15 hours.
Show certificate resource status message:
-.. code:: bash
+.. code:: sh
$ kubectl -n stackspin-apps get certificate stackspin-zulip -o jsonpath="{.status.conditions[*]['message']}"
Waiting for CertificateRequest "stackspin-zulip-2045852889" to complete
-We see that the `certificate` is waiting for the `certificaterequest`, lets
-query its status message:
+We see that the `certificate` is waiting for the `certificaterequest`,
+let\'s query its status message:
-.. code:: bash
+.. code:: sh
$ kubectl -n stackspin-apps get certificaterequest stackspin-zulip-2045852889 -o jsonpath="{.status.conditions[*]['message']}"
Waiting on certificate issuance from order stackspin-apps/stackspin-zulip-2045852889-1775447563: "pending"
Show the related order resource and look at the status and events:
-.. code:: bash
+.. code:: sh
$ kubectl -n stackspin-apps describe order stackspin-zulip-2045852889-1775447563
Show the failed challenge resource reason:
-.. code:: bash
+.. code:: sh
$ kubectl -n stackspin-apps get challenge stackspin-zulip-2045852889-1775447563-837515681 -o jsonpath='{.status.reason}'
Waiting for http-01 challenge propagation: wrong status code '503', expected '200'
@@ -359,46 +362,54 @@ Show the failed challenge resource reason:
In this example, deleting the challenge fixed the issue and a proper certificate
could get fetched:
-.. code:: bash
+.. code:: sh
$ kubectl -n stackspin-apps delete challenges.acme.cert-manager.io stackspin-zulip-2045852889-1775447563-837515681
Application installation or upgrade failures
--------------------------------------------
-Application installations and upgrades are managed by `flux`_. Flux uses
-``helm-controller`` to install and upgrade applications with ``helm charts``.
+Application installations and upgrades are managed by `flux`_.
+Flux uses ``helm-controller`` to install and upgrade applications
+with `helm charts <https://helm.sh/docs/topics/charts/>`__.
+
+An application installed with Flux consists of a *kustomization*.
+This resource defines where the information about the application is stored in our Git repository.
+The *kustomization* contains a *helmrelease*,
+which is an object that represents an installation of a Helm chart.
+Read more about the difference between *kustomizations* and *helmreleases*
+in the `flux documentation <https://fluxcd.io/docs>`__.
-An application installed with Flux consists of a ``kustomization``. This is a
-resource that defines where the information about the application is stored in
-our Git repository. The ``kustomization`` contains a ``helmrelease``, which is
-an object that represents an installation of a Helm chart. Read more about the
-difference between ``kustomizations`` and ``helmreleases`` in the `flux
-documentation <https://fluxcd.io/docs>`__
+To find out if all *kustomizations* have been applied correctly,
+run the following flux command in your cluster or from the provisioning machine:
-To find out if all ``kustomizations`` have been applied correctly, run the
-following flux command in your cluster:
+.. code:: sh
-.. code:: bash
+ cluster$ flux get kustomizations -A
- cluster$ flux get kustomizations -A
+If there is an issue, use ``kubectl`` to inspect the respective service,
+for example nginx:
-If all your ``kustomizations`` are in a ``Ready`` state, take a look at your
-``helmreleases``:
+.. code:: sh
-.. code:: bash
+ kubectl describe helmrelease -n stackspin nginx | less
- cluster$ flux get helmreleases -A
+If all your *kustomizations* are in a *Ready* state,
+take a look at your *helmreleases*:
-Often, you can resolve complications with ``kustomizations`` or ``helmreleases``
+.. code:: sh
+
+ cluster$ flux get helmreleases -A
+
+Often, you can resolve complications with *kustomizations* or *helmreleases*
by telling Flux to *reconcile* them:
-.. code:: bash
+.. code:: sh
- cluster$ flux reconcile helmrelease nextcloud
+ cluster$ flux reconcile helmrelease nextcloud
-Will make sure that the Nextcloud ``helmrelease`` gets brought into a state that
-our Stackspin wants it to be in.
+This brings the Nextcloud *helmrelease* into a state
+that our Stackspin wants it to be in.
Common installation failures
@@ -411,37 +422,38 @@ When you execute ``flux get kustomization`` and you see this error:
.. code::
- var substitution failed for 'kube-prometheus-stack': YAMLToJSON: yaml: line 32: found character that cannot start any token
+ var substitution failed for 'kube-prometheus-stack': YAMLToJSON: yaml: line 32: found character that cannot start any token
That can mean that one of your values contains a double quote (``"``) or that
-you quoted a value in ``.flux.env`` during the :ref:`flux_config`. Make sure
-that ``.flux.env`` does not contain any values that are quoted.
+you quoted a value in `.flux.env` during the :ref:`flux_config`. Make sure
+that `.flux.env` does not contain any values that are quoted.
-If you need to change ``.flux.env``, run the following commands:
+If you need to change `.flux.env`, run the following commands:
-.. code:: bash
+.. code:: sh
$ kubectl apply -k $CLUSTER_DIR
-Afterwards, you can speed up the process that fixes your ``kustomization``, by
-running ``flux reconcile kustomization kube-prometheus-stack`` (replace
-``kube-prometheus-stack`` with the kustomization is mentioned in the error
-message).
+Afterwards, you can speed up the process that fixes your *kustomization*,
+by running ``flux reconcile kustomization kube-prometheus-stack``
+(replace *kube-prometheus-stack* with the *kustomization* mentioned in the error message).
Purge Stackspin and install from scratch
----------------------------------------
-If ever things fail beyond possible recovery, here's how to completely purge an
-Stackspin installation in order to start from scratch:
-
.. warning::
- **You will lose all your data!** This completely destroys Stackspin and
- takes everything offline. If you chose to do this, you will need to
- re-install Stackspin and make sure that your data is stored somewhere
- other than the VPS that runs Stackspin.
+ You will lose all your data!
+ This completely destroys Stackspin and takes everything offline.
+ If you chose to do this, you will need to re-install Stackspin
+ and make sure that your data is stored somewhere
+ other than the VPS that runs Stackspin.
+
+If things ever fail beyond possible recovery,
+here is how to completely purge a Stackspin installation
+to start from scratch:
-.. code:: bash
+.. code:: sh
cluster$ /usr/local/bin/k3s-killall.sh
cluster$ systemctl disable k3s