Add customizing docs

Closes: #285

docs/README.md

@@ -4,7 +4,9 @@ This folder contains the documentation. You can find the documentation at
 https://openappstack.readthedocs.io/. It is also possible to build the
 documentation, using Sphinx:
 
-```
-pip install sphinx recommonmark sphinx_rtd_theme
-make html
-```
+    pip install sphinx recommonmark sphinx_rtd_theme
+    make html
+
+Then point your browser to the newly generated pages:
+
+    firefox _build/html/index.html

docs/advanced_installation.rst

+=====================
+Advanced installation
+=====================
+
+In this guide we show how to customize your cluster installation, i.e. if you
+want to install additional applications, or change the configuration of extisting
+apps installed by OAS this is the right place. Customizing other parts of your
+cluster might be possible but not yet covered by this guide.
+As the name of this guide implies, it is written for users with advanced knowledge
+of the tools behind Openappstack, most importantly: Kubernetes, Helm, Ansible and Flux.
+
+.. warning::
+
+  Customizing your OAS cluster could break your cluster in a way that it's not
+  easy to recover. Please be aware of the potential risk when proceeding.
+
+
+Prerequisites
+=============
+
+* A functional OAS cluster installed following the
+  `Openappstack installation instructions <installation_instructions.html>`__
+
+Customize OAS applications
+==========================
+
+Apps deployed by OAS are configured using helm values from templates in
+``ansible/roles/apps/templates/settings/APPNAME.yaml``. To add new helm values
+or override existing values you can use ``APPNAME_extra_values`` in your
+``CLUSTERDIR/group_vars/all/settings.yaml``.
+Look for the correct ``APPNAME`` in ``flux/<namespace>/APPNAME.yaml``.
+
+We use the `ansible combine filter with "recursive=True" <https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#combining-hashes-dictionaries>`__
+to merge the ``APPNAME_extra_values`` variable into our default settings.
+A possible pitfall with customizing values from ansible templates is that
+you need to replace all dynamic expressions which would be templated.
+We will cover this in the example below.
+
+Example: Customize prometheus data retention
+--------------------------------------------
+
+If you want to lower the data retention value (currently 10 days)
+to lets say 5 days you need to add the following addition to
+``prometheus_extra_values`` in ``CLUSTERDIR/group_vars/all/settings.yaml``.
+(For all prometheus default values set by Openappstack see
+``ansible/roles/apps/templates/settings/prometheus.yaml``.)
+
+.. code-block:: yaml
+
+   prometheus_extra_values:
+     server:
+       retention: "5d"
+
+After this, run the
+`Openappstack installation procedure <installation_instructions.html#step-4-installation>`__
+again. Until
+`helm-operator detects changes to configmaps and secrets <https://open.greenhost.net/openappstack/openappstack/-/issues/617>`__
+by itself and updates a helm release you need to manully restart the
+helm-operator pod in order to apply the config change to prometheus:
+
+::
+
+   kubectl -n oas delete pod -l 'app=helm-operator'
+
+
+Deploying custom apps with an extra flux instance
+=================================================
+
+Openappstack uses `Flux <https://fluxcd.io>`__ to deploy and auto-update
+applications. See the `Automatic updates <design.html#automatic-updates>`__
+section in the design doc for more information abouot Flux.
+
+You can configure an extra Flux instance running on your OAS cluster,
+which can monitor your own git repo with flux definitions.
+Be aware that you only need a second Flux instance, the existing helm-operator
+will deploy the helm releases from both Flux installations.
+
+In order to do that you first need to enable ``flux-custom`` in the list of
+``enabled_applications`` in your ``settings.yml`` file. Then configure
+``flux-custom_extra_values`` to your needs (see the `Flux helm chart <https://github.com/fluxcd/flux/blob/master/chart/flux/README.md>`__
+for all possible values). An example is shown below:
+
+.. code-block:: yaml
+
+   flux-custom_extra_values:
+     # https://github.com/fluxcd/flux/tree/master/chart/flux#configuration
+     git:
+       url: git@my.git.server.org:myself/flux-config.git
+       branch: master
+       readonly: true
+       pollInterval: 1h
+     registry:
+       excludeImage: '*'
+     # If you use ssh instead of https, like in above git repo URL you need to
+     # provide the ssh hostkey pringerprint shown with
+     #   `ssh-keyscan -t ssh-rsa my.git.server.org`:
+     ssh:
+       known_hosts: 'my.git.server.org ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZ2JHLwWuixvUvx630E3fInoJKyABLPzgDH4k4dM6ptQXVfK0Hu53nhIjsCbp/i4u4GLH2B8Tv2umVh8EejvjbqsZpD4fX6PWbM131vA3sPSD4q5qJV1xkZdc4+3STrndRD02py96OtjBl/f6EJW3Upz/xWX/iiL6crp8RZzOsSn9dv/gg4Bn42G3gcNZJVgNJHO8yPAYf1fMVlDYpiKlib6Vuow8EKerqByDnLm1vbnnMHes+F6Pt1JkLzURGE83AwAUfZZTRGyzZEkXVNpcV1Pq3esg1bBAW6dfxeyzF9SJrpYkv0lbUkuBFEz9ExzWpiKe/q/O61W0u9Q6sKwev'
+     sync:
+       state: secret
+     syncGarbageCollection:
+       enabled: true
+
+I your Flux definitions in your git repo contain ssh git URLs you also
+need to configure ``helm-operator`` to trust the fingerprints of the
+servers your git repositories are hosted at.  If they are hosted on the same
+server your Flux repo is at i.e.:
+
+.. code-block:: yaml
+
+   helm_operator:
+     extra_opts: "--set git.ssh.known_hosts='my.git.server.org ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZ2JHLwWuixvUvx630E3fInoJKyABLPzgDH4k4dM6ptQXVfK0Hu53nhIjsCbp/i4u4GLH2B8Tv2umVh8EejvjbqsZpD4fX6PWbM131vA3sPSD4q5qJV1xkZdc4+3STrndRD02py96OtjBl/f6EJW3Upz/xWX/iiL6crp8RZzOsSn9dv/gg4Bn42G3gcNZJVgNJHO8yPAYf1fMVlDYpiKlib6Vuow8EKerqByDnLm1vbnnMHes+F6Pt1JkLzURGE83AwAUfZZTRGyzZEkXVNpcV1Pq3esg1bBAW6dfxeyzF9SJrpYkv0lbUkuBFEz9ExzWpiKe/q/O61W0u9Q6sKwev'
+
+After this, run the
+`Openappstack installation procedure <installation_instructions.html#step-4-installation>`__
+again.
+
+
+Using a different storage class
+===============================
+
+Openappstack uses the `local path
+provisioner <https://github.com/rancher/local-path-provisioner>`__ to
+create local path persistent volumes on the host. If you want to use a
+different storage provisioner you need to first set the setting
+``storageClass.defaultClass`` to ``false`` (see
+``ansible/roles/apps/templates/settings/local-path-provisioner.yaml``) and
+can then remove the ``local-path-provisioner`` from the list of
+``enabled_applications``
+
+After this, run the
+`Openappstack installation procedure <installation_instructions.html#step-4-installation>`__
+again.
+
+Now you can configure your Kubernetes cluster to use a different default storage
+provisioner which newly created persistent volumes can use.

docs/index.rst

@@ -27,6 +27,7 @@ For more information, go to `the OpenAppStack website`_.
    :caption: Contents:
 
    installation_instructions
+   advanced_installation
    testing_instructions
    usage
    troubleshooting