diff --git a/docs/conf.py b/docs/conf.py index 85ff4759ee5feef79897fd5c767ad47695656380..c41798da51e365d7acf639dd8098edf497d0c0ea 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -61,7 +61,6 @@ html_static_path = ['_static'] # 'contents' master_doc = 'index' - # https://www.sphinx-doc.org/en/master/usage/extensions/autosectionlabel.html # # Suppress autosectionlabel extension warnings about duplicate labels, i.e. @@ -69,5 +68,7 @@ master_doc = 'index' # docs/usage.rst:105: WARNING: duplicate label wordpress, other instance in docs/testing.rst # # https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-suppress_warnings -suppress_warnings = ['autosectionlabel.*'] autosectionlabel_prefix_document = True +autosectionlabel_maxdepth = 5 + +suppress_warnings = ['autosectionlabel.*'] diff --git a/docs/index.rst b/docs/index.rst index fe6d38116522326477fd536737b7293f85148c17..082dbdae91af60c29b0bbc0986ae42bcfd2eab43 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -41,6 +41,7 @@ For more information, go to `the Stackspin website`_. :maxdepth: 2 :caption: Administration + logging maintenance upgrading customizing diff --git a/docs/logging.rst b/docs/logging.rst new file mode 100644 index 0000000000000000000000000000000000000000..6f2e4d779a981e8352fd390a17552f0442621b95 --- /dev/null +++ b/docs/logging.rst @@ -0,0 +1,179 @@ +Logging +======= + +Logs from pods and containers can be read in different ways: + +- In the cluster filesystem at ``/var/log/pods/`` or + ``/var/logs/containers/``. +- Using `kubectl logs`_ +- Querying aggregated logs with Grafana, see below. + +Central log aggregation +----------------------- + +We use `Promtail`_, `Loki`_ and `Grafana`_ for easy access of aggregated +logs. The `Loki documentation`_ is a good starting point how this setup +works. +There are two ways of viewing aggregated logs: + +* Via the Grafana web interface +* Using the ``logcli`` command line tool + + +Viewing logs in Grafana +~~~~~~~~~~~~~~~~~~~~~~~ + +The `Using Loki in Grafana`_ gets you started with querying +your cluster logs with Grafana. +You will find the Loki Grafana integration on your cluster at +https://grafana.stackspin.example.org/explore together with some generic query +examples. + +Please follow :ref:`logging:LogQL query examples` for more LogQL query examples. + +Query logs with logcli +~~~~~~~~~~~~~~~~~~~~~~~ + +Please refer to `logcli`_ for installing ``logcli`` on your Laptop. +The create a port-forwarding to your cluster using the ``kubectl`` tool: + +.. code:: bash + + kubectl -n stackspin port-forward pod/loki-0 3100 + +In another terminal you can now use ``logcli`` to query ``loki`` like this: + +.. code:: bash + + logcli query '{app=~".+"}' + +Please follow :ref:`logging:LogQL query examples` for more LogQL query examples. + +Search older messages (in this case the last week and limit the output to 2000 +lines): + +.. code:: bash + + logcli query --since=168h --limit=2000 --forward '{app="helm-controller"}' + +LogQL query examples +~~~~~~~~~~~~~~~~~~~~ + +Please also refer to the `LogQL documentation`_ and the +`log queries documentation`_ . + +Query all aggregated logs (unfortunatly we can’t find a better way of +doing this since LogQL always expects a stream label to get queried): + +.. code:: PromQL + + {app=~".+"} + +Query all logs for a keyword: + +.. code:: + + {app=~".+"} |= "error" + +Query all k8s apps for errors using a regular expression: + +.. code:: + + {app=~".+"} |~ `(error|fail|exception|fatal)` + +Flux +^^^^ + +`Flux`_ is responsible for installing applications. It uses four +controllers: + +- ``source-controller`` that tracks Helm and Git repositories like + https://open.greenhost.net/stackspin/stackspin for updates. +- ``kustomize-controller`` to deploy ``kustomizations`` that often + install ``helmreleases``. +- ``helm-controller`` to deploy the ``helmreleases``. +- ``notification-controller`` that is responsible for inbound and + outbound flux messages + +Query all messages from the ``source-controller``: + +.. code:: PromQL + + {app="source-controller"} + +Query all messages from ``flux`` and ``helm-controller``: + +.. code:: PromQL + + {app=~"(source-controller|helm-controller)"} + +``helm-controller`` messages containing ``wordpress``: + +.. code:: PromQL + + '{app = "helm-controller"} |= "wordpress"' + +``helm-controller`` messages containing ``wordpress`` without +``unchanged`` events (to only show the installation messages): + +.. code:: PromQL + + '{app = "helm-controller"} |= "wordpress" != "unchanged"' + +Filter out redundant ``helm-controller`` messages: + +.. code:: PromQL + + '{app="helm-controller"} !~ `(unchanged|event=refreshed|method=Sync|component=checkpoint)`' + + +Cert-manager +^^^^^^^^^^^^ + +Cert manager is responsible for requesting Let’s Encrypt TLS +certificates. + +Query ``cert-manager`` messages containing ``chat``: + +.. code:: PromQL + + '{app="cert-manager"} |= "chat"' + +Hydra +^^^^^ + +Hydra is the single sign-on system. + +Show only warnings and errors from ``hydra``: + +.. code:: PromQL + + {container_name="hydra"} != "level=info" + +Debug oauth2 single sign-on with zulip: + +.. code:: PromQL + + {container_name=~"(hydra|zulip)"} + +Etc +^^^ + +Query kubernetes events processed by the ``eventrouter`` app containing +``warning``: + +.. code:: PromQL + + '{app="eventrouter"} |~ "warning"' + + +.. _kubectl logs: https://kubernetes.io/docs/concepts/cluster-administration/logging +.. _Promtail: https://grafana.com/docs/loki/latest/clients/promtail/ +.. _Loki: https://grafana.com/oss/loki/ +.. _Grafana: https://grafana.com/ +.. _Loki documentation: https://grafana.com/docs/loki/latest/ +.. _Using Loki in Grafana: https://grafana.com/docs/grafana/latest/datasources/loki +.. _logcli: https://grafana.com/docs/loki/latest/getting-started/logcli/ +.. _LogQL documentation: https://grafana.com/docs/loki/latest/logql +.. _log queries documentation: https://grafana.com/docs/loki/latest/logql/log_queries/ +.. _Flux: https://fluxcd.io/ diff --git a/docs/maintenance.rst b/docs/maintenance.rst index 81d002615a8ef4557e7ee267b716ae26ad2494f5..544fb0c92e13d358daa1329b2a6bd7861e3bc66d 100644 --- a/docs/maintenance.rst +++ b/docs/maintenance.rst @@ -1,133 +1,6 @@ Maintenance =========== -Logging -------- - -Logs from pods and containers can be read in different ways: - -- In the cluster filesystem at ``/var/log/pods/`` or - ``/var/logs/containers/``. -- Using `kubectl logs`_ -- Querying aggregated logs with Grafana, see below. - -Central log aggregation ------------------------ - -We use `Promtail`_, `Loki`_ and `Grafana`_ for easy access of aggregated -logs. The `Loki documentation`_ is a good starting point how this setup -works, and the `Using Loki in Grafana`_ gets you started with querying -your cluster logs with Grafana. - -You will find the Loki Grafana integration on your cluster at -https://grafana.stackspin.example.org/explore together with some generic query -examples. - -LogQL query examples -~~~~~~~~~~~~~~~~~~~~ - -Please also refer to the `LogQL documentation`_. - -Query all aggregated logs (unfortunatly we can’t find a better way of -doing this since LogQL always expects a stream label to get queried): - -.. code:: bash - - logcli query '{foo!="bar"}' - -Query all logs for a keyword: - -.. code:: bash - - logcli query '{foo!="bar"} |= "error"' - -Query all k8s apps for errors using a regular expression: - -.. code:: bash - - logcli query '{job=~".*"} |~ "error|fail|exception|fatal"' - -Flux -^^^^ - -`Flux`_ is responsible for installing applications. It uses four -controllers: - -- ``source-controller`` that tracks Helm and Git repositories like - https://open.greenhost.net/stackspin/stackspin for updates. -- ``kustomize-controller`` to deploy ``kustomizations`` that often - install ``helmreleases``. -- ``helm-controller`` to deploy the ``helmreleases``. -- ``notification-controller`` that is responsible for inbound and - outbound flux messages - -Query all messages from the ``source-controller``: - -.. code:: bash - - {app="source-controller"} - -Query all messages from ``flux`` and ``helm-controller``: - -.. code:: bash - - {app=~"(source-controller|helm-controller)"} - -``helm-controller`` messages containing ``wordpress``: - -.. code:: bash - - {app = "helm-controller"} |= "wordpress" - -``helm-controller`` messages containing ``wordpress`` without -``unchanged`` events (to only show the installation messages): - -.. code:: bash - - {app = "helm-controller"} |= "wordpress" != "unchanged" - -Filter out redundant ``helm-controller`` messages: - -.. code:: bash - - { app = "helm-controller" } !~ "(unchanged | event=refreshed | method=Sync | component=checkpoint)" - -Debug oauth2 single sign-on with zulip: - -.. code:: bash - - {container_name=~"(hydra|zulip)"} - -Query kubernetes events processed by the ``eventrouter`` app containing -``warning``: - -.. code:: bash - - logcli query '{app="eventrouter"} |~ "warning"' - -Cert-manager -^^^^^^^^^^^^ - -Cert manager is responsible for requesting Let’s Encrypt TLS -certificates. - -Query ``cert-manager`` messages containing ``chat``: - -.. code:: bash - - {app="cert-manager"} |= "chat" - -Hydra -^^^^^ - -Hydra is the single sign-on system. - -Show only warnings and errors from ``hydra``: - -.. code:: bash - - {container_name="hydra"} != "level=info" - Backup ------ @@ -204,14 +77,6 @@ following command that will apply the changes to all installed kustomizations: flux get -A kustomizations --no-header | awk -F' ' '{system("flux reconcile -n " $1 " kustomization " $2)}' -.. _kubectl logs: https://kubernetes.io/docs/concepts/cluster-administration/logging -.. _Promtail: https://grafana.com/docs/loki/latest/clients/promtail/ -.. _Loki: https://grafana.com/oss/loki/ -.. _Grafana: https://grafana.com/ -.. _Loki documentation: https://grafana.com/docs/loki/latest/ -.. _Using Loki in Grafana: https://grafana.com/docs/grafana/latest/datasources/loki -.. _LogQL documentation: https://grafana.com/docs/loki/latest/logql -.. _Flux: https://fluxcd.io/ .. _Velero’s documentation: https://velero.io/docs/v1.4/ .. _reach out to us: https://stackspin.net/contact.html .. _taints: https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/ diff --git a/docs/requirements.in b/docs/requirements.in index 16f51618fa5d19bc02c2bb909ac79eebaca227f4..957a65b28f6b240a182ba3485567cf2826f5b153 100644 --- a/docs/requirements.in +++ b/docs/requirements.in @@ -5,7 +5,7 @@ # Please add developer dependencies which are not needed to install # Stackspin to requirements-dev.txt! # +recommonmark sphinx sphinx-design sphinx-rtd-theme -recommonmark diff --git a/docs/requirements.txt b/docs/requirements.txt index f39411896b309d080bd07b6b01155ce8d45fc009..dfbd6ae61cc2f23d6c210d33e9d0e8fd46c2a70a 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -14,7 +14,7 @@ charset-normalizer==2.0.9 # via requests commonmark==0.9.1 # via recommonmark -docutils==0.18.1 +docutils==0.17.1 # via # recommonmark # sphinx @@ -30,8 +30,10 @@ markupsafe==2.0.1 packaging==21.3 # via sphinx pygments==2.10.0 - # via sphinx -pyparsing==2.4.7 + # via + # -r requirements.in + # sphinx +pyparsing==3.0.6 # via packaging pytz==2021.3 # via babel diff --git a/docs/troubleshooting.rst b/docs/troubleshooting.rst index 10fee5a0593bf7a68b212bd64bd21ed2e5a9a6ba..5db3248aaacc9c95c42371490178508d287f1b74 100644 --- a/docs/troubleshooting.rst +++ b/docs/troubleshooting.rst @@ -95,7 +95,7 @@ Tests a specific application Known Issues '''''''''''' -The Default ssh backend for testinfra tests is ``paramiko``, which doesn't work +The default ssh backend for testinfra tests is ``paramiko``, which doesn't work out of the box. It fails to connect to the host because the ``ed25519`` hostkey was not verified. Therefore we need to force plain ssh:// with either ``connection=ssh`` or ``--hosts=ssh://…`` @@ -190,8 +190,8 @@ then: gitlab-runner exec docker --env CI_REGISTRY_IMAGE="$CI_REGISTRY_IMAGE" --env SSH_PRIVATE_KEY="$SSH_PRIVATE_KEY" --env COSMOS_API_TOKEN="$COSMOS_API_TOKEN" bootstrap -Taiko tests -''''''''''' +Advanced Taiko tests +'''''''''''''''''''' If you want to use Taiko without invoking the stackspin CLI, go to the ``test/taiko`` directory and run: diff --git a/docs/upgrading.rst b/docs/upgrading.rst index 76824490a1ab9775af156ed9904997cfa2851965..9155d76909c906b3c35ab166d208aa2cb804ef20 100644 --- a/docs/upgrading.rst +++ b/docs/upgrading.rst @@ -96,10 +96,9 @@ renamed from ``oas`` to ``stackspin``. You can choose from these options: Rocket.Chat ~~~~~~~~~~~ -We replaced Rocket.Chat with `Zulip <https://zulip.com>`__ in this release. -If you want to migrate your Rocket.Chat data to your new Zulip installation -please refer to -`Import from Rocket.Chat https://api.zulip.com/help/import-from-rocketchat`__. +We replaced Rocket.Chat with `Zulip`_ in this release. +If you want to migrate your Rocket.Chat data to your new `Zulip`_ installation +please refer to `Import from Rocket.Chat`_. Monitoring ~~~~~~~~~~ @@ -131,13 +130,12 @@ from v0.6 to v0.7!** .. note:: Before you start, please ensure that you have the right ``yq`` tool installed, because you will need it later. There are two very different versions of - ``yq``. The one you need is the go based `yq from Mike Farah - <http://mikefarah.github.io/yq>`_, which installs the same binary name ``yq`` - as the `python-yq <https://github.com/kislyuk/yq>`_, while both have different - command sets. + ``yq``. The one you need is the go based `yq from Mike Farah`_, + which installs the same binary name as the `python-yq`_ one, while both have + different command sets. The yq needed here can be installed by running ``sudo snap install yq``, - ``brew install yq`` or with other methods from the `yq installation - instructions <http://mikefarah.github.io/yq/#install>`_. + ``brew install yq`` or with other methods from the + `yq installation instructions`_. If you're unsure which ``yq`` you have installed, look at the output of ``yq --help`` and make sure ``eval`` shows up under ``Available Commands:``. @@ -394,3 +392,8 @@ intervention. .. _reach out to us: https://openappstack.net/contact.html .. _Flux: https://fluxcd.io +.. _yq from Mike Farah: https://mikefarah.github.io/yq +.. _yq installation instructions: https://mikefarah.github.io/yq/#install +.. _python-yq: https://github.com/kislyuk/yq +.. _Zulip: https://zulip.com +.. _Import from Rocket.Chat: https://api.zulip.com/help/import-from-rocketchat diff --git a/docs/usage.rst b/docs/usage.rst index b618c09fa914471f1a07cdfebd594ce35e9c9c19..d7ea1299e7f7a1b03872b917062dd4f63625b707 100644 --- a/docs/usage.rst +++ b/docs/usage.rst @@ -24,7 +24,7 @@ them access to applications, take a look at the `user panel documentation .. note:: If you don't see applications, make sure you have installed at least one - optional application in :ref:`additional_apps` of the installation procedure. + optional application in :ref:`install_additional_apps` of the installation procedure. For creating users follow the `user creation documentation <https://docs.stackspin.net/projects/user-panel/en/latest/#creating-a-new-user>`_.