diff --git a/docs/maintenance.md b/docs/maintenance.md index 4b12a1136582aa4da211cf21acbe1a197e97215a..29b8bd815f93198f3f2135ce41db877a0f462171 100644 --- a/docs/maintenance.md +++ b/docs/maintenance.md @@ -5,30 +5,38 @@ Logs from pods and containers can be read in different ways: * In the cluster filesystem at `/var/log/pods/` or `/var/logs/containers/`. -* Using [kubectl logs](https://kubectl.docs.kubernetes.io/pages/container_debugging/container_logs.html). -* Querying aggregated logs with grafana, see below. +* Using [kubectl logs](https://kubernetes.io/docs/concepts/cluster-administration/logging) +* Querying aggregated logs with Grafana, see below. ## Central log aggregation -We use [promtail](https://github.com/grafana/loki/tree/master/docs/clients/promtail), -[Loki](https://grafana.com/oss/loki/) and [grafana](https://grafana.com/) for +We use [Promtail](https://grafana.com/docs/loki/latest/clients/promtail/), +[Loki](https://grafana.com/oss/loki/) and [Grafana](https://grafana.com/) for easy access of aggregated logs. -The [Loki documentation](https://github.com/grafana/loki#documentation) is a -good starting point how this setup works, and the [Using Loki in Grafana](https://grafana.com/docs/grafana/latest/features/datasources/loki/) -gets you started with querying your cluster logs with grafana. +The [Loki documentation](https://grafana.com/docs/loki/latest/) is a +good starting point how this setup works, and the [Using Loki in Grafana](https://grafana.com/docs/grafana/latest/datasources/loki) +gets you started with querying your cluster logs with Grafana. -You will find the loki grafana integration on your cluster at https://grafana.oas.example.org/explore +You will find the Loki Grafana integration on your cluster at https://grafana.oas.example.org/explore together with some generic query examples. ### LogQL query examples -Please also refer to the [LogQL documentation](https://github.com/grafana/loki/blob/master/docs/logql%2Emd). +Please also refer to the [LogQL documentation](https://grafana.com/docs/loki/latest/logql). +Query all aggregated logs (unfortunatly we can't find a better way of doing this +since LogQL always expects a stream label to get queried): -Query all apps for errors etc: + logcli query '{foo!="bar"}' + +Query all logs for a keyword: + + logcli query '{foo!="bar"} |= "error"' + +Query all k8s apps for errors using a regular expression: + + logcli query '{job=~".*"} |~ "error|fail|exception|fatal"' - {job=~".*"} |~ "error|fail|exception|fatal" - {job=~".*"} |= "level=error" #### Flux