Use coredns on OAS cluster node
While working on #485 (closed) I noticed a strange phenomena. For querying prometheus from the host using a ClusterIp, I need to find out the IP of the prometheus service.
root@varac-test:~# kubectl -n oas get service monitoring-prometheus-oper-prometheus
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
monitoring-prometheus-oper-prometheus ClusterIP 10.43.38.214 <none> 9090/TCP 17d
Then, curl 10.43.38.214:9090/alerts
would show me the alerts.
A simpler apporach would be to query coreDns for the prometheus IP. It works when querying coreDns directly (which is always at 10.43.0.10
on our clusters) with i.e. host
:
root@varac-test:~# host monitoring-prometheus-oper-prometheus.oas.svc.cluster.local 10.43.0.10
Using domain server:
Name: 10.43.0.10
Address: 10.43.0.10#53
Aliases:
monitoring-prometheus-oper-prometheus.oas.svc.cluster.local has address 10.43.38.214
So far so good.
However, in order to use curl monitoring-prometheus-oper-prometheus.oas.svc.cluster.local:9090
directly, we would need to add 10.43.0.10
as nameserver in /etc/resolv.conf
.
And this is where things become strange.