helm-operator errors regarding prometheus
The helm-operator pod always gives errors like this soon after starting:
ts=2019-12-20T10:41:13.116129927Z caller=release.go:368 component=release output="clusterrolebinding.rbac.authorization.k8s.io/psp-monitoring-kube-state-metrics annotated\npodsecuritypolicy.extensions/monitoring-grafana-test annotated\nclusterrolebinding.rbac.authorization.k
8s.io/monitoring-prometheus-oper-prometheus-psp annotated\nconfigmap/monitoring-grafana-test annotated\nconfigmap/monitoring-prometheus-oper-grafana-datasource annotated\nserviceaccount/monitoring-prometheus-oper-prometheus annotated\nservicemonitor.monitoring.coreos.com/mon
itoring-prometheus-oper-kube-proxy annotated\nservicemonitor.monitoring.coreos.com/monitoring-prometheus-oper-apiserver annotated\nconfigmap/monitoring-prometheus-oper-kubelet annotated\nservicemonitor.monitoring.coreos.com/monitoring-prometheus-oper-alertmanager annotated\n
configmap/monitoring-prometheus-oper-pods annotated\nserviceaccount/monitoring-prometheus-oper-alertmanager annotated\nclusterrolebinding.rbac.authorization.k8s.io/monitoring-kube-state-metrics annotated\nservicemonitor.monitoring.coreos.com/monitoring-prometheus-oper-operat
or annotated\nvalidatingwebhookconfiguration.admissionregistration.k8s.io/monitoring-prometheus-oper-admission annotated\nconfigmap/monitoring-prometheus-oper-node-rsrc-use annotated\nrolebinding.rbac.authorization.k8s.io/monitoring-grafana annotated\nservice/monitoring-prom
etheus-oper-operator annotated\nservicemonitor.monitoring.coreos.com/monitoring-prometheus-oper-node-exporter annotated\nconfigmap/monitoring-grafana annotated\nclusterrolebinding.rbac.authorization.k8s.io/monitoring-prometheus-oper-operator-psp annotated\nserviceaccount/mon
itoring-grafana-test annotated\nserviceaccount/monitoring-kube-state-metrics annotated\nclusterrole.rbac.authorization.k8s.io/monitoring-kube-state-metrics annotated\nservicemonitor.monitoring.coreos.com/monitoring-prometheus-oper-grafana annotated\nconfigmap/monitoring-prom
etheus-oper-proxy annotated\nclusterrole.rbac.authorization.k8s.io/psp-monitoring-kube-state-metrics annotated\nservice/monitoring-prometheus-oper-alertmanager annotated\nconfigmap/monitoring-prometheus-oper-apiserver annotated\nconfigmap/monitoring-prometheus-oper-statefuls
et annotated\nclusterrole.rbac.authorization.k8s.io/monitoring-prometheus-oper-prometheus annotated\nclusterrole.rbac.authorization.k8s.io/monitoring-prometheus-oper-prometheus-psp annotated\nconfigmap/monitoring-prometheus-oper-nodes annotated\nconfigmap/monitoring-promethe
us-oper-prometheus annotated\nconfigmap/monitoring-grafana-config-dashboards annotated\nconfigmap/monitoring-prometheus-oper-etcd annotated\nconfigmap/monitoring-prometheus-oper-k8s-resources-workload annotated\nclusterrolebinding.rbac.authorization.k8s.io/monitoring-prometheus-oper-alertmanager annotated\ndeployment.extensions/monitoring-grafana annotated\nsecret/monitoring-grafana annotated\nconfigmap/monitoring-prometheus-oper-k8s-resources-cluster annotated\nconfigmap/monitoring-prometheus-oper-k8s-resources-pod annotated\nconfigmap/monitoring-prometheus-oper-prometheus-remote-write annotated\nservice/monitoring-kube-state-metrics annotated\nservice/monitoring-prometheus-oper-prometheus annotated\npodsecuritypolicy.extensions/monitoring-kube-state-metrics annotated\ndaemonset.extensions/monitoring-prometheus-node-exporter annotated\nprometheus.monitoring.coreos.com/monitoring-prometheus-oper-prometheus annotated\npodsecuritypolicy.extensions/monitoring-prometheus-node-exporter annotated\nclusterrole.rbac.authorization.k8s.io/monitoring-grafana-clusterrole annotated\nservicemonitor.monitoring.coreos.com/monitoring-prometheus-oper-kubelet annotated\ndeployment.extensions/monitoring-prometheus-oper-operator annotated\nservicemonitor.monitoring.coreos.com/monitoring-prometheus-oper-coredns annotated\nconfigmap/monitoring-prometheus-oper-k8s-resources-workloads-namespace annotated\nclusterrolebinding.rbac.authorization.k8s.io/monitoring-prometheus-oper-operator annotated\nclusterrolebinding.rbac.authorization.k8s.io/monitoring-prometheus-oper-prometheus annotated\nrole.rbac.authorization.k8s.io/monitoring-grafana annotated\npodsecuritypolicy.extensions/monitoring-prometheus-oper-alertmanager annotated\npersistentvolumeclaim/monitoring-grafana annotated\nclusterrole.rbac.authorization.k8s.io/monitoring-prometheus-oper-operator annotated\nmutatingwebhookconfiguration.admissionregistration.k8s.io/monitoring-prometheus-oper-admission annotated\nservicemonitor.monitoring.coreos.com/monitoring-prometheus-oper-kube-etcd annotated\nconfigmap/monitoring-prometheus-oper-node-cluster-rsrc-use annotated\nserviceaccount/monitoring-prometheus-node-exporter annotated\nservicemonitor.monitoring.coreos.com/monitoring-prometheus-oper-prometheus annotated\nrole.rbac.authorization.k8s.io/monitoring-grafana-test annotated\ndeployment.extensions/monitoring-kube-state-metrics annotated\nconfigmap/monitoring-prometheus-oper-k8s-coredns annotated\nconfigmap/monitoring-prometheus-oper-k8s-resources-namespace annotated\nconfigmap/monitoring-prometheus-oper-persistentvolumesusage annotated\ningress.extensions/monitoring-grafana annotated\npodsecuritypolicy.extensions/monitoring-prometheus-oper-prometheus annotated\nserviceaccount/monitoring-grafana annotated\nclusterrole.rbac.authorization.k8s.io/monitoring-prometheus-oper-operator-psp annotated\nservice/monitoring-grafana annotated\nalertmanager.monitoring.coreos.com/monitoring-prometheus-oper-alertmanager annotated\nsecret/alertmanager-monitoring-prometheus-oper-alertmanager annotated\nserviceaccount/monitoring-prometheus-oper-operator annotated\nclusterrole.rbac.authorization.k8s.io/monitoring-prometheus-oper-alertmanager annotated\nrolebinding.rbac.authorization.k8s.io/monitoring-grafana-test annotated\npodsecuritypolicy.extensions/monitoring-prometheus-oper-operator annotated\nclusterrolebinding.rbac.authorization.k8s.io/monitoring-grafana-clusterrolebinding annotated\nclusterrolebinding.rbac.authorization.k8s.io/psp-monitoring-prometheus-node-exporter annotated\nservice/monitoring-prometheus-node-exporter annotated\nservicemonitor.monitoring.coreos.com/monitoring-prometheus-oper-kube-state-metrics annotated\npodsecuritypolicy.extensions/monitoring-grafana annotated\nclusterrole.rbac.authorization.k8s.io/psp-monitoring-prometheus-node-exporter annotated\nconfigmap/monitoring-prometheus-oper-k8s-resources-node annotated\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\nError from server (InternalError): Internal error occurred: failed calling webhook \"prometheusrulemutate.monitoring.coreos.com\": Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refused\n" err="exit status 1"The last part:
Post https://monitoring-prometheus-oper-operator.oas.svc:443/admission-prometheusrules/mutate?timeout=30s: dial tcp 10.43.35.235:443: connect: connection refusedsuggests that perhaps some microservice is not running, or maybe a port needs to be opened up.