All Questions
Tagged with prometheus-alertmanager devops
12
questions
0
votes
0
answers
19
views
Is there a way to suppress 'resolve' messages sent to pager duty
Our pager duty is integrated to alert manager, and we want to stop an specific alert from auto resolving. To my knowledge, the best way to do that would be changing the alert manager configuration so ...
0
votes
0
answers
113
views
Alertmanager v0.25 inhibit rule
I am running kube-prometheus-stack 43.3.0 which includes Alertmanager 0.25 on a k8s cluster.
My alertmanager config yaml looks like this:
apiVersion: monitoring.coreos.com/v1alpha1
kind: ...
- 89
3
votes
1
answer
1k
views
Go Template if condition
how can i combine and and eq/ne functions together?
I wrote this snippet
{{ define "opsgenie.default.tmpl" }}
<font size="+0"><b>{{.CommonLabels.alertname }}</b>...
- 121
1
vote
1
answer
475
views
Time Based escalation in alertmanager
Let's say I have an alert related to disk space, which is triggered at the following levels.
First at 85%(receiver for this is admin), then at 90%(receiver for this is super-admin) and finally at 95%(...
- 11
1
vote
1
answer
316
views
Alertmanager `inhibit_rules` not working?
I'm trying to configure something that seems very simple:
"If the datasource (prometheus) is down, don't spam alerts, and only notify that Prometheus is down"
Immediately I went to ...
- 678
0
votes
1
answer
7k
views
Prometheus/Alertmanager inhibit rules
I have the following inhibit rule:
inhibit_rules:
- source_match:
severity: 'critical'
target_match:
severity: 'high'
equal: ['alertname]
and also two alerts accordingly with severity ...
- 121
0
votes
1
answer
162
views
whats the meaning of "verb=LIST" in kubernetes apiserver logs and alerts?
Recently i renewed certificates on cluster using kubeadm alpha certs renew and then I saw logs in Kubernetes Apiserver pods as below:
kubectl -n kube-system logs --tail 10 kube-apiserver-master-1
...
- 43
0
votes
0
answers
425
views
Prometheus alert relabeling to apply fallback label
I want to configure prometheus to add a fallback label team=devops to the alert labels, when not provided by the alert rule.
Example of alert rule with team label:
- alert: ...
- 690
0
votes
0
answers
2k
views
Prometheus Opsgenie Integration
We use Prometheus alert manager for alerts, we have configured two receivers as attached below, Slack and Opsgenie.
The issue is that we are getting alerts in slack only but we are not receiving ...
- 11
2
votes
3
answers
6k
views
How to monitor disk space usage for Kafka Brokers in AWS MSK cluster
We need to Monitor disk space usage for Kafka Brokers running in AWS MSK cluster.
There're several metrices emitted by Kafka which can be used to monitor various aspects. But I was unable to find any ...
- 339
2
votes
0
answers
1k
views
Prometheus alert rules - check if a label exists in the message field
An example of the rule I want to append with a label if that label exists.
- alert: TargetDown
annotations:
message: '{{`{{`}} printf "%.4g" $value {{`}}`}}% of the {{`{{`}} $...
- 615
1
vote
0
answers
148
views
Need to have blacklisting of Endpoints on Prometheus/Alertmanager
Creating a blacklist configuration in Prometheus/Alertmanger that allow excluding individual hosts and/or host/service combinations from creating alerts, e.g.:
blacklist_endpoints:
name: ...
- 1,496