feat: extend monitoring to include VIN Decoder API
Validate Flux manifests / kustomize-build (pull_request) Failing after 17s
Validate Flux manifests / kustomize-build (pull_request) Failing after 17s
- flux/monitoring/prometheusrule-apis.yaml: extend all 5 SLO alert rules to include vin-decoder in the job regex filter - flux/monitoring/grafana-dashboard-apis.yaml: update all PromQL expressions to include vin-decoder in the api regex filter; update dashboard description - flux/monitoring/vin-decoder-dashboard.yaml: new dedicated Grafana dashboard for VIN Decoder showing request rate by route, 5xx error rate, P50/P95/P99 latency by route, and data freshness gauge - flux/monitoring/kustomization.yaml: add vin-decoder-dashboard.yaml - flux/vin-decoder/servicemonitor.yaml: Prometheus ServiceMonitor for auto-discovery of /metrics on the vin-decoder service - flux/vin-decoder/kustomization.yaml: add servicemonitor.yaml Closes leeworks-agents/api-company#129
This commit is contained in:
@@ -18,9 +18,9 @@ spec:
|
||||
- alert: APIHighErrorRate
|
||||
expr: |
|
||||
(
|
||||
sum by (job) (rate(api_requests_total{status=~"5..", job=~"zip|holidays|air-quality"}[5m]))
|
||||
sum by (job) (rate(api_requests_total{status=~"5..", job=~"zip|holidays|air-quality|vin-decoder"}[5m]))
|
||||
/
|
||||
sum by (job) (rate(api_requests_total{job=~"zip|holidays|air-quality"}[5m]))
|
||||
sum by (job) (rate(api_requests_total{job=~"zip|holidays|air-quality|vin-decoder"}[5m]))
|
||||
) > 0.05
|
||||
for: 5m
|
||||
labels:
|
||||
@@ -36,9 +36,9 @@ spec:
|
||||
- alert: APIHighErrorRate
|
||||
expr: |
|
||||
(
|
||||
sum by (job) (rate(api_requests_total{status=~"5..", job=~"zip|holidays|air-quality"}[5m]))
|
||||
sum by (job) (rate(api_requests_total{status=~"5..", job=~"zip|holidays|air-quality|vin-decoder"}[5m]))
|
||||
/
|
||||
sum by (job) (rate(api_requests_total{job=~"zip|holidays|air-quality"}[5m]))
|
||||
sum by (job) (rate(api_requests_total{job=~"zip|holidays|air-quality|vin-decoder"}[5m]))
|
||||
) > 0.20
|
||||
for: 5m
|
||||
labels:
|
||||
@@ -55,7 +55,7 @@ spec:
|
||||
expr: |
|
||||
histogram_quantile(
|
||||
0.95,
|
||||
sum by (job, le) (rate(api_response_duration_seconds_bucket{job=~"zip|holidays|air-quality"}[5m]))
|
||||
sum by (job, le) (rate(api_response_duration_seconds_bucket{job=~"zip|holidays|air-quality|vin-decoder"}[5m]))
|
||||
) > 2
|
||||
for: 5m
|
||||
labels:
|
||||
@@ -70,7 +70,7 @@ spec:
|
||||
# -------------------------------------------------------------------
|
||||
- alert: APIDataStale
|
||||
expr: |
|
||||
api_data_freshness_seconds{job=~"zip|holidays|air-quality"} > 2592000
|
||||
api_data_freshness_seconds{job=~"zip|holidays|air-quality|vin-decoder"} > 2592000
|
||||
for: 30m
|
||||
labels:
|
||||
severity: warning
|
||||
@@ -84,7 +84,7 @@ spec:
|
||||
# -------------------------------------------------------------------
|
||||
- alert: APIDataCriticallyStale
|
||||
expr: |
|
||||
api_data_freshness_seconds{job=~"zip|holidays|air-quality"} > 5184000
|
||||
api_data_freshness_seconds{job=~"zip|holidays|air-quality|vin-decoder"} > 5184000
|
||||
for: 1h
|
||||
labels:
|
||||
severity: critical
|
||||
@@ -98,9 +98,9 @@ spec:
|
||||
# -------------------------------------------------------------------
|
||||
- alert: APIDown
|
||||
expr: |
|
||||
absent(up{job=~"zip|holidays|air-quality"} == 1)
|
||||
absent(up{job=~"zip|holidays|air-quality|vin-decoder"} == 1)
|
||||
or
|
||||
up{job=~"zip|holidays|air-quality"} == 0
|
||||
up{job=~"zip|holidays|air-quality|vin-decoder"} == 0
|
||||
for: 2m
|
||||
labels:
|
||||
severity: critical
|
||||
|
||||
Reference in New Issue
Block a user