monitoring: off-host Prometheus dead-man's-switch watchdog (Pi .227)

Independent-hardware systemd timer on raspberrypi (.227) probing media-server
Prometheus /-/healthy every 60s; pages ntfy prometheus-critical after 3 fails
and on recovery. Catches media-server-dark, which the same-host Uptime-Kuma
monitor structurally cannot. Deployed + live-outage tested 2026-06-30.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
tommy
2026-06-30 21:57:56 -05:00
parent 0be9ae426b
commit 6af09ef9c6
5 changed files with 88 additions and 0 deletions
@@ -0,0 +1,8 @@
[Unit]
Description=Off-host Prometheus dead-man's-switch (probes media-server .183)
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
EnvironmentFile=/etc/prom-watchdog.env
ExecStart=/usr/local/bin/prometheus-watchdog.sh