monitoring: off-host Prometheus dead-man's-switch watchdog (Pi .227)

Independent-hardware systemd timer on raspberrypi (.227) probing media-server
Prometheus /-/healthy every 60s; pages ntfy prometheus-critical after 3 fails
and on recovery. Catches media-server-dark, which the same-host Uptime-Kuma
monitor structurally cannot. Deployed + live-outage tested 2026-06-30.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
tommy
2026-06-30 21:57:56 -05:00
parent 0be9ae426b
commit 6af09ef9c6
5 changed files with 88 additions and 0 deletions
@@ -0,0 +1,5 @@
# Copy to /etc/prom-watchdog.env (mode 600) on the Pi.
# NTFY_TOKEN stays empty while the prometheus-critical topic is unauthenticated;
# if ntfy auth is ever enabled, set it in the DEPLOYED file only (never commit — *.env is gitignored).
NTFY_URL=http://192.168.99.198:8095/prometheus-critical
NTFY_TOKEN=