Commit Graph

4 Commits

Author SHA1 Message Date
tommy 5c66bbacdd traefik: nextcloud-transport (3600s responseHeaderTimeout) — fix large-upload 504
Real root cause of Nextcloud "Unknown error during upload" on large files:
Traefik's default-transport responseHeaderTimeout=30s killed the chunk-assembly
MOVE (which takes >30s for multi-GB files) -> 504. Added a dedicated
nextcloud-transport (3600s) and pointed nextcloud-service at it, on both nodes.
Proven: 8GB upload MOVE now 201 @ 31.7s (was 504 @ 30.0s).

Rewrote the runbook: corrected cause (was mis-attributed to Cloudflare/DoH),
plus the hard-won gotcha on editing single-file-bind-mounted dynamic_conf
(in-place cp only; never mv; validate before write).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 19:07:04 -05:00
tommy 0dea324551 traefik: de-secret CrowdSec LAPI key -> crowdsecLapiKeyFile (rotated)
Old bouncer key was committed in cleartext (hist 7fac4fc) and is now rotated
(new traefik-bouncer-v2 + old bouncer deleted, verified 200 on both nodes).
Config now references /plugins-storage/crowdsec_lapi_key (mode-600, gitignored,
lives only on the nodes). Old key in history is inert post-rotation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 22:41:35 -05:00
tommy 10b60761ff traefik: add cert-expiry-check.sh with Prometheus textfile output
Reads acme.json hourly on docker-node01, writes:
  traefik_cert_expiry_days{domain=X} N
  traefik_cert_check_last_run_seconds EPOCH

Two Grafana alert thresholds:
  Warning  < 30d: auto-renewal window opened, ntfy high priority
  Critical < 14d: ACME renewal failed, ntfy urgent

Textfile at /var/lib/node_exporter/textfile/cert_expiry.prom
Scraped by existing node-exporter job on 192.168.99.186:9100
Grafana rules: cfl8jqdlhu680d (warning), afl8jqdoepwqod (critical)
Break-tested: 35d threshold fired for vault/pdf/scrutiny/gitea correctly.

Cron: 0 * * * * sudo /usr/local/bin/cert-expiry-check.sh
2026-05-06 05:34:31 -05:00
tommy 7fac4fc9c7 traefik: sync dynamic_conf.yml and add drift-check cron
node02 was missing two blocks from node01 (canonical):
- strip-trailing-dot-speedtest middleware (regex redirect for speedtest.goattw.net. URLs)
- speedtest-trailing-dot router (catches trailing-dot Host header variant)

crowdsecLapiHost intentionally differs: node01 uses Docker service name
(crowdsec:8080, container on same host); node02 points to node01 IP
(192.168.99.186:8081, node02 has no local CrowdSec instance).

Added traefik-drift-check.sh — runs daily at 06:00 on ansible-control,
diffs both configs (excluding known crowdsecLapiHost difference),
posts to ntfy homelab-alerts on unexpected divergence.

Traefik hot-reloaded on node02 via SIGHUP — no restart required.
2026-05-05 20:17:17 -05:00