tommy/homelab-configs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

runbooks: fix VAULT1 CIFS creds + add credential recovery procedure

Browse Source 
VAULT1 supervisor mount had missing credential file since at least
2026-05-01 (lost after unclean shutdown). 22 days of Frigate recordings
accumulated on sda8 (299 GB) instead of the 4 TB USB drive. Fixed via
PUT /mounts/VAULT1 with hassio credentials; mount now shows 3.6T/915G.
Orphan dirs documented; RB-001 updated with credential recovery steps.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Tommy
2026-05-23 21:27:14 -05:00
parent 50e8dfccc7
commit 7e8ea78cee
1 changed files with 26 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
runbooks/phase5-incident-log.md
+26 -2
View File
@@ -1,6 +1,6 @@
# Phase 5 — Incident Log / State Drift
Last updated: 2026-05-23 (INC-007 added — Frigate/VAULT1 USB enclosure outage)
Last updated: 2026-05-23 (INC-007 added; VAULT1 CIFS cred fixed; RB-001 credential recovery added)
Purpose: Consolidates real findings from the Phase 4 worksession. Inputs to Phase 5 planning and repair.
---
@@ -318,7 +318,7 @@ Subscriber reminder: subscribe ntfy client to topic `health-digest` on 192.168.9
| PBS ZFS pool feature upgrade | usb1-zfs, usb2-zfs | `zpool upgrade` deferred until pools are stable post-hardware fix |
| grub cmdline not Ansible-enforced | PBS, Beast | `usb-storage.quirks=174c:55aa:u` applied manually; archived by backup playbook but not templated. Risk: silent regression if grub is rewritten by kernel upgrade or manual edit. Convert to Ansible-managed template for USB-quirk nodes. Not urgent. |
| Alertmanager permission errors | media-server | Resolved (C2 May-05) — no action needed |
| VAULT1 CIFS credentials file | HAOS supervisor | `/mnt/data/supervisor/.mounts_credentials/VAULT1` does not exist; supervisor retries every 15 min but fails. Separate from USB health — Frigate accesses ext4 mounts directly, not via CIFS. Credential file needed if VAULT1 CIFS mount is intentional. |
| VAULT1 orphan dirs on sda8 | HAOS /media | `VAULT1.local-orphan-20260501-2200` (982 MB) and `VAULT1.local-orphan-20260523-0224` (299 GB) on sda8. ~22 days of recordings accumulated on HAOS system partition while CIFS was down. VAULT1 CIFS is now fixed (2026-05-23). Orphan dirs are outside Frigate's media path — Frigate will NOT clean them up. Move useful recordings to `/media/VAULT1` manually, or delete once retention window passes. sda8 has 1.4 TB free so no immediate risk. |
---
@@ -375,6 +375,30 @@ curl -s -H "Authorization: Bearer $TOKEN" 'http://supervisor/host/logs?lines=500
~3 minutes from enclosure reseat to Frigate healthy (HAOS reboot path).
~10–15 minutes if Frigate add-on restart is needed instead of full reboot.
#### VAULT1 CIFS credential recovery (when credential file is lost)
The supervisor credential file at `/mnt/data/supervisor/.mounts_credentials/VAULT1` can be lost after an unclean HAOS shutdown (filesystem dirty bit). Symptoms: supervisor log shows `error 2 (No such file or directory) opening credential file` every 15 min; `/media/VAULT1` falls through to sda8 and Frigate writes recordings there instead of the USB drive.
To recover (requires ~2 min Frigate downtime):
```bash
# 1. Stop Frigate
TOKEN=$(sudo cat /proc/*/environ 2>/dev/null | tr '\0' '\n' | grep '^SUPERVISOR_TOKEN=' | head -1 | cut -d= -f2-)
curl -s -X POST -H "Authorization: Bearer $TOKEN" http://supervisor/addons/ccab4aaf_frigate-fa/stop
# 2. Move any accumulated orphan data off the path (Samba creds: hassio / check bootconfig.json)
sudo mv /media/VAULT1 /media/VAULT1.local-orphan-$(date -u +%Y%m%d-%H%M)
# 3. Re-PUT the mount with credentials (this recreates the credential file)
curl -s -X PUT -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
-d '{"name":"VAULT1","type":"cifs","usage":"media","server":"172.30.32.1","share":"VAULT1","username":"hassio","password":"<from /addon_configs/1a32f091_sambanas/bootconfig.json>"}' \
http://supervisor/mounts/VAULT1
# 4. Verify: df -h /media/VAULT1 should show //172.30.32.1/VAULT1 at 3.6T
# 5. Start Frigate
curl -s -X POST -H "Authorization: Bearer $TOKEN" http://supervisor/addons/ccab4aaf_frigate-fa/start
```
Orphan dirs on sda8 are outside Frigate's media path — handle manually (move or delete).
#### Prevention / monitoring
- Uptime Kuma "Frigate Stats" monitor (http://192.168.99.100:5000/api/stats, 60s, 2 retries) → ntfy `grafana-alerts` (added 2026-05-23)
- Uptime Kuma "Home Assistant" monitor (http://192.168.99.100:8123, 60s) → ntfy `grafana-alerts` (added 2026-05-23)