LubeLogger: back up data+keys volumes to PBS via the playbook

Close the volume-backup gap: the pg_dump covered Postgres only; the ASP.NET DataProtection keyring (root-user credential) and uploaded docs live in the lubelogger-{data,keys} volumes. The docker-volume-backup sidecar can't target PBS natively, so both volumes now ride the same proxmox-backup-client job as the SQL dump — one host/lubelogger snapshot, three .pxar archives, keep-daily 14. Backup script+units are now installed by the Ansible playbook (idempotent, changed=0); PBS token from ansible-vault. Restore verified: keyring XML + sentinel doc byte-identical (sha256). Removes the now-playbook-managed standalone script/units. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 15:28:31 -05:00
parent c9aac20e73
commit 6de37c8712
5 changed files with 116 additions and 49 deletions
@@ -18,14 +18,20 @@ Inline-compose house style: `copy:` the compose + `community.docker.docker_compo
 DB password is sourced from ansible-vault (`vault_lubelogger_db_password`) — never inline.
 Second converge is `changed=0`.

-## Backup (PBS)
-`lubelogger-backup.{sh,service,timer}` on node01 — daily 03:30, `pg_dump` → PBS
-(`rust-usb`, host/lubelogger), prune `--keep-daily 14`. PBS creds live in
-`/etc/lubelogger-backup.env` (root:600, NOT in git). Restore verified via full
-PBS round-trip into a throwaway postgres:18 (live↔restored row-count parity).
-NOTE: the `lubelogger-data`/`lubelogger-keys` volumes (uploads, config, the ROOT USER
-credential, ASP.NET keyring) are a SEPARATE layer — covered by docker-volume-backup
-(`stop-during-backup` labels already set). pg_dump captures Postgres data only.
+## Backup (PBS) — deployed BY the playbook
+The backup script + systemd units are installed and enabled by `deploy-lubelogger.yml`
+(inline, idempotent). node01, daily 03:30, one PBS snapshot (`rust-usb`, host/lubelogger)
+with **three archives** in a single `proxmox-backup-client` job, prune `--keep-daily 14`:
+- `lubelogger-db.pxar`   — `pg_dump` (app schema + data)
+- `lubelogger-data.pxar` — `/App/data` volume (uploads, config, cartracker.db)
+- `lubelogger-keys.pxar` — ASP.NET DataProtection keyring (the ROOT USER credential + keys)
+
+The `docker-volume-backup` sidecar can't target a PBS datastore natively, so the volumes
+ride the same pbc job as the SQL dump (one datastore, one retention). PBS token comes from
+ansible-vault (`vault_lubelogger_pbs_token`) → templated to `/etc/lubelogger-backup.env`
+(root:600, NOT in git). Restore VERIFIED: SQL round-trip into throwaway postgres:18
+(row-count parity) + data/keys restore (keyring XML + a sentinel doc, byte-identical sha256).
+The `stop-during-backup` labels remain in case a sidecar is later added for a second target.

 ## Reminders → ntfy
 `lubelogger-reminders-probe.{sh}` + units on ansible-control (.190) — daily 07:30.
@@ -6,6 +6,11 @@
    app_dir: /home/tommy/lubelogger
    app_port: 8099

+  handlers:
+    - name: reload systemd
+      ansible.builtin.systemd_service:
+        daemon_reload: true
+
  tasks:
    - name: Create lubelogger directory
      ansible.builtin.file:
@@ -102,3 +107,100 @@
    - name: Show deployment result
      ansible.builtin.debug:
        msg: "lubelogger is {{ 'running (changed)' if app_result.changed else 'already up (no-op)' }} on port {{ app_port }}"
+
+    # ---- Backup: pg_dump + data/keys volumes -> PBS (host/lubelogger, keep-daily 14) ----
+    # docker-volume-backup sidecar cannot target a PBS datastore natively, so the two
+    # volumes ride the same proxmox-backup-client job as the SQL dump: one snapshot,
+    # three archives, one retention. Secrets come from ansible-vault, never committed.
+
+    - name: Install PBS backup credentials (from vault)
+      ansible.builtin.copy:
+        dest: /etc/lubelogger-backup.env
+        owner: root
+        group: root
+        mode: "0600"
+        content: |
+          # Managed by deploy-lubelogger.yml. PBS push creds — NOT in git.
+          PBS_PASSWORD={{ vault_lubelogger_pbs_token }}
+          PBS_FINGERPRINT=90:6B:3E:85:C4:E7:77:F0:24:25:1D:B0:2C:F9:6D:08:AA:CE:08:CE:7B:04:4C:C7:C9:E0:CF:DD:99:44:DB:21
+          PBS_REPOSITORY='root@pam!vaultwarden-backup@192.168.99.153:rust-usb'
+      no_log: true
+
+    - name: Install lubelogger backup script
+      ansible.builtin.copy:
+        dest: /usr/local/bin/lubelogger-backup.sh
+        owner: root
+        group: root
+        mode: "0700"
+        content: |
+          #!/bin/bash
+          # LubeLogger backup -> Proxmox Backup Server.
+          # One snapshot (host/lubelogger), three archives: Postgres logical dump +
+          # the data and keys volumes (uploads, config, ASP.NET DataProtection keyring).
+          # Managed by deploy-lubelogger.yml; driven by lubelogger-backup.timer.
+          set -euo pipefail
+          source /etc/lubelogger-backup.env
+          export PBS_PASSWORD PBS_FINGERPRINT
+
+          DATA_VOL=/var/lib/docker/volumes/lubelogger_lubelogger-data/_data
+          KEYS_VOL=/var/lib/docker/volumes/lubelogger_lubelogger-keys/_data
+
+          TMPDIR=$(mktemp -d)
+          trap 'rm -rf "$TMPDIR"' EXIT
+
+          # Logical DB dump (app schema + data); app stays online.
+          docker exec lubelogger-db pg_dump -U lubelogger lubelogger > "$TMPDIR/lubelogger.sql"
+
+          proxmox-backup-client backup \
+            "lubelogger-db.pxar:$TMPDIR" \
+            "lubelogger-data.pxar:$DATA_VOL" \
+            "lubelogger-keys.pxar:$KEYS_VOL" \
+            --repository "$PBS_REPOSITORY" \
+            --backup-type host \
+            --backup-id lubelogger
+
+          proxmox-backup-client prune "host/lubelogger" \
+            --repository "$PBS_REPOSITORY" \
+            --keep-daily 14
+      notify: reload systemd
+
+    - name: Install backup systemd service unit
+      ansible.builtin.copy:
+        dest: /etc/systemd/system/lubelogger-backup.service
+        owner: root
+        group: root
+        mode: "0644"
+        content: |
+          [Unit]
+          Description=LubeLogger backup (pg_dump + volumes) -> PBS
+          After=docker.service
+          Requires=docker.service
+
+          [Service]
+          Type=oneshot
+          ExecStart=/usr/local/bin/lubelogger-backup.sh
+      notify: reload systemd
+
+    - name: Install backup systemd timer unit
+      ansible.builtin.copy:
+        dest: /etc/systemd/system/lubelogger-backup.timer
+        owner: root
+        group: root
+        mode: "0644"
+        content: |
+          [Unit]
+          Description=Daily LubeLogger backup to PBS
+
+          [Timer]
+          OnCalendar=*-*-* 03:30:00
+          Persistent=true
+
+          [Install]
+          WantedBy=timers.target
+      notify: reload systemd
+
+    - name: Enable lubelogger backup timer
+      ansible.builtin.systemd_service:
+        name: lubelogger-backup.timer
+        enabled: true
+        state: started
@@ -1,8 +0,0 @@
-[Unit]
-Description=LubeLogger Postgres dump -> PBS
-After=docker.service
-Requires=docker.service
-
-[Service]
-Type=oneshot
-ExecStart=/usr/local/bin/lubelogger-backup.sh
@@ -1,24 +0,0 @@
-#!/bin/bash
-# LubeLogger Postgres logical backup -> Proxmox Backup Server.
-# Installed at /usr/local/bin/lubelogger-backup.sh ; driven by lubelogger-backup.timer.
-set -euo pipefail
-source /etc/lubelogger-backup.env
-export PBS_PASSWORD PBS_FINGERPRINT
-
-TMPDIR=$(mktemp -d)
-trap 'rm -rf "$TMPDIR"' EXIT
-
-# Consistent logical dump (includes the `app` schema + data). App stays online.
-docker exec lubelogger-db pg_dump -U lubelogger lubelogger > "$TMPDIR/lubelogger.sql"
-
-# Push to PBS
-proxmox-backup-client backup \
-  "lubelogger-db.pxar:$TMPDIR" \
-  --repository "$PBS_REPOSITORY" \
-  --backup-type host \
-  --backup-id lubelogger
-
-# Retain ~14 daily snapshots
-proxmox-backup-client prune "host/lubelogger" \
-  --repository "$PBS_REPOSITORY" \
-  --keep-daily 14
@@ -1,9 +0,0 @@
-[Unit]
-Description=Daily LubeLogger Postgres backup to PBS
-
-[Timer]
-OnCalendar=*-*-* 03:30:00
-Persistent=true
-
-[Install]
-WantedBy=timers.target