runbooks: Nextcloud large-upload fix (CT103) — browser DoH, not a size limit
Full diagnosis + fix for "Unknown error during upload" on large files: root cause is browser DNS-over-HTTPS bypassing LAN DNS -> Cloudflare mangles the parallel chunk stream. Server/DNS already correct (Technitium+UniFi return internal .185). Fix = disable browser DoH (.reg/PowerShell/GPO included). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
tommy
@@ -0,0 +1,63 @@
|
|||||||
|
# Nextcloud large-file upload fix (CT103) — "Unknown error during upload"
|
||||||
|
|
||||||
|
**Date:** 2026-07-05
|
||||||
|
**Symptom:** Uploading large movie files via the Nextcloud **web UI** fails with *"Unknown error during upload"* (small files fine).
|
||||||
|
**TL;DR root cause:** **Browser DNS-over-HTTPS (Secure DNS).** The browser bypasses LAN DNS, resolves `cloud.goattw.net` via Cloudflare, and Cloudflare mangles the parallel chunk stream. **Not** a size limit, **not** the server, **not** DNS zone config.
|
||||||
|
**TL;DR fix:** Disable browser DoH on the upload machines (see `.reg` below). Nothing server-side to change.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Setup (so future-you knows the layout)
|
||||||
|
- Nextcloud = **AIO (All-In-One)**, running **Docker inside LXC CT103** on Beast (.200). App container `nextcloud-aio-nextcloud` (PHP-FPM), front `nextcloud-aio-apache` (:11000), managed by `nextcloud-aio-mastercontainer` (:8080). Compose: `/root/nextcloud-aio/docker-compose.yml`. NC v32.
|
||||||
|
- **Access CT103:** `.103` is NOT LAN-reachable / no direct SSH. Use: `ssh tommy@192.168.99.200` then **`sudo pct exec 103 -- ...`** (`sudo pct` is NOPASSWD on Beast). occ = `docker exec -u www-data nextcloud-aio-nextcloud php /var/www/html/occ ...`.
|
||||||
|
- Data dir `/mnt/ncdata` = NFS from TrueNAS (.29), 17T. Fronted at `cloud.goattw.net` via **Cloudflare → Traefik (.186) → NC**.
|
||||||
|
|
||||||
|
## Why it fails (the evidence chain)
|
||||||
|
1. **Size limits were never the issue** — already 16G (PHP upload_max_filesize/post_max_size=16G, APACHE_MAX_SIZE=16G, max_execution_time=3600). In AIO you don't edit php.ini; it's env-managed by the mastercontainer.
|
||||||
|
2. **Real error** (nextcloud.log): chunked-upload **assembly** fails — `MOVE …/uploads/…/.file "Could not open file …/<N>, file doesn't seem to exist"` (Sabre ServiceUnavailable / AssemblyStream.php). A chunk is missing at assembly.
|
||||||
|
3. **Backend + Traefik are fine** — proven by a browser-like parallel(5) 2GB/205×10MiB chunked upload: succeeds straight to the backend (`localhost:11000`) AND through Traefik (`.185` and `.186`, Cloudflare bypassed via `curl --resolve`). 0 failures, exact assembly, every time.
|
||||||
|
4. **Cloudflare is the culprit.** `cloud.goattw.net` public record → Cloudflare (2606:4700::/32). CF body-limit probe: 100MB→passes, 105MB→413. With 10MiB chunks the *size* isn't the trip — CF drops/errors a parallel chunk PUT in transit → browser doesn't retry → assembly finds it missing.
|
||||||
|
5. **DNS is already split-horizon-correct.** Technitium (.184) AND the UniFi gateway (.1) both answer `cloud.goattw.net` A → **192.168.99.185** (internal Traefik, valid Let's Encrypt cert, >100MB passes). So any client using LAN DNS gets the good internal path.
|
||||||
|
6. **So why does it still fail?** The browser uses **DoH (Secure DNS)** → ignores LAN DNS/UniFi/Technitium → asks Cloudflare directly → takes the broken CF path. `nslookup cloud.goattw.net` shows `.185` (OS is fine) while Chrome still hits Cloudflare = DoH red-handed.
|
||||||
|
|
||||||
|
## THE FIX — disable browser DoH (machine-wide, locks it off)
|
||||||
|
Save as `disable-doh.reg`, run as admin on each upload machine (or push via GPO/RMM):
|
||||||
|
|
||||||
|
```reg
|
||||||
|
Windows Registry Editor Version 5.00
|
||||||
|
|
||||||
|
[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Google\Chrome]
|
||||||
|
"DnsOverHttpsMode"="off"
|
||||||
|
|
||||||
|
[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Edge]
|
||||||
|
"DnsOverHttpsMode"="off"
|
||||||
|
|
||||||
|
[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\BraveSoftware\Brave]
|
||||||
|
"DnsOverHttpsMode"="off"
|
||||||
|
|
||||||
|
[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Mozilla\Firefox\DNSOverHTTPS]
|
||||||
|
"Enabled"=dword:00000000
|
||||||
|
"Locked"=dword:00000001
|
||||||
|
```
|
||||||
|
|
||||||
|
PowerShell (as admin):
|
||||||
|
```powershell
|
||||||
|
'Google\Chrome','Microsoft\Edge','BraveSoftware\Brave' | ForEach-Object {
|
||||||
|
$k = "HKLM:\SOFTWARE\Policies\$_"; New-Item $k -Force | Out-Null
|
||||||
|
Set-ItemProperty $k -Name DnsOverHttpsMode -Value 'off' -Type String }
|
||||||
|
$ff='HKLM:\SOFTWARE\Policies\Mozilla\Firefox\DNSOverHTTPS'; New-Item $ff -Force | Out-Null
|
||||||
|
Set-ItemProperty $ff -Name Enabled -Value 0 -Type DWord
|
||||||
|
Set-ItemProperty $ff -Name Locked -Value 1 -Type DWord
|
||||||
|
```
|
||||||
|
|
||||||
|
GPO equivalents (domain-wide): Chrome *DNS-over-HTTPS mode = off*; Edge *Configure DNS-over-HTTPS = Disabled*; Firefox *DNS Over HTTPS Enabled=false, Locked=true*.
|
||||||
|
|
||||||
|
**After:** fully restart the browser → verify at `chrome://policy` / `edge://policy` / `about:policies` (Firefox) → retry the big upload. It now rides the internal `.185` path, no Cloudflare.
|
||||||
|
|
||||||
|
## Alternatives (not chosen)
|
||||||
|
- **Nextcloud desktop/WebDAV client** — retries dropped chunks, works through CF as-is (good interim, no config).
|
||||||
|
- **Grey-cloud `cloud.goattw.net`** (DNS-only in Cloudflare) — fixes every client incl. DoH, but exposes origin + drops CF WAF/DDoS. Also may break if CF ingress is a Tunnel. Not done.
|
||||||
|
- Raising `max_chunk_size` — MISFIRE (100MiB sits on CF's edge). Reverted; NC left at stock 10MiB default.
|
||||||
|
|
||||||
|
## Gotcha
|
||||||
|
UniFi DHCP was already handing out a resolver that returns `.185` — the DNS side needed **no change**. The only lever is the browser. DoH is the recurring trap: it silently overrides all network DNS.
|
||||||
Reference in New Issue
Block a user