Files
homelab-configs/runbooks/nextcloud-large-upload-fix.md
T
tommy c6861c71f0 runbooks: Nextcloud large-upload fix (CT103) — browser DoH, not a size limit
Full diagnosis + fix for "Unknown error during upload" on large files:
root cause is browser DNS-over-HTTPS bypassing LAN DNS -> Cloudflare mangles
the parallel chunk stream. Server/DNS already correct (Technitium+UniFi return
internal .185). Fix = disable browser DoH (.reg/PowerShell/GPO included).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 18:17:48 -05:00

4.9 KiB
Raw Blame History

Nextcloud large-file upload fix (CT103) — "Unknown error during upload"

Date: 2026-07-05 Symptom: Uploading large movie files via the Nextcloud web UI fails with "Unknown error during upload" (small files fine). TL;DR root cause: Browser DNS-over-HTTPS (Secure DNS). The browser bypasses LAN DNS, resolves cloud.goattw.net via Cloudflare, and Cloudflare mangles the parallel chunk stream. Not a size limit, not the server, not DNS zone config. TL;DR fix: Disable browser DoH on the upload machines (see .reg below). Nothing server-side to change.


Setup (so future-you knows the layout)

  • Nextcloud = AIO (All-In-One), running Docker inside LXC CT103 on Beast (.200). App container nextcloud-aio-nextcloud (PHP-FPM), front nextcloud-aio-apache (:11000), managed by nextcloud-aio-mastercontainer (:8080). Compose: /root/nextcloud-aio/docker-compose.yml. NC v32.
  • Access CT103: .103 is NOT LAN-reachable / no direct SSH. Use: ssh tommy@192.168.99.200 then sudo pct exec 103 -- ... (sudo pct is NOPASSWD on Beast). occ = docker exec -u www-data nextcloud-aio-nextcloud php /var/www/html/occ ....
  • Data dir /mnt/ncdata = NFS from TrueNAS (.29), 17T. Fronted at cloud.goattw.net via Cloudflare → Traefik (.186) → NC.

Why it fails (the evidence chain)

  1. Size limits were never the issue — already 16G (PHP upload_max_filesize/post_max_size=16G, APACHE_MAX_SIZE=16G, max_execution_time=3600). In AIO you don't edit php.ini; it's env-managed by the mastercontainer.
  2. Real error (nextcloud.log): chunked-upload assembly fails — MOVE …/uploads/…/.file "Could not open file …/<N>, file doesn't seem to exist" (Sabre ServiceUnavailable / AssemblyStream.php). A chunk is missing at assembly.
  3. Backend + Traefik are fine — proven by a browser-like parallel(5) 2GB/205×10MiB chunked upload: succeeds straight to the backend (localhost:11000) AND through Traefik (.185 and .186, Cloudflare bypassed via curl --resolve). 0 failures, exact assembly, every time.
  4. Cloudflare is the culprit. cloud.goattw.net public record → Cloudflare (2606:4700::/32). CF body-limit probe: 100MB→passes, 105MB→413. With 10MiB chunks the size isn't the trip — CF drops/errors a parallel chunk PUT in transit → browser doesn't retry → assembly finds it missing.
  5. DNS is already split-horizon-correct. Technitium (.184) AND the UniFi gateway (.1) both answer cloud.goattw.net A → 192.168.99.185 (internal Traefik, valid Let's Encrypt cert, >100MB passes). So any client using LAN DNS gets the good internal path.
  6. So why does it still fail? The browser uses DoH (Secure DNS) → ignores LAN DNS/UniFi/Technitium → asks Cloudflare directly → takes the broken CF path. nslookup cloud.goattw.net shows .185 (OS is fine) while Chrome still hits Cloudflare = DoH red-handed.

THE FIX — disable browser DoH (machine-wide, locks it off)

Save as disable-doh.reg, run as admin on each upload machine (or push via GPO/RMM):

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Google\Chrome]
"DnsOverHttpsMode"="off"

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Edge]
"DnsOverHttpsMode"="off"

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\BraveSoftware\Brave]
"DnsOverHttpsMode"="off"

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Mozilla\Firefox\DNSOverHTTPS]
"Enabled"=dword:00000000
"Locked"=dword:00000001

PowerShell (as admin):

'Google\Chrome','Microsoft\Edge','BraveSoftware\Brave' | ForEach-Object {
  $k = "HKLM:\SOFTWARE\Policies\$_"; New-Item $k -Force | Out-Null
  Set-ItemProperty $k -Name DnsOverHttpsMode -Value 'off' -Type String }
$ff='HKLM:\SOFTWARE\Policies\Mozilla\Firefox\DNSOverHTTPS'; New-Item $ff -Force | Out-Null
Set-ItemProperty $ff -Name Enabled -Value 0 -Type DWord
Set-ItemProperty $ff -Name Locked  -Value 1 -Type DWord

GPO equivalents (domain-wide): Chrome DNS-over-HTTPS mode = off; Edge Configure DNS-over-HTTPS = Disabled; Firefox DNS Over HTTPS Enabled=false, Locked=true.

After: fully restart the browser → verify at chrome://policy / edge://policy / about:policies (Firefox) → retry the big upload. It now rides the internal .185 path, no Cloudflare.

Alternatives (not chosen)

  • Nextcloud desktop/WebDAV client — retries dropped chunks, works through CF as-is (good interim, no config).
  • Grey-cloud cloud.goattw.net (DNS-only in Cloudflare) — fixes every client incl. DoH, but exposes origin + drops CF WAF/DDoS. Also may break if CF ingress is a Tunnel. Not done.
  • Raising max_chunk_size — MISFIRE (100MiB sits on CF's edge). Reverted; NC left at stock 10MiB default.

Gotcha

UniFi DHCP was already handing out a resolver that returns .185 — the DNS side needed no change. The only lever is the browser. DoH is the recurring trap: it silently overrides all network DNS.