plakar-incus-integration: analyze the giant-instance seek-bound wall

Per-file access to cold, tiny-file trees on HDD is ~275 IOPS whatever
the importer; only zfs send reads sequentially. Options recorded,
direction: per-file for the 16 small instances, zfs send for
nextcloud/seafile.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

ks4/plakar-incus-integration.md

@@ -119,6 +119,41 @@ to the installed plakar line (v1.1.x), rebuild the ptar on a plakar
 major upgrade; a mismatch surfaces at backup time, so run a manual
 `plakar backup @ks4-incus` after plakar upgrades.
 
+## The giant-instance problem (2026-08-24, seed attempt #1)
+
+The per-file S3 seed of the whole `backup` project stalled at
+~1 MiB/s once it reached the big replicas: sdb at 96 % util doing
+~275 × 4 KiB reads/s — **cold random tiny-file reads on an HDD are
+seek-bound**, regardless of importer (the fs importer hit the same
+wall on the live pool; sftp changes nothing). At ~275 IOPS,
+nextcloud (725 G, ~500 k files) + seafile (933 G, millions of block
+files) need *weeks* to seed. The 16 small instances are unaffected
+(solar smoke test: 5 GiB in 2m38s).
+
+Options considered for nextcloud/seafile:
+
+1. **backups-API tarball (`mode=image`)** — sequential-ish but incus
+   materializes the tarball server-side first (no room on `/`; would
+   need `storage.backups_volume` on the backup pool) and the tar walk
+   itself seeks like any tree walk. Double IO nightly. Weak.
+2. **`zfs send` of the replica dataset** — the only truly sequential
+   read (disk block order, ~100+ MiB/s, no temp space): seed in
+   hours. Full send nightly is ~4 h read for both giants; CDC chunking
+   dedups unchanged stream regions. Restore = `zfs receive` +
+   `incus admin recover` (document!). Backend-specific — acceptable as
+   an opt-in mode for exactly these two.
+3. **Per-file + warm metadata** — after a seed, nightly walks are
+   stat-only; keeping dnodes resident (`zfs_arc_min` ≈ 8 G) could make
+   them fast. Doesn't solve the *seed*.
+4. **Exclude giants from this leg** — leaves seafile without any
+   fresh off-site copy (data leg already excludes it). Not acceptable
+   long-term.
+
+Direction (pending decision): seed the 16 small instances per-file
+(`exclude_instances=nextcloud,seafile`), then implement option 2 as a
+`zfs+send://` mode or side-channel for the two giants, and revisit 3
+for nightly stat-walk speed.
+
 ## Open questions
 
 - uid/gid view through the instance sftp endpoint (idmap handling for