plakar-incus-integration: analyze the giant-instance seek-bound wall

Per-file access to cold, tiny-file trees on HDD is ~275 IOPS whatever
the importer; only zfs send reads sequentially. Options recorded,
direction: per-file for the 16 small instances, zfs send for
nextcloud/seafile.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

README.md

@@ -4,6 +4,11 @@ Personal infrastructure notes.
 
 - [`nuc/`](nuc/) — home lab on `nuc` (see below)
 - [`ks4/`](ks4/) — prod server `ks4` at OVH (see below)
+- [`ks2/`](ks2/) — legacy backup server being decommissioned
+  (rental ends Sep 30, 2026): [plan](ks2/plan.md)
+- [`archer-c7/`](archer-c7/) — home router (TP-Link Archer C7 v5,
+  OpenWrt, `192.168.0.2`, WAN = phone USB tether):
+  [upgrade to 25.12](archer-c7/upgrade-openwrt-25.12.md)
 
 ## Homelab: nuc
 
@@ -56,10 +61,15 @@ Incus host at OVH — public-facing self-hosted services.
   outline, login (SSO), wireguard, + more — full table in
   [ks4/install.md](ks4/install.md).
 - ⚠️ The ZFS `data` pool is single-disk (not mirrored); durability
-  rests on two nightly root cron jobs — `incus copy --refresh` to remote
-  host `ks2` (moving to nuc pool `ks4backup`, see
-  [ks4/incus-copy.md](ks4/incus-copy.md)), and an rsync backup
-  (`scripts/incus-backup.sh`) to `164.132.173.57:/backup/ks4`.
+  rests on nightly root cron jobs — 01:00 `incus copy --refresh` of
+  all instances to the local `backup` pool on sdb5
+  ([ks4/local-backup-cron.md](ks4/local-backup-cron.md),
+  [ks4/incus-copy.md](ks4/incus-copy.md)), 04:30 plakar backups
+  (DB dumps + data trees) to S3
+  ([ks4/plakar-s3-data.md](ks4/plakar-s3-data.md)), plus during the
+  transition a 04:00 rsync stopgap to ks2
+  ([ks2 decommission plan](ks2/plan.md)); a pull leg to nuc pool
+  `ks4backup` awaits FTTH.
 
 ## Network flows (nuc <-> ks4)
 

archer-c7/upgrade-openwrt-25.12.md

@@ -0,0 +1,193 @@
+# Archer C7: upgrade OpenWrt 19.07.4 → 25.12.5
+
+> **Outcome (2026-08-20):** upgrade done (flashed clean per this doc).
+> The USB-tether flapping survived the new kernel → root cause is the
+> C7's USB port lacking charging current (phone dropped off the bus
+> whenever it drew charge). **WAN is now the phone's Wi-Fi hotspot**:
+> 2.4 GHz `radio1` in STA mode → interface `wwan` (`phy1-sta0`, in the
+> wan zone), phone on a mains charger. `usbwan` stays configured for a
+> future powered-USB-hub wired option. Post-flash gotcha: fw4 compiled
+> NAT for `eth0.2` only while `usbwan` was down at reload time — LAN
+> had no internet until `service firewall restart` with the WAN
+> interface up.
+
+Home router **TP-Link Archer C7 v5** (`tplink,archer-c7-v5`, target
+`ath79/generic`, 16 MB flash / 128 MB RAM) at `192.168.0.2`, running
+OpenWrt 19.07.4 (2020, kernel 4.14, EOL since 2022). Target release:
+**OpenWrt 25.12.5** (kernel 6.12.94, firewall4/nftables, `apk` package
+manager instead of `opkg`).
+
+Context (2026-08-19): after a reboot the router came up with **dnsmasq,
+firewall and odhcpd disabled** (missing `/etc/rc.d/` symlinks — no DHCP,
+no NAT). All three were restarted and re-enabled, but the incident plus
+the EOL release motivated this upgrade. A fresh flash also rules out
+any other overlay damage.
+
+Current setup worth knowing:
+
+- **WAN = OPPO phone (SM8350) USB-tethered** on `usb0` via RNDIS
+  (`kmod-usb-net-rndis`), interface `usbwan`, DHCP from the phone
+  (10.205.61.0/24). The ethernet WAN port is unused — the ISP box at
+  `.1` is dead, which is also why the router LAN IP is `.2`.
+- SSH: `ssh -i id_rsa_claude root@192.168.0.2` (key at perso repo root;
+  19.07's dropbear needs
+  `-o HostKeyAlgorithms=+ssh-rsa -o PubkeyAcceptedAlgorithms=+ssh-rsa`
+  from a modern OpenSSH; 25.12 won't)
+- DNS: dnsmasq forwards everything to **blocky** at `192.168.0.254`
+  (`noresolv` + `server=192.168.0.254`), and DHCP option 6 hands blocky
+  directly to clients
+- Wi-Fi: 5 GHz only (`radio0`, ch 36, VHT80, SSID `HOME`, WPA2-PSK);
+  2.4 GHz radio disabled
+- 7 static DHCP leases (see script below)
+- Only user-installed packages on 19.07: `kmod-usb-net-rndis` + deps
+
+## Prerequisites (done 2026-08-19)
+
+- [x] Config backup: `sysupgrade -b` → `~/code/perso/backup-archer-c7-20260819.tar.gz`
+      (laptop; contains Wi-Fi PSK and dropbear host keys — never commit)
+- [x] Firmware `openwrt-25.12.5-ath79-generic-tplink_archer-c7-v5-squashfs-sysupgrade.bin`
+      downloaded, **sha256 verified** (`f15b16b3…`) against the release
+      `sha256sums`, staged in `~/code/perso/archer-c7-upgrade/` and on
+      the router at `/tmp/` (`/tmp` is RAM — gone after any reboot,
+      re-`scp` if needed)
+- [x] **Offline driver packages** staged in `~/code/perso/archer-c7-upgrade/`:
+      `kmod-mii`, `kmod-usb-net`, `kmod-usb-net-cdc-ether`,
+      `kmod-usb-net-rndis` (all `6.12.94-r1.apk`). Critical: the phone
+      tether is the only WAN and the RNDIS driver is **not** in the
+      default image, so after flashing the router has no internet to
+      install it from — it must be `scp`'d in.
+
+## Upgrade
+
+**Do not keep settings across this jump** (19.07 → 25.12 spans the
+fw3→fw4 and network-syntax migrations). Flash clean with `-n` and
+reconfigure with the script below.
+
+You need the **laptop on ethernet** to a LAN port: the fresh image
+boots at `192.168.1.1` with Wi-Fi disabled.
+
+```sh
+# on the router — image already in /tmp, verify then flash
+sha256sum /tmp/openwrt-25.12.5-ath79-generic-tplink_archer-c7-v5-squashfs-sysupgrade.bin
+# expect f15b16b36df7fce8a5cf1302320fdb428c3bb6fbc2949a4858afa257f4f364a2
+sysupgrade -n -v /tmp/openwrt-25.12.5-ath79-generic-tplink_archer-c7-v5-squashfs-sysupgrade.bin
+```
+
+The router reboots (~3 min, don't power-cycle). Then from the laptop
+(ethernet, DHCP gives a 192.168.1.x address):
+
+```sh
+# passwordless root at first boot — set a password and install the key
+ssh root@192.168.1.1        # no password
+passwd
+mkdir -p /etc/dropbear && exit
+scp ~/code/perso/id_rsa_claude.pub root@192.168.1.1:/etc/dropbear/authorized_keys
+
+# push the offline RNDIS drivers
+scp ~/code/perso/archer-c7-upgrade/kmod-*.apk root@192.168.1.1:/tmp/
+```
+
+## Reconfigure
+
+Run on the router. Single source of truth for the whole config;
+`<wifi-psk>` is in the backup tarball (`etc/config/wireless`).
+
+```sh
+#!/bin/sh
+set -eux
+
+# RNDIS drivers (offline — phone tether is the only WAN)
+apk add --allow-untrusted /tmp/kmod-mii-*.apk /tmp/kmod-usb-net-6*.apk \
+    /tmp/kmod-usb-net-cdc-ether-*.apk /tmp/kmod-usb-net-rndis-*.apk
+
+# --- network: keep the generated switch/bridge config, change only what we need
+uci set network.lan.ipaddr='192.168.0.2'
+uci set network.usbwan=interface
+uci set network.usbwan.proto='dhcp'
+uci set network.usbwan.device='usb0'
+uci commit network
+
+# --- firewall: usbwan joins the wan zone (zone[1] = 'wan' in the default config)
+uci add_list firewall.@zone[1].network='usbwan'
+uci commit firewall
+
+# --- dhcp/dns: forward to blocky, hand blocky to clients, static leases
+uci set dhcp.@dnsmasq[0].noresolv='1'
+uci add_list dhcp.@dnsmasq[0].server='192.168.0.254'
+uci add_list dhcp.lan.dhcp_option='6,192.168.0.254'
+add_host() {
+  uci add dhcp host >/dev/null
+  uci set dhcp.@host[-1].mac="$1"
+  uci set dhcp.@host[-1].ip="$2"
+  uci set dhcp.@host[-1].name="$3"
+  uci set dhcp.@host[-1].dns='1'
+}
+add_host 'F4:5C:89:BE:C1:C3' '192.168.0.20'  'LAPTOP719974'
+add_host '10:66:6A:87:A1:49' '192.168.0.254' 'blocky'
+add_host '10:66:6A:C8:E1:0A' '192.168.0.5'   'jellyfin-server'
+add_host '10:66:6A:FF:59:1C' '192.168.0.6'   'jellyfin-client'
+add_host '10:66:6A:85:20:B7' '192.168.0.11'  'privoxy'
+add_host '10:66:6A:55:9F:10' '192.168.0.7'   'transmission-bt'
+add_host '50:9A:4C:20:D9:2B' '192.168.0.21'  'patate'
+uci commit dhcp
+
+# --- wifi: 5 GHz AP as before, upgraded to WPA2/WPA3 mixed; 2.4 GHz stays off
+uci set wireless.radio0.channel='36'
+uci set wireless.radio0.htmode='VHT80'
+uci set wireless.radio0.disabled='0'
+uci set wireless.default_radio0.ssid='HOME'
+uci set wireless.default_radio0.encryption='sae-mixed'
+uci set wireless.default_radio0.key='<wifi-psk>'
+uci set wireless.default_radio0.network='lan'
+uci commit wireless
+
+reload_config
+/etc/init.d/network restart
+```
+
+The LAN IP change kicks you off; reconnect at `192.168.0.2` (plug the
+phone back in for WAN).
+
+## Verify
+
+```sh
+ssh root@192.168.0.2
+ifstatus usbwan | grep -E 'up|address'   # up:true, 10.205.61.x
+ping -c2 8.8.8.8 && ping -c2 openwrt.org # WAN + DNS via blocky
+nft list chain inet fw4 srcnat_wan       # masquerade on usbwan
+ls /etc/rc.d/ | grep -E 'dnsmasq|firewall|odhcpd'   # all enabled
+```
+
+Then from a LAN client: renew DHCP (expect 192.168.0.100–249, DNS
+`.254`), check internet, check the static leases land on their IPs.
+
+## Known issue: 5 GHz `ath10k-ct`
+
+The C7 v2/v5 has a long-standing 5 GHz issue with the default
+`ath10k-ct` driver: `SWBA overrun on vdev 0` log spam plus client
+drops under some loads. **Only if it shows up**, swap to the non-ct
+driver (needs working WAN, or pre-download the same way as the RNDIS
+apks):
+
+```sh
+apk del kmod-ath10k-ct ath10k-firmware-qca988x-ct
+apk add kmod-ath10k ath10k-firmware-qca988x
+reboot
+```
+
+## Recovery
+
+If the flash bricks: C7 v5 has TFTP recovery — hold **Reset** while
+powering on, router pulls `ArcherC7v5_tp_recovery.bin` (the OpenWrt
+**factory** image renamed to that) from a TFTP server at
+`192.168.0.66`. Factory image is in the same release directory as the
+sysupgrade one.
+
+## Post-upgrade cleanup
+
+- Delete `/tmp` copies (automatic on reboot)
+- Keep `~/code/perso/backup-archer-c7-20260819.tar.gz` until the new
+  setup has survived a reboot + a few days
+- `~/code/perso/archer-c7-upgrade/` can go once the RNDIS drivers are
+  installed and Wi-Fi is confirmed stable (or keep the apks in case of
+  a re-flash)

ks2/decommission.md

@@ -0,0 +1,60 @@
+# ks2 decommission — final checklist
+
+Status: **prepared — execute only when every gate in
+[plan.md](plan.md) is ticked.** Rental ends **Sep 30, 2026**; leave a
+few days of margin.
+
+## Preconditions (the plan's release gates)
+
+- [ ] plakar S3 leg running nightly for ≥ a week, `done (rc=0)`,
+      restore test passed ([plakar-s3-data.md](../ks4/plakar-s3-data.md) §7)
+- [ ] nuc pull leg seeded and one instance test-restored
+      ([nuc-seed.md](nuc-seed.md))
+- [ ] local leg (sdb5) cron green in `/var/log/incus-copy.log`
+
+## 1. Cut the last flows to ks2 (root on ks4)
+
+```sh
+crontab -e   # delete the 04:00 incus-backup.sh line (164.132.173.57)
+incus remote list                 # if a "ks2" remote is still defined:
+incus remote remove ks2
+# drop the rsyncbackup SSH key if it exists only for ks2:
+ls /root/.ssh/id_rsyncbackup* && rm -i /root/.ssh/id_rsyncbackup*
+```
+
+## 2. Last look at ks2 before wiping (root on ks2)
+
+```sh
+# anything in the rsync tree newer than the plakar cutover?
+find /backup/ns3061243 -newermt '2026-08-23' | head
+# pre-2023 dirs (catc, mythoughts, qcm, scripts): instances still
+# exist stopped on ks4 — spot-check nothing unique remains:
+ls /backup/ns3061243/{catc,mythoughts,qcm,scripts}
+```
+
+Archive anything unique to the plakar dump dir on ks4 (it rides the
+nightly `dumps` snapshot to S3 for free once it stops changing).
+
+## 3. Wipe and terminate
+
+```sh
+# on ks2 — destroy the data (both pools hold copies of personal data):
+zpool destroy backup && zpool destroy data
+# optional: shred the OS disks via OVH rescue mode / reinstall
+```
+
+Then in the OVH manager: cancel the ks2 (Kimsufi) server before the
+renewal date.
+
+## 4. Post-mortem cleanup
+
+- [ ] Update [doc/README.md](../README.md): remove ks2 from the ks4
+      durability bullet and the index line for `ks2/`s "being
+      decommissioned" wording (keep the folder as history)
+- [ ] Retire `scripts/incus-backup.sh` + `incus-backup.db` in the repo
+      (superseded by `scripts/plakar-backup.sh`) — keep for reference,
+      note the retirement in their headers
+- [ ] `login` container cleanup parked in
+      [plakar-s3-data.md](../ks4/plakar-s3-data.md): `dropdb outline`,
+      remove native postgres
+- [ ] Mark the ks2 line in `plan.md` done, date it

ks2/nuc-seed.md

@@ -0,0 +1,49 @@
+# nuc pull leg — seed after FTTH
+
+Status: **prepared, waiting on the FTTH link.** Everything is already
+configured on nuc (see the main [README](../README.md)): incus remote
+`ks4` over the WireGuard tunnel (`wg-ks4`, 10.8.0.20 → 10.8.0.1),
+target pool `ks4backup` on the USB ZFS pool (`usb4t/backup/ks4`) —
+only the seed itself waited on bandwidth.
+
+## Seed (root on nuc)
+
+```sh
+# sanity: remote reachable through the tunnel
+incus list ks4: | head
+
+# full pull of every ks4 instance into pool ks4backup (screen/tmux —
+# first pass moves ~1.7 T through the WG tunnel)
+/root/scripts/incus-copy.sh -r ks4 -s ks4backup 2>&1 | tee -a /var/log/incus-copy-ks4.log
+```
+
+Notes:
+
+- First pass is a full send per instance; later refreshes are
+  ZFS-incremental **as long as they run at least every
+  `snapshots.expiry` (7 d on ks4)** — same caveat as
+  [ks4's local leg](../ks4/local-backup-cron.md).
+- Replicas arrive stopped with `boot.autostart=false` (script does
+  this) — they must never come up on the LAN with ks4's proxy devices.
+
+## Cron (after the seed)
+
+Add to nuc's root crontab, offset from the 03:30 local nucbackup copy
+and ks4's own 01:00/04:30 jobs:
+
+```cron
+0 5 * * * /root/scripts/incus-copy.sh -r ks4 -s ks4backup >> /var/log/incus-copy-ks4.log 2>&1
+```
+
+## Verification (release gate for ks2)
+
+```sh
+incus list --project backup 2>/dev/null || incus list | grep -c .   # all ks4 instances present
+# test-restore one instance: copy a replica to the default pool,
+# start it isolated, check the service answers, then delete it
+incus copy solar solar-restoretest -s default
+incus start solar-restoretest && incus exec solar-restoretest -- systemctl is-system-running
+incus delete -f solar-restoretest
+```
+
+Once verified, tick the nuc gate in the [ks2 plan](plan.md).

ks2/plan.md

@@ -0,0 +1,78 @@
+# ks2 decommission — summary plan
+
+Goal: release `ks2.lutran.fr` (`164.132.173.57`, SSH :2233, host
+`ns3247221`) once ks4 has a real 3-2-1 backup without it.
+**Deadline: rental ends Sep 30, 2026.**
+
+## What ks2 does today (updated 2026-08-22)
+
+One remaining role, fed nightly by a root cron **on ks4**:
+
+- `04:00` `incus-backup.sh … -d 164.132.173.57` — DB dumps + selected
+  paths rsync'd to ks2 `/backup/ns3061243` on pool `backup` (1.59 T,
+  88 % full), snapshotted daily on ks2 (`zfs-auto-snapshot.sh`,
+  2-month expiry). **Kept as a stopgap** until the plakar S3 leg
+  replaces it.
+
+Retired: the `01:00` replica push to ks2 pool `data` — replaced by
+the local leg on ks4's sdb5 pool
+([local-backup-cron.md](../ks4/local-backup-cron.md)); `biwiki` and `spot`
+deleted from ks2.
+
+## Inventory findings (2026-08-22)
+
+- `biwiki` exists only on ks2, `spot` only on ks2 + ks4's `backup`
+  project — **decision: both abandoned**, safe to delete from backups.
+- `incus-backup.db` is stale (lists `spot`, misses `livetrail`,
+  `outline`, `login`, …) — forgotten-manifest drift is exactly what
+  the plakar-based replacement must eliminate (back up *all*
+  instances by default, opt-out instead of opt-in).
+- `livetrail` (running on ks4) is **absent from ks4's local `backup`
+  project** — the local leg was run manually once and never again;
+  automating its cron fixes this.
+- Both ks4 backup crons were **commented out — no ks4 backup ran at
+  all** (hence the stale ks2 rsync data: dirs `Jul 23`, incus dumps
+  `Aug 9`). Fixed 2026-08-22 — see
+  [local-backup-cron.md](../ks4/local-backup-cron.md).
+- `/backup/ns3061243` also holds pre-2023 dirs (`catc`, `mythoughts`,
+  `qcm`) — those instances still exist stopped on ks4, so nothing
+  unique expected there; spot-check before wiping.
+- Losing ks2 also loses its 2 months of rsync-snapshot history —
+  acceptable once plakar has built equivalent retention.
+
+## Target architecture (3-2-1 for ks4)
+
+| Leg | Mechanism | Status |
+|---|---|---|
+| local, 2nd disk | `incus-copy.sh -p backup -s backup` → sdb5 pool | works, **cron missing** |
+| off-site, nuc | nuc pulls `ks4:*` → pool `ks4backup` over WG | waiting FTTH seed |
+| off-site, S3 (data+DB) | plakar on ks4 → S3 (replaces `incus-backup.sh`) | to do |
+| off-site, S3 (instances) | plakar **incus integration** (to develop) → S3 | to do |
+
+## Actions (backup work documented in [`../ks4/`](../ks4/), ks2-only tasks here)
+
+1. ~~salvage biwiki/spot~~ — decided 2026-08-22: both abandoned,
+   **deleted** from ks2 and ks4's `backup` project
+2. ~~[local-backup-cron.md](../ks4/local-backup-cron.md)~~ — **done
+   2026-08-22**: leg 1 cron re-enabled, ks2 rsync kept as stopgap;
+   `livetrail` verified present in the `backup` project
+3. [plakar-s3-data.md](../ks4/plakar-s3-data.md) — **seeding S3
+   2026-08-22** (direct-to-S3 kloset `lutran-ks4-plakar-data`, driver
+   script `scripts/plakar-backup.sh`, pipeline validated locally);
+   next: restore test + cron
+4. [plakar-incus-integration.md](../ks4/plakar-incus-integration.md) —
+   design seeded 2026-08-22: per-file importer over the incus sftp
+   API, instances → S3
+5. [nuc-seed.md](nuc-seed.md) — **prepared**; after FTTH: seed nuc
+   pull leg, verify all instances, test-restore one
+6. [decommission.md](decommission.md) — **prepared**; cut flows,
+   final diff of `/backup/ns3061243`, wipe pools, terminate at OVH
+
+## Release gates — ks2 can be dropped only when
+
+- [x] biwiki + spot consciously abandoned (2026-08-22)
+- [ ] local leg cron running ≥ a few days, all instances present
+- [ ] nuc leg fully seeded **and** one instance test-restored
+- [ ] plakar S3 data/DB backups running daily **and** one DB + one
+      file tree test-restored
+- [ ] ks4 crons pointing at ks2 disabled

ks4/local-backup-cron.md

@@ -0,0 +1,56 @@
+# local backup cron on ks4 (leg 1) — re-enabled
+
+Done 2026-08-22. Context: **both** ks4 backup crons had been commented
+out — no backup of any kind was running.
+
+## What runs now (root crontab on ks4)
+
+```cron
+0 1 * * * /root/scripts/incus-copy.sh -p backup -s backup >> /var/log/incus-copy.log 2>&1
+0 4 * * * /root/scripts/incus-backup.sh -d 164.132.173.57 -u rsyncbackup -i /root/.ssh/id_rsyncbackup -p 2233 -f /root/scripts/incus-backup.db -s data >> /var/log/incus-backup.log 2>&1
+```
+
+- `01:00` — **local leg**: `incus copy --refresh` of all instances
+  into project `backup` on the sdb5 `backup` zpool (replaces the old
+  push to ks2; see [incus-copy.md](incus-copy.md)).
+- `04:00` — **stopgap**: `incus-backup.sh` rsync to ks2 kept alive
+  (manifest pruned of `spot`) until the plakar S3 leg replaces it —
+  then this line goes away as part of
+  [decommissioning ks2](../ks2/plan.md). ⚠️ keep
+  `scripts/incus-backup.db` in this repo in sync with
+  `/root/scripts/incus-backup.db` on ks4.
+
+Both were also run once by hand (in a `screen`) on 2026-08-22 to
+refresh the backups immediately. The old push-to-ks2 copy line stays
+commented out — replica refreshes to ks2 are retired.
+
+## Cleanup done the same day
+
+- ks2: `incus delete biwiki spot` (abandoned instances — freed space
+  on the 95 %-full `data` pool)
+- ks4: `incus delete spot --project backup`
+
+## Verification
+
+```bash
+# all instances present, incl. livetrail (first copy is a full send)
+sudo incus list --project backup -c ns -f csv
+# cron runs land here (manual screen runs don't):
+sudo tail /var/log/incus-copy.log /var/log/incus-backup.log
+# rsync leg freshness, on ks2:
+ls -lat /backup/ns3061243/ | head
+```
+
+Notes:
+
+- `flock` in both scripts makes overlapping cron/manual runs abort
+  safely instead of racing.
+- Replicas get `boot.autostart=false` set by the script — they must
+  never fight the live instances for their static IPs.
+- Keep `snapshots.schedule` set on the sources so refreshes stay
+  ZFS-incremental (a refresh without common snapshots falls back to
+  rsync). Corollary: the cron must run at least every
+  `snapshots.expiry` (7 d) — a longer gap expires every common
+  snapshot and the next refresh silently degrades to a **full
+  re-send** (observed 2026-08-22: seafile re-sent 933 G after the
+  weeks-long gap while the crons were commented out).

ks4/plakar-incus-integration.md

@@ -0,0 +1,175 @@
+# plakar incus integration — design notes
+
+Status: **importer scaffolded and compiling 2026-08-23** in
+`perso/plakar/integration-incus` (walker modeled on the official sftp
+integration; enumerate project → per-instance sftp walk → per-file
+records + synthetic `.incus.yaml`; allow/deny lists; VM sftp failures
+reported as record errors, not fatal). See its README for
+build/install/TODO. Untested against a live incus yet.
+
+Goal: plakar source connector for Incus so instance backups land in a
+plakar kloset on S3 (leg 4 of the ks4 3-2-1 plan — see
+[plan.md](../ks2/plan.md)).
+
+## Decision 1 — per-file importer, not tarball streaming
+
+The proxmox integration streams one full vzdump archive per VM
+(`FLAG_STREAM`). For ks4 that model reads ~1.7 TB nightly (seafile
+933 G, nextcloud 725 G): the incus backup/export API produces **full
+tarballs only** — `shared/api/instance_backup.go` has just
+`instance_only` and `optimized_storage`, no incremental option.
+
+Instead the importer emits **one record per file** with real stat info
+and a lazy reader. kloset's engine then does the incremental work for
+us: `snapshot/backup.go` (`checkVFSCache`) reuses the cached object
+when a path's stat matches the previous run and **never opens the
+file**. Nightly cost after the initial seed = stat-walk + changed
+files only; S3 receives only new chunks. Bonus: per-file browse and
+restore of any snapshot.
+
+File access is native incus REST: `GET /1.0/instances/{name}/sftp`
+(a real SFTP session, also available per storage volume). Works over
+the local unix socket or remote :8443, on stopped containers (the
+daemon mounts the volume on demand), no ZFS assumptions.
+
+## Decision 2 — do not reuse `incus copy --refresh` / migration API
+
+Considered and rejected. `copy --refresh` is `POST /1.0/instances`
+with `source.refresh=true` (`shared/api/instance.go:373`) and is
+**not** zfs-only — transports are negotiated per storage driver
+(`MigrationTypes()`: zfs/btrfs/ceph native streams, universal RSYNC
+fallback, `driver_common.go:230`). But:
+
+- the receiver must be another incus daemon speaking the migration
+  websocket protocol (`internal/migration/migrate.proto`);
+- the optimized payload is an opaque `zfs send` stream — storing
+  those in plakar means chain-of-increments restore onto a real
+  zpool, no per-file browse, no pruning, one corrupt link breaks the
+  chain;
+- the RSYNC transport needs a materialized previous copy on the
+  receiver — a full-size local mirror, i.e. leg 1 rebuilt inside a
+  plugin;
+- gotcha: `driver_zfs.go:801` — refresh **without** snapshot copying
+  forces RSYNC even zfs↔zfs (keep `snapshots.schedule` on sources).
+
+`copy --refresh` stays what legs 1–2 use (host↔host replication);
+plakar is content-addressed archiving to dumb storage. Complementary,
+not competing.
+
+## Decision 3 — read the `backup` project replicas, not live instances
+
+Reading a live rootfs is fuzzy. Leg 1 (`incus-copy.sh -p backup -s
+backup`, 01:00) already produces quiescent, crash-consistent stopped
+replicas — the importer targets those (`?project=backup`), scheduled
+after the copy completes. DB consistency remains the mariadb-dump
+job's responsibility (plakar `mysql` importer / dump step).
+
+## Decision 4 — enumerate everything by default
+
+`incus-backup.db` manifest drift (stale `spot`, missing `livetrail`,
+`outline`, `login`…) is the failure mode to kill: the importer backs
+up **all instances in the project by default**, opt-out via config,
+never opt-in.
+
+## Other design points
+
+- Location scheme `incus://<remote>/<project>/<instance>/...` —
+  stable pathnames keep the kloset VFS cache effective.
+- Synthetic records per instance: config + profiles
+  (`GET /1.0/instances/{name}`, expanded) so restore can recreate the
+  instance before pushing files back.
+- Optional later `mode=image`: full tarball via the backups API for
+  occasional exact-image restores (`incus import`); plakar dedups
+  unchanged chunks between runs. Not the nightly path.
+- Restore path (exporter): create instance from stored config → push
+  file tree back via sftp. Weaker than `incus import` but granular.
+
+## Upstreaming (decided 2026-08-23: Route B — community integration)
+
+Repo: `github.com/jlutran/plakar-integration-incus` (remote set,
+initial commit done). Path to publication, in order:
+
+1. unit tests + first real run against ks4's `backup` project
+2. open a PlakarKorp/plakar issue proposing the integration
+   (disclose the Apache-2.0 `lxc/incus` client dependency)
+3. push the repo, then PR a recipe against `PlakarKorp/hub` →
+   installable as `plakar pkg add incus`
+4. (optional, later) adoption into the `integrations` monorepo:
+   module rename to `github.com/PlakarKorp/integrations/incus`,
+   orphan `integration/incus` branch, `incus/incus` layout
+
+## Packaging & deployment
+
+Plugins are standalone executables (gRPC over stdio) — no rebuild of
+plakar, fully compatible with the APT-installed binary on ks4:
+
+```sh
+cd plakar/integration-incus && make          # build importer/exporter
+plakar pkg create manifest.yaml v0.1.0       # → incus_v0.1.0_linux_amd64.ptar
+# on ks4, as root (plugins are per-user, cron runs as root):
+plakar pkg add ./incus_v0.1.0_linux_amd64.ptar
+plakar source add ks4-incus incus://...      # then list it in /root/scripts/plakar-sources
+```
+
+Installing a *local* `.ptar` file needs no `plakar login` (the plugin
+registry is only for fetching by name). Model the `manifest.yaml` on
+integration-proxmox (`tier: third-party`, `api_version: v1.1.0`).
+Compatibility contract = the go-kloset-sdk wire protocol: pin the SDK
+to the installed plakar line (v1.1.x), rebuild the ptar on a plakar
+major upgrade; a mismatch surfaces at backup time, so run a manual
+`plakar backup @ks4-incus` after plakar upgrades.
+
+## The giant-instance problem (2026-08-24, seed attempt #1)
+
+The per-file S3 seed of the whole `backup` project stalled at
+~1 MiB/s once it reached the big replicas: sdb at 96 % util doing
+~275 × 4 KiB reads/s — **cold random tiny-file reads on an HDD are
+seek-bound**, regardless of importer (the fs importer hit the same
+wall on the live pool; sftp changes nothing). At ~275 IOPS,
+nextcloud (725 G, ~500 k files) + seafile (933 G, millions of block
+files) need *weeks* to seed. The 16 small instances are unaffected
+(solar smoke test: 5 GiB in 2m38s).
+
+Options considered for nextcloud/seafile:
+
+1. **backups-API tarball (`mode=image`)** — sequential-ish but incus
+   materializes the tarball server-side first (no room on `/`; would
+   need `storage.backups_volume` on the backup pool) and the tar walk
+   itself seeks like any tree walk. Double IO nightly. Weak.
+2. **`zfs send` of the replica dataset** — the only truly sequential
+   read (disk block order, ~100+ MiB/s, no temp space): seed in
+   hours. Full send nightly is ~4 h read for both giants; CDC chunking
+   dedups unchanged stream regions. Restore = `zfs receive` +
+   `incus admin recover` (document!). Backend-specific — acceptable as
+   an opt-in mode for exactly these two.
+3. **Per-file + warm metadata** — after a seed, nightly walks are
+   stat-only; keeping dnodes resident (`zfs_arc_min` ≈ 8 G) could make
+   them fast. Doesn't solve the *seed*.
+4. **Exclude giants from this leg** — leaves seafile without any
+   fresh off-site copy (data leg already excludes it). Not acceptable
+   long-term.
+
+Direction (pending decision): seed the 16 small instances per-file
+(`exclude_instances=nextcloud,seafile`), then implement option 2 as a
+`zfs+send://` mode or side-channel for the two giants, and revisit 3
+for nightly stat-walk speed.
+
+## Open questions
+
+- uid/gid view through the instance sftp endpoint (idmap handling for
+  unprivileged containers) — verify what stat returns and what restore
+  must remap.
+- xattrs / ACLs / device nodes over sftp — coverage and fidelity.
+- VMs: file access needs a running incus-agent — out of scope for ks4
+  (containers only); document the limitation.
+- ~~Scheduling/locking~~ decided 2026-08-23: the S3 instance leg is
+  only as fresh as the last **completed** local incus-copy, so both
+  run chained in one cron entry —
+  `incus-copy.sh -p backup -s backup ; <plakar incus leg>` — never at
+  a fixed offset (a long refresh, e.g. a post-gap full re-send, would
+  race it). `;` not `&&`: a partially failed copy leaves stale but
+  internally consistent replicas, still worth uploading. Same rule
+  for manual test runs: check `/var/log/incus-copy.log` completed
+  first.
+- S3 hardening: scoped credentials (no delete), versioning/object
+  lock, plakar client-side encryption passphrase storage.

ks4/plakar-s3-data.md

@@ -0,0 +1,235 @@
+# plakar data + DB backups on ks4 (leg 3)
+
+Status: **live since 2026-08-24** — S3 kloset
+(`lutran-ks4-plakar-data`) seeded (dumps + 9 fs sources), restore
+test passed (file tree + 10 GiB dumps snapshot), 04:30 nightly cron +
+Sunday `check -latest` installed. Replaces `incus-backup.sh` (its
+04:00 ks2 rsync cron is disabled). Open items: `seafile-fs` excluded
+(see §3 warning), `nextcloud-fs` costs 3–4 h/night (cost scales with
+tree size — upstream issue drafted, observing a few nights before
+filing).
+
+Architecture: plakar runs **on ks4** (decision recorded in
+[plan.md](../ks2/plan.md)) and backs up **directly into an S3 kloset**
+(`plakar at @s3 backup @src`). A local staging kloset on sdb5 was
+considered and rejected: it would double-store ~1.5 T that the leg-1
+replicas already hold on the same disk, and local restore is already
+leg 1's job. Direct-to-S3 keeps the incremental behavior — the
+skip-unchanged baseline is the parent snapshot's VFS read from the
+kloset (plakar caches repository metadata locally; v1.1.4's dirpack
+prefetcher targets exactly this remote-backend walk). Driver script:
+[`scripts/plakar-backup.sh`](../../scripts/plakar-backup.sh)
+(flock, incus DB dumps, iterates a sources file, prune + maintenance;
+its `-k` takes any `plakar at` target, so `-k @s3` — a throwaway
+local kloset, e.g. `-k /backup/plakar-test`, works for testing before
+credentials arrive and can be deleted after).
+
+## 1. Install plakar via the official APT repo (root on ks4)
+
+Repo verified reachable 2026-08-22 (suite `stable`, amd64/arm64);
+upgrades then ride the normal `apt upgrade` flow.
+
+```sh
+curl -fsSL https://plakar.io/dist/keys/community-v1.1.0.gpg \
+    | gpg --dearmor -o /usr/share/keyrings/plakar.gpg
+echo "deb [signed-by=/usr/share/keyrings/plakar.gpg] https://plakar.io/dist/repos/deb/ stable main" \
+    > /etc/apt/sources.list.d/plakar.list
+apt update && apt install plakar
+plakar version           # v1.1.4 at time of writing
+```
+
+The mysql integration is a plugin; official plugins install from the
+plugin server after `plakar login`, or build locally (needs make + Go):
+
+```sh
+plakar pkg add mysql        # or: clone integrations, cd mysql && make install
+```
+
+## 2. Kloset + passphrase
+
+```sh
+head -c 32 /dev/urandom | base64 > /root/.plakar-passphrase
+chmod 600 /root/.plakar-passphrase
+# ⚠️ copy the passphrase to the password manager NOW — without it
+# the S3 kloset is unreadable garbage.
+export PLAKAR_PASSPHRASE=$(cat /root/.plakar-passphrase)
+
+# when S3 credentials are available:
+plakar store add s3 s3://<endpoint>/<bucket> access_key=<AK> secret_access_key=<SK> use_tls=true
+plakar at @s3 create
+# meanwhile, a throwaway kloset lets everything below be tested now:
+plakar at /backup/plakar-test create
+```
+
+## 3. Database dumps — `incus exec`, no DB users needed
+
+The plakar mysql integration was considered and rejected: it connects
+over TCP, which would have required a bind-address change plus a
+dedicated backup user in every DB container. Instead the driver
+script keeps the old `incus-backup.sh` method — `incus exec <ct> --
+mariadb-dump --single-transaction ...` (container-local root over the
+unix socket) — with one upgrade: **databases are auto-discovered**,
+so there is no manifest to go stale (the failure mode that bit
+`incus-backup.db`). Three probes per *running* container:
+
+- native MariaDB/MySQL (either binary naming — `mail` has a
+  hand-installed `mariadb-dump` + mysql-named client; every
+  non-system DB) → `/backup/plakar-dumps/mariadb/<ct>/<db>.sql`,
+  plus a replayable users/grants dump (`SHOW GRANTS` per user, incl.
+  password hashes) → `grants.sql` — so the S3 dump path is
+  self-sufficient without the replicas, mirroring pg's `globals.sql`
+- native PostgreSQL (`pg_dump` per DB + `pg_dumpall --globals-only`
+  for roles, peer auth as `postgres`) →
+  `/backup/plakar-dumps/postgres/<ct>/`
+- PostgreSQL **inside docker** (covers `outline`/`login`): docker
+  containers whose image name contains "postgres", dump user from
+  `$POSTGRES_USER` → `/backup/plakar-dumps/postgres/<ct>/<docker-name>/`
+  (a pg bundled in an image *not* named `*postgres*` would be missed —
+  check the nightly log's `dump` lines after adding such a service)
+
+Stopped containers (e.g. `databap`) are skipped —
+their static state is covered by the leg-1 replicas.
+
+Opt-out: `/root/scripts/plakar-db-exclude` (optional) lists dump
+paths to skip, one per line, exactly as the `dump` log lines print
+them (`<ct>/<db>` or `<ct>/<docker-name>/<db>`). Every skip is logged
+(`SKIP <path>`), so unlike the old manifest the list can't rot
+silently. Applied 2026-08-22: `livetrail/geo` (10 GiB static map
+data, 3 h dump). Its last dump deliberately stays in the dump dir:
+unchanged → skipped by the VFS cache, so every nightly `dumps`
+snapshot keeps carrying it to S3 at zero cost. To refresh after a
+re-import: drop the exclude line for one night.
+
+Dumps are staged **plain, not gzipped**: plakar chunks and compresses
+anyway, and CDC dedup across near-identical consecutive dumps only
+works on uncompressed input. The dump tree (incus's own DBs +
+MariaDB) is backed up as one snapshot tagged `dumps`. The nightly log
+lists every `dump <ct>/<db>` line — a database disappearing from the
+log is visible. (The installed mysql plugin is now unused; harmless
+to keep.)
+
+Beyond databases, `outline`/`login` may hold non-DB state (uploaded
+files, docker volumes) — decide per instance whether an fs source is
+needed in §4.
+
+⚠️ **seafile-fs excluded from the data leg (2026-08-24).** Its S3
+seed ran 12 h with no visible upload and was killed. Successive
+theories (EINTR livelock — `GODEBUG=asyncpreemptoff=1` changed
+nothing; kernel reclaim stall — PSI counters too low; ARC-starved
+lstats — disproved by `find` walking nextcloud's whole tree in 8m20
+cold / 19 s warm) all fell to evidence; cause **unknown**. Next
+diagnostic: bisect by seeding seafile-fs into a *local* scratch
+kloset (`plakar at /backup/plakar-scratch create` + timed backup) —
+completes fine → S3 store path implicated; crawls → importer-side on
+that tree shape (millions of tiny block files); then the
+maintainer-suggested `-trace=all` (writes to stdout). Until resolved,
+seafile's fresh off-site coverage waits on the incus leg reading the
+sdb replica.
+
+Findings from the first dry-run (2026-08-22):
+
+- `livetrail`'s `/root/.my.cnf` had a stale `database = spot` default
+  that broke discovery — fixed 2026-08-22 (its DB is `livetrail`).
+- `login` still runs a **native** postgres holding the pre-migration
+  `outline` DB (live Outline uses `outline-postgres-1`). Dumped
+  harmlessly for now; cleanup once the S3 leg is verified:
+  `dropdb outline` + remove the native postgres package.
+
+## 4. Filesystem sources
+
+Derived from the old `incus-backup.db` manifest (minus `spot`; DBs
+are handled by §3's auto-discovery). FS paths are read from the live
+containers' rootfs on the host — crash-consistent, same fidelity as
+the old rsync (the per-file
+[incus integration](plakar-incus-integration.md) supersedes this
+later). Verify the pool mount prefix first:
+`ls /var/lib/incus/storage-pools/data/containers/mail/rootfs/`.
+
+`outline`/`login` non-DB state identified 2026-08-22 via
+`docker inspect` (Outline's local file storage volume; authentik's
+data + custom templates — `/opt/authentik` also catches the compose
+file). ⚠️ Still to review: whether `livetrail` holds state.
+
+Register sources (root on ks4):
+
+```sh
+R=/var/lib/incus/storage-pools/data/containers
+plakar source add bitwarden-fs  $R/bitwarden/rootfs/opt/bitwarden
+plakar source add gateway-fs    $R/gateway/rootfs/var/www
+plakar source add git-fs        $R/git/rootfs/home/git/projects
+plakar source add login-fs      $R/login/rootfs/opt/authentik
+plakar source add mail-vmail    $R/mail/rootfs/var/vmail
+plakar source add mail-www      $R/mail/rootfs/var/www
+plakar source add nextcloud-fs  $R/nextcloud/rootfs/nextcloud
+plakar source add outline-fs    $R/outline/rootfs/var/lib/docker/volumes/outline_storage-data/_data
+plakar source add seafile-fs    $R/seafile/rootfs/opt/seafile
+plakar source add solar-fs      $R/solar/rootfs/var/www/html/solar
+```
+
+Then list the names (one per line) in `/root/scripts/plakar-sources` —
+the script iterates that file.
+
+## 5. Script + cron
+
+```sh
+# copy scripts/plakar-backup.sh from this repo to /root/scripts/, then:
+chmod +x /root/scripts/plakar-backup.sh
+/root/scripts/plakar-backup.sh -k /backup/plakar-test   # dry-run the pipeline now
+/root/scripts/plakar-backup.sh -k @s3                   # first real run (once creds exist)
+plakar at @s3 ls                                        # inspect snapshots
+```
+
+Cron (04:30, after the 04:00 stopgap so they don't hammer the same
+DBs simultaneously; move to 04:00 once the rsync stopgap is dropped).
+Inline `backup -check` was removed 2026-08-22: against S3 it re-reads
+every written chunk (~2.6 MB/s latency-bound; gateway-fs: 7 min
+backup, 23 min check — the giants would add 10–20 h nightly).
+Verification is a weekly deep check of the latest snapshots instead:
+
+```cron
+30 4 * * * /root/scripts/plakar-backup.sh -k @s3 >> /var/log/plakar-backup.log 2>&1
+0 12 * * 0 PLAKAR_PASSPHRASE=$(cat /root/.plakar-passphrase) plakar -quiet at @s3 check -latest >> /var/log/plakar-check.log 2>&1
+```
+
+Once validated against @s3, delete the throwaway kloset:
+`rm -rf /backup/plakar-test`.
+
+## 6. S3 hardening
+
+Scoped credentials (no delete permission if the provider allows it),
+bucket versioning or object lock, and prefer an OVH region — ks4 →
+OVH Object Storage stays on the OVH backbone, so backup and restore
+speed is LAN-like.
+
+## 7. Restore test (release gate)
+
+Two restores from `@s3`, exercising both data types:
+
+```sh
+export PLAKAR_PASSPHRASE=$(cat /root/.plakar-passphrase)
+plakar at @s3 ls          # expect 11 snapshots: dumps + 10 fs sources
+
+# (a) file tree — restore solar-fs and diff against the live tree
+plakar at @s3 restore -to /backup/restore-test <solar-fs-snap-id>
+diff -r /backup/restore-test/var/lib/incus/storage-pools/data/containers/solar/rootfs/var/www/html/solar \
+        /var/lib/incus/storage-pools/data/containers/solar/rootfs/var/www/html/solar
+rm -rf /backup/restore-test
+
+# (b) database — restore a dump and load it into a scratch container
+plakar at @s3 restore -to /backup/restore-db <dumps-snap-id>
+incus launch images:ubuntu/24.04 dbtest
+incus exec dbtest -- apt-get install -y mariadb-server
+incus file push /backup/restore-db/backup/plakar-dumps/mariadb/freshrss/freshrss.sql dbtest/root/
+incus exec dbtest -- sh -c 'mariadb < /root/freshrss.sql'   # dump embeds CREATE DATABASE
+incus exec dbtest -- mariadb -e 'SELECT COUNT(*) FROM information_schema.tables WHERE table_schema="freshrss"'
+incus delete -f dbtest && rm -rf /backup/restore-db
+```
+
+Pass = diff empty (modulo files changed since the snapshot) and the
+scratch DB loads with a sane table count.
+
+## Retirement
+
+Once this leg is verified (and once more from @s3): remove the 04:00
+`incus-backup.sh` cron to ks2 — precondition for
+[decommissioning ks2](../ks2/plan.md).

ks4/seafile-gcid-fix.md

@@ -0,0 +1,109 @@
+# seafile — GCID table loss, 6-month CPU burn (fixed 2026-08-22)
+
+`seaf-server` in the `seafile` container sat at 40–47 % CPU for months
+while appearing to do nothing: negligible client traffic on `eth0`
+(344 MB sent over 102 days) but 82 GB and 385 M packets over loopback,
+and ~11 **days** of accumulated CPU time.
+
+## Root cause
+
+The `seafile-db` MariaDB database was missing the `GCID` and
+`LastGCID` tables. Since Seafile 11 every client sync operation checks
+a per-repo GC id (`SELECT gc_id FROM GCID WHERE repo_id = ? FOR
+UPDATE`); with the table gone the query failed, the client retried,
+and the cycle repeated forever — no sync ever completed:
+
+- 6 of seaf-server's fileserver worker threads spinning in pure
+  user-space, ~3 new MySQL connections/s (167 loopback `TIME_WAIT`
+  to :3306 at any moment),
+- `seafile.log` grew to **1.8 GB** of the single warning
+  `Failed to prepare sql SELECT gc_id FROM GCID … Table
+  'seafile-db.GCID' doesn't exist`, several times per second.
+
+First occurrence **2026-03-01 22:17** — under 11.0.9, i.e. *before*
+both the 2026-03-16 restart and the 2026-05-11 upgrade to 12.0.14.
+`GCID` is an old core table (the 6.3.0 upgrade SQL already ALTERs
+it), so it existed and was lost. **What dropped it around 2026-03-01
+was never identified** — if tables vanish again, start there.
+
+## Diagnosis trail (what worked without container root)
+
+From the host as `julien` (sudo grant covers read-only `incus`
+subcommands only):
+
+```sh
+sudo incus info seafile                    # PID 2410, 1005 GiB, 2 GiB RAM
+ps -eo pid,%cpu,args --sort=-%cpu          # seaf-server pid 10217 top hog
+# per-thread CPU from /proc (world-readable, no root needed):
+for t in /proc/10217/task/*; do awk '{print $1, $14+$15}' $t/stat; done
+# utime vs stime split → pure user-space compute, ~0 syscalls
+# container netns TCP table via the process:
+awk 'NR>1 && $4=="06"' /proc/10217/net/tcp | wc -l   # 167 TIME_WAIT → :3306
+```
+
+Reading `/opt/seafile/logs/seafile.log` needed a temporary sudoers
+extension (`/usr/bin/incus exec *`, `/usr/bin/incus file pull *` in a
+drop-in — **removed again after the fix**). The log line then made the
+cause obvious.
+
+## Fix
+
+The exact DDL ships inside the server binary:
+
+```sh
+strings /opt/seafile/seafile-server-12.0.14/seafile/bin/seaf-server \
+  | grep 'CREATE TABLE IF NOT EXISTS.*GCID'
+```
+
+Applied (root in the container; additive, no restart needed):
+
+```sql
+CREATE TABLE IF NOT EXISTS GCID (
+  id BIGINT NOT NULL PRIMARY KEY AUTO_INCREMENT,
+  repo_id CHAR(36), gc_id CHAR(36),
+  UNIQUE INDEX(repo_id)) ENGINE=INNODB;
+CREATE TABLE IF NOT EXISTS LastGCID (
+  id BIGINT NOT NULL PRIMARY KEY AUTO_INCREMENT,
+  repo_id CHAR(36), client_id VARCHAR(128), gc_id CHAR(36),
+  UNIQUE INDEX(repo_id, client_id)) ENGINE=INNODB;
+```
+
+The warning storm stopped the same second, seaf-server dropped to
+0 % CPU, host load fell from ~8–12 toward normal. `seafile.log` was
+truncated (`truncate -s 0`).
+
+## Log rotation (added 2026-08-22)
+
+Seafile ships **no rotation** — that's how one warning reached 1.8 GB
+unnoticed. `/etc/logrotate.d/seafile` in the container:
+
+```
+/opt/seafile/logs/*.log {
+	weekly
+	maxsize 100M
+	rotate 8
+	missingok
+	notifempty
+	compress
+	delaycompress
+	copytruncate
+}
+```
+
+`copytruncate` avoids pid/SIGUSR1 handling for the various daemons
+(seaf-server, seahub, seafevents…) at the cost of possibly losing a
+few lines at rotation — fine here. `maxsize 100M` caps a future spam
+storm at the next daily logrotate run instead of next week.
+
+## Related cleanup / leftovers
+
+- Legacy 4 AM cron on ks4 rsyncing seafile's rootfs to old **ks2**
+  (`164.132.173.57`) removed 2026-08-22 — redundant with the
+  [incus-copy](incus-copy.md) legs and it burned another ~47 % CPU
+  per run.
+- Still missing from `seafile-db` (harmless, nothing queries them
+  unless per-folder permissions are used): `FolderUserPerm`,
+  `FolderGroupPerm`, `FolderPermTimestamp`. Same `strings`/upgrade-SQL
+  trick provides the DDL if ever needed.
+- Host swap (1 GB) was 100 % full during the incident; unrelated but
+  worth keeping an eye on.