diff --git a/archer-c7/upgrade-openwrt-25.12.md b/archer-c7/upgrade-openwrt-25.12.md new file mode 100644 index 0000000..a36df88 --- /dev/null +++ b/archer-c7/upgrade-openwrt-25.12.md @@ -0,0 +1,193 @@ +# Archer C7: upgrade OpenWrt 19.07.4 → 25.12.5 + +> **Outcome (2026-08-20):** upgrade done (flashed clean per this doc). +> The USB-tether flapping survived the new kernel → root cause is the +> C7's USB port lacking charging current (phone dropped off the bus +> whenever it drew charge). **WAN is now the phone's Wi-Fi hotspot**: +> 2.4 GHz `radio1` in STA mode → interface `wwan` (`phy1-sta0`, in the +> wan zone), phone on a mains charger. `usbwan` stays configured for a +> future powered-USB-hub wired option. Post-flash gotcha: fw4 compiled +> NAT for `eth0.2` only while `usbwan` was down at reload time — LAN +> had no internet until `service firewall restart` with the WAN +> interface up. + +Home router **TP-Link Archer C7 v5** (`tplink,archer-c7-v5`, target +`ath79/generic`, 16 MB flash / 128 MB RAM) at `192.168.0.2`, running +OpenWrt 19.07.4 (2020, kernel 4.14, EOL since 2022). Target release: +**OpenWrt 25.12.5** (kernel 6.12.94, firewall4/nftables, `apk` package +manager instead of `opkg`). + +Context (2026-08-19): after a reboot the router came up with **dnsmasq, +firewall and odhcpd disabled** (missing `/etc/rc.d/` symlinks — no DHCP, +no NAT). All three were restarted and re-enabled, but the incident plus +the EOL release motivated this upgrade. A fresh flash also rules out +any other overlay damage. + +Current setup worth knowing: + +- **WAN = OPPO phone (SM8350) USB-tethered** on `usb0` via RNDIS + (`kmod-usb-net-rndis`), interface `usbwan`, DHCP from the phone + (10.205.61.0/24). The ethernet WAN port is unused — the ISP box at + `.1` is dead, which is also why the router LAN IP is `.2`. +- SSH: `ssh -i id_rsa_claude root@192.168.0.2` (key at perso repo root; + 19.07's dropbear needs + `-o HostKeyAlgorithms=+ssh-rsa -o PubkeyAcceptedAlgorithms=+ssh-rsa` + from a modern OpenSSH; 25.12 won't) +- DNS: dnsmasq forwards everything to **blocky** at `192.168.0.254` + (`noresolv` + `server=192.168.0.254`), and DHCP option 6 hands blocky + directly to clients +- Wi-Fi: 5 GHz only (`radio0`, ch 36, VHT80, SSID `HOME`, WPA2-PSK); + 2.4 GHz radio disabled +- 7 static DHCP leases (see script below) +- Only user-installed packages on 19.07: `kmod-usb-net-rndis` + deps + +## Prerequisites (done 2026-08-19) + +- [x] Config backup: `sysupgrade -b` → `~/code/perso/backup-archer-c7-20260819.tar.gz` + (laptop; contains Wi-Fi PSK and dropbear host keys — never commit) +- [x] Firmware `openwrt-25.12.5-ath79-generic-tplink_archer-c7-v5-squashfs-sysupgrade.bin` + downloaded, **sha256 verified** (`f15b16b3…`) against the release + `sha256sums`, staged in `~/code/perso/archer-c7-upgrade/` and on + the router at `/tmp/` (`/tmp` is RAM — gone after any reboot, + re-`scp` if needed) +- [x] **Offline driver packages** staged in `~/code/perso/archer-c7-upgrade/`: + `kmod-mii`, `kmod-usb-net`, `kmod-usb-net-cdc-ether`, + `kmod-usb-net-rndis` (all `6.12.94-r1.apk`). Critical: the phone + tether is the only WAN and the RNDIS driver is **not** in the + default image, so after flashing the router has no internet to + install it from — it must be `scp`'d in. + +## Upgrade + +**Do not keep settings across this jump** (19.07 → 25.12 spans the +fw3→fw4 and network-syntax migrations). Flash clean with `-n` and +reconfigure with the script below. + +You need the **laptop on ethernet** to a LAN port: the fresh image +boots at `192.168.1.1` with Wi-Fi disabled. + +```sh +# on the router — image already in /tmp, verify then flash +sha256sum /tmp/openwrt-25.12.5-ath79-generic-tplink_archer-c7-v5-squashfs-sysupgrade.bin +# expect f15b16b36df7fce8a5cf1302320fdb428c3bb6fbc2949a4858afa257f4f364a2 +sysupgrade -n -v /tmp/openwrt-25.12.5-ath79-generic-tplink_archer-c7-v5-squashfs-sysupgrade.bin +``` + +The router reboots (~3 min, don't power-cycle). Then from the laptop +(ethernet, DHCP gives a 192.168.1.x address): + +```sh +# passwordless root at first boot — set a password and install the key +ssh root@192.168.1.1 # no password +passwd +mkdir -p /etc/dropbear && exit +scp ~/code/perso/id_rsa_claude.pub root@192.168.1.1:/etc/dropbear/authorized_keys + +# push the offline RNDIS drivers +scp ~/code/perso/archer-c7-upgrade/kmod-*.apk root@192.168.1.1:/tmp/ +``` + +## Reconfigure + +Run on the router. Single source of truth for the whole config; +`` is in the backup tarball (`etc/config/wireless`). + +```sh +#!/bin/sh +set -eux + +# RNDIS drivers (offline — phone tether is the only WAN) +apk add --allow-untrusted /tmp/kmod-mii-*.apk /tmp/kmod-usb-net-6*.apk \ + /tmp/kmod-usb-net-cdc-ether-*.apk /tmp/kmod-usb-net-rndis-*.apk + +# --- network: keep the generated switch/bridge config, change only what we need +uci set network.lan.ipaddr='192.168.0.2' +uci set network.usbwan=interface +uci set network.usbwan.proto='dhcp' +uci set network.usbwan.device='usb0' +uci commit network + +# --- firewall: usbwan joins the wan zone (zone[1] = 'wan' in the default config) +uci add_list firewall.@zone[1].network='usbwan' +uci commit firewall + +# --- dhcp/dns: forward to blocky, hand blocky to clients, static leases +uci set dhcp.@dnsmasq[0].noresolv='1' +uci add_list dhcp.@dnsmasq[0].server='192.168.0.254' +uci add_list dhcp.lan.dhcp_option='6,192.168.0.254' +add_host() { + uci add dhcp host >/dev/null + uci set dhcp.@host[-1].mac="$1" + uci set dhcp.@host[-1].ip="$2" + uci set dhcp.@host[-1].name="$3" + uci set dhcp.@host[-1].dns='1' +} +add_host 'F4:5C:89:BE:C1:C3' '192.168.0.20' 'LAPTOP719974' +add_host '10:66:6A:87:A1:49' '192.168.0.254' 'blocky' +add_host '10:66:6A:C8:E1:0A' '192.168.0.5' 'jellyfin-server' +add_host '10:66:6A:FF:59:1C' '192.168.0.6' 'jellyfin-client' +add_host '10:66:6A:85:20:B7' '192.168.0.11' 'privoxy' +add_host '10:66:6A:55:9F:10' '192.168.0.7' 'transmission-bt' +add_host '50:9A:4C:20:D9:2B' '192.168.0.21' 'patate' +uci commit dhcp + +# --- wifi: 5 GHz AP as before, upgraded to WPA2/WPA3 mixed; 2.4 GHz stays off +uci set wireless.radio0.channel='36' +uci set wireless.radio0.htmode='VHT80' +uci set wireless.radio0.disabled='0' +uci set wireless.default_radio0.ssid='HOME' +uci set wireless.default_radio0.encryption='sae-mixed' +uci set wireless.default_radio0.key='' +uci set wireless.default_radio0.network='lan' +uci commit wireless + +reload_config +/etc/init.d/network restart +``` + +The LAN IP change kicks you off; reconnect at `192.168.0.2` (plug the +phone back in for WAN). + +## Verify + +```sh +ssh root@192.168.0.2 +ifstatus usbwan | grep -E 'up|address' # up:true, 10.205.61.x +ping -c2 8.8.8.8 && ping -c2 openwrt.org # WAN + DNS via blocky +nft list chain inet fw4 srcnat_wan # masquerade on usbwan +ls /etc/rc.d/ | grep -E 'dnsmasq|firewall|odhcpd' # all enabled +``` + +Then from a LAN client: renew DHCP (expect 192.168.0.100–249, DNS +`.254`), check internet, check the static leases land on their IPs. + +## Known issue: 5 GHz `ath10k-ct` + +The C7 v2/v5 has a long-standing 5 GHz issue with the default +`ath10k-ct` driver: `SWBA overrun on vdev 0` log spam plus client +drops under some loads. **Only if it shows up**, swap to the non-ct +driver (needs working WAN, or pre-download the same way as the RNDIS +apks): + +```sh +apk del kmod-ath10k-ct ath10k-firmware-qca988x-ct +apk add kmod-ath10k ath10k-firmware-qca988x +reboot +``` + +## Recovery + +If the flash bricks: C7 v5 has TFTP recovery — hold **Reset** while +powering on, router pulls `ArcherC7v5_tp_recovery.bin` (the OpenWrt +**factory** image renamed to that) from a TFTP server at +`192.168.0.66`. Factory image is in the same release directory as the +sysupgrade one. + +## Post-upgrade cleanup + +- Delete `/tmp` copies (automatic on reboot) +- Keep `~/code/perso/backup-archer-c7-20260819.tar.gz` until the new + setup has survived a reboot + a few days +- `~/code/perso/archer-c7-upgrade/` can go once the RNDIS drivers are + installed and Wi-Fi is confirmed stable (or keep the apks in case of + a re-flash) diff --git a/ks4/plakar-s3-data.md b/ks4/plakar-s3-data.md index 458dd96..692e6fc 100644 --- a/ks4/plakar-s3-data.md +++ b/ks4/plakar-s3-data.md @@ -112,6 +112,17 @@ Beyond databases, `outline`/`login` may hold non-DB state (uploaded files, docker volumes) — decide per instance whether an fs source is needed in §4. +⚠️ **seafile-fs excluded from the data leg (2026-08-24).** The S3 +seed wedged for 12 h inside a single `lstat()` in seafile's block +store (millions of tiny files on the contended live pool). SIGQUIT +goroutine dump: main goroutine looping in `os.ignoringEINTR` around +`fstatat` — a Go preemption/EINTR **livelock** (100 % CPU, no +progress; dmesg clean, so not a kernel hang). `seafile-fs` is +commented out of `plakar-sources`; seafile's S3 coverage comes from +the incus leg instead, which reads the quiesced replica on the idle +`backup` pool. If a live-pool retry is ever wanted: +`GODEBUG=asyncpreemptoff=1` is the standard mitigation. + Findings from the first dry-run (2026-08-22): - `livetrail`'s `/root/.my.cnf` had a stale `database = spot` default diff --git a/ks4/seafile-gcid-fix.md b/ks4/seafile-gcid-fix.md new file mode 100644 index 0000000..d524e94 --- /dev/null +++ b/ks4/seafile-gcid-fix.md @@ -0,0 +1,109 @@ +# seafile — GCID table loss, 6-month CPU burn (fixed 2026-08-22) + +`seaf-server` in the `seafile` container sat at 40–47 % CPU for months +while appearing to do nothing: negligible client traffic on `eth0` +(344 MB sent over 102 days) but 82 GB and 385 M packets over loopback, +and ~11 **days** of accumulated CPU time. + +## Root cause + +The `seafile-db` MariaDB database was missing the `GCID` and +`LastGCID` tables. Since Seafile 11 every client sync operation checks +a per-repo GC id (`SELECT gc_id FROM GCID WHERE repo_id = ? FOR +UPDATE`); with the table gone the query failed, the client retried, +and the cycle repeated forever — no sync ever completed: + +- 6 of seaf-server's fileserver worker threads spinning in pure + user-space, ~3 new MySQL connections/s (167 loopback `TIME_WAIT` + to :3306 at any moment), +- `seafile.log` grew to **1.8 GB** of the single warning + `Failed to prepare sql SELECT gc_id FROM GCID … Table + 'seafile-db.GCID' doesn't exist`, several times per second. + +First occurrence **2026-03-01 22:17** — under 11.0.9, i.e. *before* +both the 2026-03-16 restart and the 2026-05-11 upgrade to 12.0.14. +`GCID` is an old core table (the 6.3.0 upgrade SQL already ALTERs +it), so it existed and was lost. **What dropped it around 2026-03-01 +was never identified** — if tables vanish again, start there. + +## Diagnosis trail (what worked without container root) + +From the host as `julien` (sudo grant covers read-only `incus` +subcommands only): + +```sh +sudo incus info seafile # PID 2410, 1005 GiB, 2 GiB RAM +ps -eo pid,%cpu,args --sort=-%cpu # seaf-server pid 10217 top hog +# per-thread CPU from /proc (world-readable, no root needed): +for t in /proc/10217/task/*; do awk '{print $1, $14+$15}' $t/stat; done +# utime vs stime split → pure user-space compute, ~0 syscalls +# container netns TCP table via the process: +awk 'NR>1 && $4=="06"' /proc/10217/net/tcp | wc -l # 167 TIME_WAIT → :3306 +``` + +Reading `/opt/seafile/logs/seafile.log` needed a temporary sudoers +extension (`/usr/bin/incus exec *`, `/usr/bin/incus file pull *` in a +drop-in — **removed again after the fix**). The log line then made the +cause obvious. + +## Fix + +The exact DDL ships inside the server binary: + +```sh +strings /opt/seafile/seafile-server-12.0.14/seafile/bin/seaf-server \ + | grep 'CREATE TABLE IF NOT EXISTS.*GCID' +``` + +Applied (root in the container; additive, no restart needed): + +```sql +CREATE TABLE IF NOT EXISTS GCID ( + id BIGINT NOT NULL PRIMARY KEY AUTO_INCREMENT, + repo_id CHAR(36), gc_id CHAR(36), + UNIQUE INDEX(repo_id)) ENGINE=INNODB; +CREATE TABLE IF NOT EXISTS LastGCID ( + id BIGINT NOT NULL PRIMARY KEY AUTO_INCREMENT, + repo_id CHAR(36), client_id VARCHAR(128), gc_id CHAR(36), + UNIQUE INDEX(repo_id, client_id)) ENGINE=INNODB; +``` + +The warning storm stopped the same second, seaf-server dropped to +0 % CPU, host load fell from ~8–12 toward normal. `seafile.log` was +truncated (`truncate -s 0`). + +## Log rotation (added 2026-08-22) + +Seafile ships **no rotation** — that's how one warning reached 1.8 GB +unnoticed. `/etc/logrotate.d/seafile` in the container: + +``` +/opt/seafile/logs/*.log { + weekly + maxsize 100M + rotate 8 + missingok + notifempty + compress + delaycompress + copytruncate +} +``` + +`copytruncate` avoids pid/SIGUSR1 handling for the various daemons +(seaf-server, seahub, seafevents…) at the cost of possibly losing a +few lines at rotation — fine here. `maxsize 100M` caps a future spam +storm at the next daily logrotate run instead of next week. + +## Related cleanup / leftovers + +- Legacy 4 AM cron on ks4 rsyncing seafile's rootfs to old **ks2** + (`164.132.173.57`) removed 2026-08-22 — redundant with the + [incus-copy](incus-copy.md) legs and it burned another ~47 % CPU + per run. +- Still missing from `seafile-db` (harmless, nothing queries them + unless per-folder permissions are used): `FolderUserPerm`, + `FolderGroupPerm`, `FolderPermTimestamp`. Same `strings`/upgrade-SQL + trick provides the DDL if ever needed. +- Host swap (1 GB) was 100 % full during the incident; unrelated but + worth keeping an eye on.