From 66c7bca28f8e9c28d927b6469997997eec600a86 Mon Sep 17 00:00:00 2001 From: Julien Lutran Date: Sun, 30 Aug 2026 23:53:30 +0200 Subject: [PATCH] ks2: ks4 pull leg and its WireGuard tunnel move from nuc to nas MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit nuc-seed.md -> nas-seed.md. The leg was designed around nuc's USB pool, which is exactly the device it must not depend on. Target pool ks4backup now lives on tank; nas becomes WG peer 10.8.0.22 and nuc's tunnel retires once seeded — nuc no longer needs one at all, since transmission-bt (the only other user) moved to nas with its own in-container tunnel. ks4 needs no change: traffic arrives masqueraded as the wireguard container whichever peer sent it. Co-Authored-By: Claude Opus 5 (1M context) --- ks2/decommission.md | 4 +-- ks2/nas-seed.md | 83 +++++++++++++++++++++++++++++++++++---------- ks2/plan.md | 15 ++++---- 3 files changed, 75 insertions(+), 27 deletions(-) diff --git a/ks2/decommission.md b/ks2/decommission.md index d2018f7..22f00f3 100644 --- a/ks2/decommission.md +++ b/ks2/decommission.md @@ -8,8 +8,8 @@ few days of margin. - [ ] restic S3 leg running nightly for ≥ a week, `done (rc=0)`, restore test passed ([restic-backup.md](../ks4/restic-backup.md) §7) -- [ ] nuc pull leg seeded and one instance test-restored - ([nuc-seed.md](nuc-seed.md)) +- [ ] **nas** pull leg seeded and one instance test-restored + ([nas-seed.md](nas-seed.md)) - [ ] local leg (sdb5) cron green in `/var/log/incus-copy.log` ## 1. Cut the last flows to ks2 (root on ks4) diff --git a/ks2/nas-seed.md b/ks2/nas-seed.md index b819a5d..7fe4f08 100644 --- a/ks2/nas-seed.md +++ b/ks2/nas-seed.md @@ -1,19 +1,57 @@ -# nuc pull leg — seed after FTTH +# ks4 pull leg — seed after FTTH -Status: **prepared, waiting on the FTTH link.** Everything is already -configured on nuc (see the main [README](../README.md)): incus remote -`ks4` over the WireGuard tunnel (`wg-ks4`, 10.8.0.20 → 10.8.0.1), -target pool `ks4backup` on the USB ZFS pool (`usb4t/backup/ks4`) — -only the seed itself waited on bandwidth. +Status: **prepared, waiting on the FTTH link.** -## Seed (root on nuc) +⚠️ **Changed 2026-08-30: this leg lands on `nas`, not on nuc.** It was +originally designed for nuc's USB pool `usb4t`, but that pool proved to +be the least reliable device in the setup +([nuc/usb4t-dropouts.md](../nuc/usb4t-dropouts.md)) — which is exactly +what an off-site copy of ks4 must not be. The 4 TB disk moved to direct +SATA on the new host `nas` (`192.168.0.4`, +[nas/nas-install.md](../nas/nas-install.md)), and the target pool +`ks4backup` moved with it. + +Consequences versus the original plan: + +- Target pool `ks4backup` is now backed by `tank/backup/ks4` on nas. +- The **WireGuard tunnel moves too**: nas becomes peer `10.8.0.22`; + nuc's `wg-ks4` (`10.8.0.20`) is retired once this works. nuc no longer + needs a tunnel at all — `transmission-bt`, its only other user, now + runs on nas and carries its own in-container tunnel (`10.8.0.21`, + unchanged, ks4 needs no edit for it). +- ks4's ufw rule is unchanged: traffic arrives masqueraded as the + `wireguard` container (`192.168.1.18`) whichever peer sent it. + +## Prerequisites + +- [ ] `tank` running on SATA for ≥ 7 days with zero pool suspensions — + the gate that replaces "fix the USB enclosure" +- [ ] FTTH up (the first pass moves ~1.75 TiB) + +## Setup (root on nas) ```sh -# sanity: remote reachable through the tunnel -incus list ks4: | head +# 1. peer nas on ks4's wireguard container +# (run on ks4) — from /etc/wireguard/wg-ks4.key on nas +incus exec wireguard -- wg set wg0 peer allowed-ips 10.8.0.22/32 +incus exec wireguard -- wg-quick save wg0 -# full pull of every ks4 instance into pool ks4backup (screen/tmux — -# first pass moves ~1.7 T through the WG tunnel) +# 2. tunnel on nas: /etc/wireguard/wg-ks4.conf, modelled on nuc's +# Address = 10.8.0.22/32, peer pubkey TVs6d7…, +# Endpoint = 193.70.35.17:51845, +# AllowedIPs = 10.8.0.0/24, 192.168.1.1/32, keepalive 25 +systemctl enable --now wg-quick@wg-ks4 + +# 3. incus remote over the tunnel +incus remote add ks4 https://192.168.1.1:8443 --accept-certificate --token '…' +incus list ks4: | head # sanity: remote reachable +``` + +## Seed + +```sh +# full pull of every ks4 instance into pool ks4backup (tmux — first pass +# moves ~1.75 TiB through the WG tunnel) /root/scripts/incus-copy.sh -r ks4 -s ks4backup 2>&1 | tee -a /var/log/incus-copy-ks4.log ``` @@ -23,13 +61,13 @@ Notes: ZFS-incremental **as long as they run at least every `snapshots.expiry` (7 d on ks4)** — same caveat as [ks4's local leg](../ks4/local-backup-cron.md). -- Replicas arrive stopped with `boot.autostart=false` (script does +- Replicas arrive stopped with `boot.autostart=false` (the script does this) — they must never come up on the LAN with ks4's proxy devices. ## Cron (after the seed) -Add to nuc's root crontab, offset from the 03:30 local nucbackup copy -and ks4's own 01:00/05:00 jobs: +Add to nas's root crontab, offset from the 03:30 nuc→nas push, the +04:00 nas→nuc push and ks4's own 01:00/05:00 jobs: ```cron 0 5 * * * /root/scripts/incus-copy.sh -r ks4 -s ks4backup >> /var/log/incus-copy-ks4.log 2>&1 @@ -38,12 +76,21 @@ and ks4's own 01:00/05:00 jobs: ## Verification (release gate for ks2) ```sh -incus list --project backup 2>/dev/null || incus list | grep -c . # all ks4 instances present -# test-restore one instance: copy a replica to the default pool, +incus list --project backup -c ns -f csv # all ks4 instances present +# test-restore one instance: copy a replica to the local pool, # start it isolated, check the service answers, then delete it -incus copy solar solar-restoretest -s default +incus copy solar solar-restoretest -s incus incus start solar-restoretest && incus exec solar-restoretest -- systemctl is-system-running incus delete -f solar-restoretest ``` -Once verified, tick the nuc gate in the [ks2 plan](plan.md). +Once verified, tick the nas gate in the [ks2 plan](plan.md) and retire +nuc's tunnel: + +```sh +# on nuc +systemctl disable --now wg-quick@wg-ks4 && rm /etc/wireguard/wg-ks4.conf +# on ks4 +incus exec wireguard -- wg set wg0 peer remove +incus exec wireguard -- wg-quick save wg0 +``` diff --git a/ks2/plan.md b/ks2/plan.md index 37b5e1c..1e67811 100644 --- a/ks2/plan.md +++ b/ks2/plan.md @@ -17,10 +17,11 @@ What remains on the box is **cold history**: instance replicas on pool `/backup/ns3061243` on pool `backup` (last refreshed 2026-08-28), snapshotted daily there (`zfs-auto-snapshot.sh`, 2-month expiry). -⚠️ While the nuc leg waits for FTTH, instances have no *fresh* +⚠️ While the nas leg waits for FTTH, instances have no *fresh* off-site copy — the ks2 push is to be re-enabled as soon as the initial restic sync finishes (decided 2026-08-28), and retired again -when nuc takes over. +when nas takes over. (The leg moved from nuc to the new host `nas` +on 2026-08-30 — see [nas-seed.md](nas-seed.md).) ## Inventory findings (2026-08-22) @@ -51,8 +52,8 @@ when nuc takes over. | local, 2nd disk | `incus-copy.sh -p backup -s backup` → sdb5 pool | **live** (01:00) | | off-site, S3 (data+DB) | **restic** → bucket `restic-data` ([restic-backup.md](../ks4/restic-backup.md)) | **live** (05:00) | | off-site, S3 (instances) | restic over `incus file mount` of the `backup`-project replicas | **shelved** 2026-08-28 (replication covers instances) | -| off-site, nuc | nuc pulls `ks4:*` → pool `ks4backup` over WG ([nuc-seed.md](nuc-seed.md)) | waiting FTTH (< Sep 30) | -| off-site, ks2 (interim) | `incus-copy.sh -d ks2 -m push` at 02:00 until the nuc leg seeds | to re-enable once the restic seed finishes | +| off-site, nas | **nas** pulls `ks4:*` → pool `ks4backup` over WG ([nas-seed.md](nas-seed.md)) | waiting FTTH (< Sep 30); moved off nuc 2026-08-30 | +| off-site, ks2 (interim) | `incus-copy.sh -d ks2 -m push` at 02:00 until the nas leg seeds | to re-enable once the restic seed finishes | ## Actions (backup work documented in [`../ks4/`](../ks4/), ks2-only tasks here) @@ -65,9 +66,9 @@ when nuc takes over. ([restic-backup.md](../ks4/restic-backup.md), live 2026-08-28; the predecessor's doc is kept as reference) 4. ~~instance leg to S3~~ — **shelved 2026-08-28**: instances are - protected by replication (sdb + nuc/ks2), their data and configs by + protected by replication (sdb + nas/ks2), their data and configs by `restic-data` -5. [nuc-seed.md](nuc-seed.md) — **prepared**; after FTTH: seed nuc +5. [nas-seed.md](nas-seed.md) — **prepared**; after FTTH: seed the nas pull leg, verify all instances, test-restore one 6. [decommission.md](decommission.md) — **prepared**; cut flows, final diff of `/backup/ns3061243`, wipe pools, terminate at OVH @@ -77,7 +78,7 @@ when nuc takes over. - [x] biwiki + spot consciously abandoned (2026-08-22) - [x] local leg cron running since 2026-08-22, **18/18 instances** replicated (verified 2026-08-28) -- [ ] nuc leg fully seeded **and** one instance test-restored +- [ ] **nas** leg fully seeded **and** one instance test-restored - [x] restic S3 backups live (05:00) **and** restore drill passed 2026-08-28: tree restored byte-identical to live, dump restored and loaded into a scratch MariaDB (12/12 tables)