Julien LutranandClaude Fable 5
ef88ef9342
backup-strategy (the entry point) still described nuc pulling the ks4 replicas: the leg, its WireGuard peer and the target pool now live on nas with the 4 TB on direct SATA. Also: ks4 README flow chart redrawn for the new topology, incus-copy leg 2 retargeted, usb4t-dropouts marked RESOLVED (kept for the diagnosis method and the alerting gap), and ks2/plan records that the interim push was deliberately not re-enabled. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
90 lines
4.7 KiB
Markdown
90 lines
4.7 KiB
Markdown
# ks2 decommission — summary plan
|
|
|
|
Goal: release `ks2.lutran.fr` (`164.132.173.57`, SSH :2233, host
|
|
`ns3247221`) once ks4 has a real 3-2-1 backup without it.
|
|
**Deadline: rental ends Sep 30, 2026.**
|
|
|
|
## What ks2 does today (updated 2026-08-28)
|
|
|
|
**Nothing is written to ks2 any more.** Both feeds are retired: the
|
|
`01:00` replica push (replaced by the local leg on ks4's sdb5 pool,
|
|
[local-backup-cron.md](../ks4/local-backup-cron.md)) and the `04:00`
|
|
`incus-backup.sh` rsync (replaced by restic → S3,
|
|
[restic-backup.md](../ks4/restic-backup.md)).
|
|
|
|
What remains on the box is **cold history**: instance replicas on pool
|
|
`data` (last refreshed 2026-08-09) and the rsync tree
|
|
`/backup/ns3061243` on pool `backup` (last refreshed 2026-08-28),
|
|
snapshotted daily there (`zfs-auto-snapshot.sh`, 2-month expiry).
|
|
|
|
⚠️ While the nas leg waits for FTTH, instances have no *fresh*
|
|
off-site copy. The ks2 push was **deliberately not re-enabled**
|
|
(2026-08-30): ks2's replicas are three weeks old, its `data` pool is
|
|
94 % full, no common snapshot survives ks4's 7-day expiry, and the box
|
|
is wiped within the month — so a full ~1.5 T re-send buys four weeks
|
|
of freshness on hardware already scheduled for destruction. Off-site
|
|
protection meanwhile rests on `restic-data` (data, databases and the
|
|
incus configuration — enough to rebuild). (The leg moved from nuc to
|
|
the new host `nas` on 2026-08-30 — see [nas-seed.md](nas-seed.md).)
|
|
|
|
## Inventory findings (2026-08-22)
|
|
|
|
- `biwiki` exists only on ks2, `spot` only on ks2 + ks4's `backup`
|
|
project — **decision: both abandoned**, safe to delete from backups.
|
|
- `incus-backup.db` is stale (lists `spot`, misses `livetrail`,
|
|
`outline`, `login`, …) — forgotten-manifest drift is exactly what
|
|
the replacement had to eliminate — restic's drivers back up *all*
|
|
instances and auto-discover *all* databases, opt-out instead of
|
|
opt-in.
|
|
- `livetrail` (running on ks4) is **absent from ks4's local `backup`
|
|
project** — the local leg was run manually once and never again;
|
|
automating its cron fixes this.
|
|
- Both ks4 backup crons were **commented out — no ks4 backup ran at
|
|
all** (hence the stale ks2 rsync data: dirs `Jul 23`, incus dumps
|
|
`Aug 9`). Fixed 2026-08-22 — see
|
|
[local-backup-cron.md](../ks4/local-backup-cron.md).
|
|
- `/backup/ns3061243` also holds pre-2023 dirs (`catc`, `mythoughts`,
|
|
`qcm`) — those instances still exist stopped on ks4, so nothing
|
|
unique expected there; spot-check before wiping.
|
|
- Losing ks2 also loses its 2 months of rsync-snapshot history —
|
|
acceptable: restic keeps 14 daily / 8 weekly / 6 monthly.
|
|
|
|
## Target architecture (3-2-1 for ks4)
|
|
|
|
| Leg | Mechanism | Status |
|
|
|---|---|---|
|
|
| local, 2nd disk | `incus-copy.sh -p backup -s backup` → sdb5 pool | **live** (01:00) |
|
|
| off-site, S3 (data+DB) | **restic** → bucket `restic-data` ([restic-backup.md](../ks4/restic-backup.md)) | **live** (05:00) |
|
|
| off-site, S3 (instances) | restic over `incus file mount` of the `backup`-project replicas | **shelved** 2026-08-28 (replication covers instances) |
|
|
| off-site, nas | **nas** pulls `ks4:*` → pool `ks4backup` over WG ([nas-seed.md](nas-seed.md)) | waiting FTTH (< Sep 30); moved off nuc 2026-08-30 |
|
|
| off-site, ks2 (interim) | `incus-copy.sh -d ks2 -m push` | **not re-enabled** 2026-08-30 — see above |
|
|
|
|
## Actions (backup work documented in [`../ks4/`](../ks4/), ks2-only tasks here)
|
|
|
|
1. ~~salvage biwiki/spot~~ — decided 2026-08-22: both abandoned,
|
|
**deleted** from ks2 and ks4's `backup` project
|
|
2. ~~[local-backup-cron.md](../ks4/local-backup-cron.md)~~ — **done
|
|
2026-08-22**: leg 1 cron re-enabled, ks2 rsync kept as stopgap;
|
|
`livetrail` verified present in the `backup` project
|
|
3. ~~first S3 leg~~ → **restic** is the S3 tool
|
|
([restic-backup.md](../ks4/restic-backup.md), live 2026-08-28; the
|
|
predecessor's doc is kept as reference)
|
|
4. ~~instance leg to S3~~ — **shelved 2026-08-28**: instances are
|
|
protected by replication (sdb + nas/ks2), their data and configs by
|
|
`restic-data`
|
|
5. [nas-seed.md](nas-seed.md) — **prepared**; after FTTH: seed the nas
|
|
pull leg, verify all instances, test-restore one
|
|
6. [decommission.md](decommission.md) — **prepared**; cut flows,
|
|
final diff of `/backup/ns3061243`, wipe pools, terminate at OVH
|
|
|
|
## Release gates — ks2 can be dropped only when
|
|
|
|
- [x] biwiki + spot consciously abandoned (2026-08-22)
|
|
- [x] local leg cron running since 2026-08-22, **18/18 instances**
|
|
replicated (verified 2026-08-28)
|
|
- [ ] **nas** leg fully seeded **and** one instance test-restored
|
|
- [x] restic S3 backups live (05:00) **and** restore drill passed
|
|
2026-08-28: tree restored byte-identical to live, dump restored
|
|
and loaded into a scratch MariaDB (12/12 tables)
|
|
- [ ] ks4 crons pointing at ks2 disabled
|