doc: plakar out of the current architecture, kept as reference
backup-strategy.md now leads with the three target legs (local incus push to sdb, nuc pull over WireGuard, restic to S3 for DBs and selected trees) and carries the plakar/ks2 story in a closing History section. restic-backup.md, install.md, local-backup-cron.md and the ks2 docs describe the predecessor only as 'the first S3 implementation', with one operational note that plakar is still installed for issue #2338 reproduction. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
b5aef914f5
commit
d7ec780747
+30
-36
@@ -1,25 +1,18 @@
|
||||
# restic backups on ks4 — the S3 leg
|
||||
|
||||
Status: **live since 2026-08-28** — restic replaced plakar as the S3
|
||||
backup tool (plakar crons off, its kloset kept for
|
||||
[#2338](https://github.com/PlakarKorp/plakar/issues/2338); see §8).
|
||||
Architecture: **two tools** — `incus copy` for replication (local sdb,
|
||||
nuc after FTTH), **restic** for everything S3. Data leg (`restic-data`)
|
||||
runs nightly at 05:00. The instance leg (`restic-incus`, §6) is
|
||||
**shelved** (2026-08-28): instances are covered by replication, so
|
||||
duplicating their filesystems into S3 buys little. The script and the
|
||||
(empty) repo stay in place should that change.
|
||||
Status: **live since 2026-08-28.** This is the S3 backup leg of the
|
||||
[strategy](../backup-strategy.md): database dumps + selected
|
||||
filesystem trees into the `restic-data` bucket, nightly at 05:00.
|
||||
Instances themselves are handled by replication (sdb, nuc after FTTH),
|
||||
not by this leg — an instance-filesystem leg (`restic-incus`, §6) was
|
||||
written and **shelved**; its script and empty repo stay in place
|
||||
should that change.
|
||||
|
||||
Why: plakar's incremental cost scales with tree size, not churn
|
||||
(~800 MiB parent metadata re-read from S3 + hours of CPU per night on
|
||||
a 500k-file tree — measured, reported as
|
||||
[PlakarKorp/plakar#2338](https://github.com/PlakarKorp/plakar/issues/2338)).
|
||||
restic keeps its comparison state in a local cache: a night costs a
|
||||
local stat-walk plus the churn — the rsync cost model. Neither restic
|
||||
nor rustic has plakar's source-plugin architecture, but none is
|
||||
needed: instances are reached via `incus file mount` (FUSE over the
|
||||
same per-instance sftp API our plakar integration used) — zero
|
||||
patches to maintain (see §6).
|
||||
Why restic: its comparison state lives in a **local cache**, so a
|
||||
night costs a local stat-walk plus the churn — the rsync cost model,
|
||||
with dedup, compression and encryption on top. (The predecessor and
|
||||
the measurements that motivated the switch are in
|
||||
[plakar-s3-data.md](plakar-s3-data.md) — reference only.)
|
||||
|
||||
## 1. Repositories — two buckets, two drivers (decided 2026-08-25)
|
||||
|
||||
@@ -28,7 +21,7 @@ patches to maintain (see §6).
|
||||
| `restic-data` | SQL dumps + **all** fs sources (8 small + nextcloud + seafile) | `restic-backup.sh` |
|
||||
| `restic-incus` | *(shelved — empty repo, see §6)* the `backup`-project replicas, per-file via mount | `restic-incus-backup.sh` |
|
||||
|
||||
Mirrors the plakar-era split (data leg / incus leg). Accepted
|
||||
Two repos so each can be pruned and checked independently. Accepted
|
||||
trade-off: `restic-data` carries the giants' blobs (~3–5 M once
|
||||
seafile is in), so **every operation on it loads a ~1.5–2 GiB
|
||||
index** — mitigated by running the whole nightly as a *single*
|
||||
@@ -168,9 +161,9 @@ done
|
||||
|
||||
## 5. Nightly driver — `scripts/restic-backup.sh`
|
||||
|
||||
Same shape as the plakar drivers (flock, env, loud logging). Phases:
|
||||
Shape: flock, env file, loud logging. Phases:
|
||||
|
||||
1. **dumps** — lifted verbatim from `plakar-backup.sh`: incus DB
|
||||
1. **dumps** — incus DB
|
||||
dumps + auto-discovered MariaDB/PostgreSQL (native and docker)
|
||||
dumps into the dump dir (during the parallel week restic dumps
|
||||
into its own `/backup/dumps`; carry the archived `geo.sql` over
|
||||
@@ -227,7 +220,7 @@ byte once a year, ~25 GiB read per week.
|
||||
Neither restic nor rustic accepts source plugins, and forking the
|
||||
backup tool is the wrong place to carry a patch. Instead, incus
|
||||
exposes any instance (including **stopped replicas**) as a FUSE mount
|
||||
over the same per-instance sftp API the plakar integration used:
|
||||
over the per-instance sftp API:
|
||||
|
||||
```sh
|
||||
for inst in $(incus list --project backup -c n -f csv); do
|
||||
@@ -252,8 +245,7 @@ opt-out via a variable — the no-manifest-drift rule survives the tool
|
||||
change. Chained after the 01:00 incus-copy once validated, same
|
||||
freshness reasoning as before.
|
||||
|
||||
Bonus over plakar: one shared repo dedups the Ubuntu base across all
|
||||
16 replicas. Restore granularity: per-file; instance definitions ride
|
||||
One shared repo would dedup the Ubuntu base across all replicas. Restore granularity: per-file; instance definitions ride
|
||||
the dump tree (`incus/<inst>.yaml` + `incus-global-db.sql`).
|
||||
|
||||
## 7. Restore test (gate)
|
||||
@@ -267,24 +259,26 @@ diff -r … && rm -rf /backup/restore-test
|
||||
|
||||
## 8. Cutover (done) + what is left
|
||||
|
||||
- **Cutover 2026-08-28**: plakar's 04:30 and Sunday crons are off;
|
||||
restic's 05:00 data leg is the S3 backup. The formal parallel-week
|
||||
Legacy on the box: plakar is still installed and its `plakar-data`
|
||||
bucket still exists, kept only to reproduce
|
||||
[#2338](https://github.com/PlakarKorp/plakar/issues/2338) if upstream
|
||||
asks; no cron references it. The empty `plakar-incus` bucket can be
|
||||
deleted.
|
||||
|
||||
- **Cutover 2026-08-28**: the previous S3 leg's crons are off and
|
||||
restic's 05:00 run is the S3 backup. The formal parallel-week
|
||||
comparison was cut short — the evidence had already settled it
|
||||
(plakar: ~800 MiB metadata re-read and 3–4 h per night on the
|
||||
(predecessor: ~800 MiB metadata re-read and 3–4 h per night on the
|
||||
nextcloud tree, churn-independent; restic: seed at 60–115 MiB/s,
|
||||
local-cache incrementals). plakar's measurements stay in
|
||||
local-cache incrementals). Details in
|
||||
[plakar-s3-data.md](plakar-s3-data.md).
|
||||
- **Left to do**: restore test on the complete `restic-data`; re-enable
|
||||
the interim ks2 push until FTTH; then the ks2 decommission gates
|
||||
([../ks2/plan.md](../ks2/plan.md)). The instance leg (§6) is
|
||||
shelved, not pending.
|
||||
- **plakar retirement, one exception**: keep plakar installed and
|
||||
keep the `plakar-data` kloset until
|
||||
[#2338](https://github.com/PlakarKorp/plakar/issues/2338) concludes
|
||||
— the issue offers debug runs against that repository. Delete the
|
||||
`plakar-incus` bucket immediately; `plakar-data` a few weeks later.
|
||||
The `integration-incus` repo retires from production; optionally
|
||||
still publish to PlakarKorp/hub as a community contribution.
|
||||
- **Old tool**: see the legacy note above; the `integration-incus`
|
||||
repo retires from production (optionally still published to
|
||||
PlakarKorp/hub as a community contribution).
|
||||
- **Interim ks2 push** (until the nuc leg seeds after FTTH, expected
|
||||
before end of September): re-enable the replica push so instances
|
||||
keep an off-site copy meanwhile:
|
||||
|
||||
Reference in New Issue
Block a user