julien/doc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

restic-backup: two-bucket layout (restic-data + restic-incus), two drivers

Browse Source 
Giants merged into restic-data (single nightly invocation, one index
load, --group-by host retention); instances leg gets per-instance
mountpoints — restic picks parents by host+path, a shared mountpoint
would cause nightly full re-reads.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Julien Lutran
2026-08-25 12:50:07 +02:00
co-authored by Claude Fable 5
parent f7843f9e37
commit de72120105
1 changed files with 54 additions and 44 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ks4/restic-backup.md
+54 -44
View File
@@ -17,20 +17,21 @@ needed: instances are reached via `incus file mount` (FUSE over the
same per-instance sftp API our plakar integration used) — zero same per-instance sftp API our plakar integration used) — zero
patches to maintain (see §6). patches to maintain (see §6).
## 1. Repositories — one bucket per repo ## 1. Repositories — two buckets, two drivers (decided 2026-08-25)
| bucket = repo | contents | est. size | blobs (index RAM driver) | | bucket = repo | contents | driver script |
|---|---|---|---| |---|---|---|
| `restic-data` | `/backup/plakar-dumps` dump tree + the 8 small fs sources | ~50 G | small | | `restic-data` | SQL dumps + **all** fs sources (8 small + nextcloud + seafile) | `restic-backup.sh` |
| `restic-nextcloud` | `/nextcloud` data (live rootfs path) | 737 G | ~1 M | | `restic-incus` | the `backup`-project replicas, per-file via mount | `restic-incus-backup.sh` |
| `restic-seafile` | `/opt/seafile` (live rootfs path) | 933 G | ~2–4 M |
| `restic-instances` | the `backup`-project replicas, per-file via mount | ~16 small instances | medium |
Separate repos because restic loads the whole repo index into RAM per Mirrors the plakar-era split (data leg / incus leg). Accepted
operation and prune/check are per-repo; separate **buckets** (not trade-off: `restic-data` carries the giants' blobs (~3–5 M once
prefixes) for per-bucket OVH usage metrics, independent seafile is in), so **every operation on it loads a ~1.5–2 GiB
versioning/object-lock decisions later, and bucket-native credential index** — mitigated by running the whole nightly as a *single*
scoping. One credential pair across all four to start. `restic backup` invocation (one index load, one snapshot covering
all paths), `GOGC=20` in reserve, rustic as the escape hatch.
Buckets (not prefixes) for per-bucket OVH metrics and independent
versioning/lock decisions later.
Passphrase: `/root/.restic-passphrase` (mode 600), one for all repos, Passphrase: `/root/.restic-passphrase` (mode 600), one for all repos,
**copy to the password manager immediately**. S3 creds + env in **copy to the password manager immediately**. S3 creds + env in
@@ -118,7 +119,7 @@ export RESTIC_CACHE_DIR=/backup/restic-cache
EOF EOF
chmod 600 /root/.restic-env && . /root/.restic-env && mkdir -p /backup/restic-cache chmod 600 /root/.restic-env && . /root/.restic-env && mkdir -p /backup/restic-cache
for r in restic-data restic-nextcloud restic-seafile restic-instances; do for r in restic-data restic-incus; do
restic -r s3:s3.sbg.io.cloud.ovh.net/$r init restic -r s3:s3.sbg.io.cloud.ovh.net/$r init
done done
``` ```
@@ -134,18 +135,15 @@ done
``` ```
⚠️ the floor is defended even under pressure — real headroom is ⚠️ the floor is defended even under pressure — real headroom is
~31 G − services − 8 G; revisit before adding a fat service. ~31 G − services − 8 G; revisit before adding a fat service.
2. `restic-data` (~50 G — an hour). 2. `restic-data`, staged (all into the same repo; separate
3. `restic-nextcloud` (proven ~22 MiB/s from the live pool → ~8 h), invocations so each stage is independently restartable —
screen: `restic-backup.sh -s` runs the same thing):
```sh - dumps + 8 small sources (~50 G — an hour)
restic -r s3:s3.sbg.io.cloud.ovh.net/restic-nextcloud \ - nextcloud path with the exclude file (proven ~22 MiB/s from the
backup --pack-size 64 --read-concurrency 8 -o s3.connections=8 \ live pool → ~8 h, screen)
--exclude-file /root/scripts/restic-nextcloud-exclude \ - seafile — the one-time debt: 1–2 days, restartable at any point
/var/lib/incus/storage-pools/data/containers/nextcloud/rootfs/nextcloud (committed packs dedup on retry)
``` 3. `restic-incus` (16 small replicas via §6 — a few hours).
4. `restic-instances` (16 small replicas via §6 — a few hours).
5. `restic-seafile` — the one-time debt: 1–2 days, restartable at any
point (committed packs dedup on retry).
## 5. Nightly driver — `scripts/restic-backup.sh` ## 5. Nightly driver — `scripts/restic-backup.sh`
@@ -153,14 +151,21 @@ Same shape as the plakar drivers (flock, env, loud logging). Phases:
1. **dumps** — lifted verbatim from `plakar-backup.sh`: incus DB 1. **dumps** — lifted verbatim from `plakar-backup.sh`: incus DB
dumps + auto-discovered MariaDB/PostgreSQL (native and docker) dumps + auto-discovered MariaDB/PostgreSQL (native and docker)
dumps into `/backup/plakar-dumps` (path kept), then dumps into the dump dir (during the parallel week restic dumps
`restic -r …/restic-data backup` of the dump tree **and** the 8 into its own `/backup/dumps`; carry the archived `geo.sql` over
small source paths (one snapshot, paths from a list file). once with `cp -a`).
2. **giants** — nextcloud (with excludes) and seafile into their 2. **one backup invocation** into `restic-data`: dump dir + all fs
repos. paths from `/root/scripts/restic-paths`
3. **instances** — §6 loop into `restic-instances`. (`--files-from-verbatim`), nextcloud excludes applied globally
4. **retention** — per repo: `restic forget --keep-daily 14 (patterns only match nextcloud paths). One index load, one
--keep-weekly 8 --keep-monthly 6` (instant without `--prune`). snapshot per night.
3. **retention** — `restic forget --group-by host --keep-daily 14
--keep-weekly 8 --keep-monthly 6` (instant without `--prune`;
`--group-by host` because seed-era snapshots have different path
sets than nightly ones and must age in one group).
The instances leg is `restic-incus-backup.sh` (§6), chained after the
01:00 incus-copy once validated.
```cron ```cron
0 5 * * * /root/scripts/restic-backup.sh >> /var/log/restic-backup.log 2>&1 0 5 * * * /root/scripts/restic-backup.sh >> /var/log/restic-backup.log 2>&1
@@ -190,22 +195,27 @@ exposes any instance (including **stopped replicas**) as a FUSE mount
over the same per-instance sftp API the plakar integration used: over the same per-instance sftp API the plakar integration used:
```sh ```sh
mkdir -p /run/restic-incus
for inst in $(incus list --project backup -c n -f csv); do for inst in $(incus list --project backup -c n -f csv); do
case ",$EXCLUDE_INSTANCES," in *",$inst,"*) continue;; esac # nextcloud,seafile case ",$EXCLUDE_INSTANCES," in *",$inst,"*) continue;; esac # nextcloud,seafile
incus config show "$inst" --project backup --expanded > "/backup/plakar-dumps/incus/$inst.yaml" mnt=/run/restic-incus/$inst # per-instance mountpoint: restic
incus file mount "$inst/" /run/restic-incus --project backup & # foreground process; background + kill mkdir -p "$mnt" # picks parents by host+path, so
incus config show "$inst" --project backup --expanded > "$mnt.yaml"
incus file mount "$inst/" "$mnt" --project backup & # foreground proc
MPID=$!; sleep 2 MPID=$!; sleep 2
restic -r s3:…/restic-instances backup --pack-size 64 --tag "$inst" /run/restic-incus restic -r s3:…/restic-incus backup --pack-size 64 --tag "$inst" "$mnt" "$mnt.yaml"
kill $MPID; wait $MPID 2>/dev/null; umount /run/restic-incus 2>/dev/null kill $MPID; wait $MPID 2>/dev/null; umount "$mnt" 2>/dev/null
done done
``` ```
(Exact mount lifecycle to be hardened in the script — `incus file ⚠️ **per-instance mountpoints are load-bearing**: a shared mountpoint
mount` runs in the foreground and needs `sshfs` installed.) All would make every snapshot's parent the previous *other* instance's
instances by default, opt-out via a variable — the no-manifest-drift snapshot (restic matches parents by host+path) → nightly full
rule survives the tool change. Chained after the 01:00 incus-copy in re-reads. Distinct stable paths give correct parents and stat-skip.
its cron entry once validated, same freshness reasoning as before. Mount lifecycle hardened in `restic-incus-backup.sh` (`incus file
mount` runs foreground; needs `sshfs`). All instances by default,
opt-out via a variable — the no-manifest-drift rule survives the tool
change. Chained after the 01:00 incus-copy once validated, same
freshness reasoning as before.
Bonus over plakar: one shared repo dedups the Ubuntu base across all Bonus over plakar: one shared repo dedups the Ubuntu base across all
16 replicas. Restore granularity: per-file; instance definitions ride 16 replicas. Restore granularity: per-file; instance definitions ride
@@ -216,7 +226,7 @@ the dump tree (`incus/<inst>.yaml` + `incus-global-db.sql`).
Per repo, one restore + diff (never to `/tmp` — 16 GiB tmpfs): Per repo, one restore + diff (never to `/tmp` — 16 GiB tmpfs):
```sh ```sh
restic -r s3:…/restic-nextcloud restore latest --target /backup/restore-test --include '<subtree>' restic -r s3:…/restic-data restore latest --target /backup/restore-test --include '<subtree>'
diff -r … && rm -rf /backup/restore-test diff -r … && rm -rf /backup/restore-test
``` ```