Document the ks2 decommission plan and the new ks4 backup legs
- ks2/: decommission plan with release gates (rental ends Sep 30), nuc-seed and final-decommission runbooks - ks4/local-backup-cron.md: leg-1 cron re-enabled, ks2 rsync stopgap, full-resend caveat when refreshes lapse past snapshots.expiry - ks4/plakar-s3-data.md: plakar direct-to-S3 leg (auto-discovered DB dumps incl. grants/globals, fs sources, exclude list, restore test) - ks4/plakar-incus-integration.md: design + scaffold status of the plakar incus importer - README.md: index ks2/, update the ks4 durability bullet Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Julien Lutran
co-authored by
Claude Fable 5
Claude Fable 5
parent
4b30b43c7f
commit
79ad1b7179
@@ -0,0 +1,119 @@
|
||||
# plakar incus integration — design notes
|
||||
|
||||
Status: **importer scaffolded and compiling 2026-08-23** in
|
||||
`perso/plakar/integration-incus` (walker modeled on the official sftp
|
||||
integration; enumerate project → per-instance sftp walk → per-file
|
||||
records + synthetic `.incus.yaml`; allow/deny lists; VM sftp failures
|
||||
reported as record errors, not fatal). See its README for
|
||||
build/install/TODO. Untested against a live incus yet.
|
||||
|
||||
Goal: plakar source connector for Incus so instance backups land in a
|
||||
plakar kloset on S3 (leg 4 of the ks4 3-2-1 plan — see
|
||||
[plan.md](../ks2/plan.md)).
|
||||
|
||||
## Decision 1 — per-file importer, not tarball streaming
|
||||
|
||||
The proxmox integration streams one full vzdump archive per VM
|
||||
(`FLAG_STREAM`). For ks4 that model reads ~1.7 TB nightly (seafile
|
||||
933 G, nextcloud 725 G): the incus backup/export API produces **full
|
||||
tarballs only** — `shared/api/instance_backup.go` has just
|
||||
`instance_only` and `optimized_storage`, no incremental option.
|
||||
|
||||
Instead the importer emits **one record per file** with real stat info
|
||||
and a lazy reader. kloset's engine then does the incremental work for
|
||||
us: `snapshot/backup.go` (`checkVFSCache`) reuses the cached object
|
||||
when a path's stat matches the previous run and **never opens the
|
||||
file**. Nightly cost after the initial seed = stat-walk + changed
|
||||
files only; S3 receives only new chunks. Bonus: per-file browse and
|
||||
restore of any snapshot.
|
||||
|
||||
File access is native incus REST: `GET /1.0/instances/{name}/sftp`
|
||||
(a real SFTP session, also available per storage volume). Works over
|
||||
the local unix socket or remote :8443, on stopped containers (the
|
||||
daemon mounts the volume on demand), no ZFS assumptions.
|
||||
|
||||
## Decision 2 — do not reuse `incus copy --refresh` / migration API
|
||||
|
||||
Considered and rejected. `copy --refresh` is `POST /1.0/instances`
|
||||
with `source.refresh=true` (`shared/api/instance.go:373`) and is
|
||||
**not** zfs-only — transports are negotiated per storage driver
|
||||
(`MigrationTypes()`: zfs/btrfs/ceph native streams, universal RSYNC
|
||||
fallback, `driver_common.go:230`). But:
|
||||
|
||||
- the receiver must be another incus daemon speaking the migration
|
||||
websocket protocol (`internal/migration/migrate.proto`);
|
||||
- the optimized payload is an opaque `zfs send` stream — storing
|
||||
those in plakar means chain-of-increments restore onto a real
|
||||
zpool, no per-file browse, no pruning, one corrupt link breaks the
|
||||
chain;
|
||||
- the RSYNC transport needs a materialized previous copy on the
|
||||
receiver — a full-size local mirror, i.e. leg 1 rebuilt inside a
|
||||
plugin;
|
||||
- gotcha: `driver_zfs.go:801` — refresh **without** snapshot copying
|
||||
forces RSYNC even zfs↔zfs (keep `snapshots.schedule` on sources).
|
||||
|
||||
`copy --refresh` stays what legs 1–2 use (host↔host replication);
|
||||
plakar is content-addressed archiving to dumb storage. Complementary,
|
||||
not competing.
|
||||
|
||||
## Decision 3 — read the `backup` project replicas, not live instances
|
||||
|
||||
Reading a live rootfs is fuzzy. Leg 1 (`incus-copy.sh -p backup -s
|
||||
backup`, 01:00) already produces quiescent, crash-consistent stopped
|
||||
replicas — the importer targets those (`?project=backup`), scheduled
|
||||
after the copy completes. DB consistency remains the mariadb-dump
|
||||
job's responsibility (plakar `mysql` importer / dump step).
|
||||
|
||||
## Decision 4 — enumerate everything by default
|
||||
|
||||
`incus-backup.db` manifest drift (stale `spot`, missing `livetrail`,
|
||||
`outline`, `login`…) is the failure mode to kill: the importer backs
|
||||
up **all instances in the project by default**, opt-out via config,
|
||||
never opt-in.
|
||||
|
||||
## Other design points
|
||||
|
||||
- Location scheme `incus://<remote>/<project>/<instance>/...` —
|
||||
stable pathnames keep the kloset VFS cache effective.
|
||||
- Synthetic records per instance: config + profiles
|
||||
(`GET /1.0/instances/{name}`, expanded) so restore can recreate the
|
||||
instance before pushing files back.
|
||||
- Optional later `mode=image`: full tarball via the backups API for
|
||||
occasional exact-image restores (`incus import`); plakar dedups
|
||||
unchanged chunks between runs. Not the nightly path.
|
||||
- Restore path (exporter): create instance from stored config → push
|
||||
file tree back via sftp. Weaker than `incus import` but granular.
|
||||
|
||||
## Packaging & deployment
|
||||
|
||||
Plugins are standalone executables (gRPC over stdio) — no rebuild of
|
||||
plakar, fully compatible with the APT-installed binary on ks4:
|
||||
|
||||
```sh
|
||||
cd plakar/integration-incus && make # build importer/exporter
|
||||
plakar pkg create manifest.yaml v0.1.0 # → incus_v0.1.0_linux_amd64.ptar
|
||||
# on ks4, as root (plugins are per-user, cron runs as root):
|
||||
plakar pkg add ./incus_v0.1.0_linux_amd64.ptar
|
||||
plakar source add ks4-incus incus://... # then list it in /root/scripts/plakar-sources
|
||||
```
|
||||
|
||||
Installing a *local* `.ptar` file needs no `plakar login` (the plugin
|
||||
registry is only for fetching by name). Model the `manifest.yaml` on
|
||||
integration-proxmox (`tier: third-party`, `api_version: v1.1.0`).
|
||||
Compatibility contract = the go-kloset-sdk wire protocol: pin the SDK
|
||||
to the installed plakar line (v1.1.x), rebuild the ptar on a plakar
|
||||
major upgrade; a mismatch surfaces at backup time, so run a manual
|
||||
`plakar backup @ks4-incus` after plakar upgrades.
|
||||
|
||||
## Open questions
|
||||
|
||||
- uid/gid view through the instance sftp endpoint (idmap handling for
|
||||
unprivileged containers) — verify what stat returns and what restore
|
||||
must remap.
|
||||
- xattrs / ACLs / device nodes over sftp — coverage and fidelity.
|
||||
- VMs: file access needs a running incus-agent — out of scope for ks4
|
||||
(containers only); document the limitation.
|
||||
- Scheduling/locking: chain after the 01:00 incus-copy (flock on the
|
||||
same lock, or a wrapper script).
|
||||
- S3 hardening: scoped credentials (no delete), versioning/object
|
||||
lock, plakar client-side encryption passphrase storage.
|
||||
Reference in New Issue
Block a user