From ee9eaee889bfa322f038767757a0f949ba86bac7 Mon Sep 17 00:00:00 2001 From: Julien Lutran Date: Sun, 9 Aug 2026 22:50:07 +0200 Subject: [PATCH] Initial import: nuc and ks4 infrastructure documentation README with instance tables and nuc<->ks4 network flow chart; per-container install/troubleshooting docs for nuc (jellyfin server/client, transmission-bt, bare-metal reinstall) and the ks4 two-leg backup scheme (incus-copy over wireguard). Co-Authored-By: Claude Fable 5 --- README.md | 102 ++++++++++++++++ ks4/incus-copy.md | 147 +++++++++++++++++++++++ ks4/install.md | 232 ++++++++++++++++++++++++++++++++++++ nuc/jellyfin-client.md | 260 ++++++++++++++++++++++++++++++++++++++++ nuc/jellyfin-server.md | 107 +++++++++++++++++ nuc/nuc-install.md | 261 +++++++++++++++++++++++++++++++++++++++++ nuc/transmission-bt.md | 146 +++++++++++++++++++++++ 7 files changed, 1255 insertions(+) create mode 100644 README.md create mode 100644 ks4/incus-copy.md create mode 100644 ks4/install.md create mode 100644 nuc/jellyfin-client.md create mode 100644 nuc/jellyfin-server.md create mode 100644 nuc/nuc-install.md create mode 100644 nuc/transmission-bt.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..f1ecc35 --- /dev/null +++ b/README.md @@ -0,0 +1,102 @@ +# perso + +Personal infrastructure notes. + +- [`nuc/`](nuc/) — home lab on `nuc` (see below) +- [`ks4/`](ks4/) — prod server `ks4` at OVH (see below) + +## Homelab: nuc + +Incus host on the LAN. + +- Host: `nuc` — `192.168.0.3`, Debian 13, Intel Alder Lake-N + (iGPU `i915`, shared by both Jellyfin containers) — + bare-metal reinstall: [nuc/nuc-install.md](nuc/nuc-install.md) +- SSH: `ssh -i id_rsa_claude root@192.168.0.3` (keys at repo root, gitignored) +- Instances are bridged onto the LAN (192.168.0.0/24) +- USB 4 TB WD Red: ZFS pool `usb4t` — `usb4t/backup` → `/backup` + (incus exports, `nuc/` + `ks4/` subdatasets, 1 TB quota) and + `usb4t/media` → `/srv/media` (media library, shared into containers + via `shift=true` disk devices; works because ZFS ≥ 2.2 supports + idmapped mounts) +- NAS: `//192.168.0.10/Shared` (CIFS) — original media source, normally + offline; not mountable into containers (idmapped mounts unsupported + on CIFS) +- Backups: all local instances replicated to the USB pool + (`/root/scripts/incus-copy.sh -p backup -s nucbackup`; replicas + stopped, autostart off) — see + [nuc/nuc-install.md](nuc/nuc-install.md); ks4 replicas pulled into + pool `ks4backup` — see [ks4/incus-copy.md](ks4/incus-copy.md) + +### Instances + +| Name | IP | Doc | Features | +|---|---|---|---| +| [jellyfin-server](nuc/jellyfin-server.md) | 192.168.0.5 | ✅ | unprivileged, autostart; iGPU render node (`gpu` device, render gid) for QSV/VAAPI transcoding; `/srv/media` disk device (`shift=true`); proxy device → host :8096 | +| [jellyfin-client](nuc/jellyfin-client.md) | 192.168.0.6 | ✅ | **privileged**, autostart; full iGPU (`gpu` device, gid 44) → HDMI kiosk (cage + Jellyfin Media Player); custom `raw.lxc` (bind `/dev/snd`, `/dev/input`, host `/run/udev`); Pioneer USB audio as ALSA default; FR keymap; go-librespot Spotify Connect ("Pioneer A-70") | +| [transmission-bt](nuc/transmission-bt.md) | 192.168.0.7 | ✅ | unprivileged, autostart; always-on WireGuard full tunnel → ks4 (egress = 193.70.35.17, kill switch: no default route); `/srv/media` disk device (`shift=true`), downloads to `/media/downloads`; web UI :9091 (LAN only) | +| blocky | 192.168.0.254 | — | unprivileged, autostart; DNS ad-blocker | +| privoxy | 192.168.0.11 | — | unprivileged, autostart; filtering HTTP proxy | +| pihole | (stopped) | — | unprivileged; superseded by blocky | +| homeassistant | (stopped) | — | **virtual machine**, 50 GiB root disk on pool `data` | + +## Prod: ks4 + +Incus host at OVH — public-facing self-hosted services. + +- Host: `ks4.lutran.fr` — `193.70.35.17`, **SSH on port 2233**, + Debian 13, Xeon D-1521 / 32 GiB, 2× 6 TB (OS on mdraid RAID1, + ZFS pool `data` on `sda5`) — setup & rebuild: + [ks4/install.md](ks4/install.md) +- SSH: `ssh -i id_rsa_claude -p 2233 julien@193.70.35.17` + (`julien` has passwordless `sudo incus …`) +- Instances are on a NAT bridge `incusbr0` (192.168.1.0/24); public + ingress via Incus proxy devices. `gateway` (nginx) is the sole HTTP/S + entry point and fans out to the app containers. +- Services: gateway, mail, nextcloud, seafile, git, freshrss, bitwarden, + outline, login (SSO), wireguard, + more — full table in + [ks4/install.md](ks4/install.md). +- ⚠️ The ZFS `data` pool is single-disk (not mirrored); durability + rests on two nightly root cron jobs — `incus copy --refresh` to remote + host `ks2` (moving to nuc pool `ks4backup`, see + [ks4/incus-copy.md](ks4/incus-copy.md)), and an rsync backup + (`scripts/incus-backup.sh`) to `164.132.173.57:/backup/ks4`. + +## Network flows (nuc <-> ks4) + +``` + nuc — home LAN 192.168.0.0/24 ks4 — OVH 193.70.35.17 ++-----------------------------------+ +-------------------------------------+ +| | | | +| host: wg-ks4 (10.8.0.20) | | [wireguard] 192.168.1.18 | +| incus remote "ks4" ------+--WG-->| wg0 10.8.0.1/24, udp 51845 | +| pull ks4:* -> pool ks4backup | udp | | masquerade -> eth0 | +| on usb4t [pending FTTH seed] | 51845 | | | +| | | +-> incus API 192.168.1.1:8443 | +| [transmission-bt] wg0 (10.8.0.21) | | | (ufw: only from .18) | +| full tunnel 0.0.0.0/0 ------+--WG-->| | | +| kill switch: no default route | udp | +-> WAN egress: torrents + | +| downloads -> /srv/media | 51845 | apt of transmission-bt | +| (usb4t/media, read by jellyfin) | | exit as 193.70.35.17 | +| | | | +| 03:00 instance snapshots | | 03:00 instance snapshots | +| 03:30 incus-copy: all instances | | 01:00 incus-copy: all instances | +| -> project backup, pool | | -> project backup, zpool | +| nucbackup (usb4t/backup/nuc) | | backup (sdb5) [planned] | +| 05:00 apt upgrade all containers | | 04:00 incus-backup.sh: DB dumps | +| | | -> /backup (sdb5) [planned] | ++-----------------------------------+ +-------------------------------------+ + phones/laptops: WG peers 10.8.0.2-3 reach 192.168.1.x through the same endpoint +``` + +Both tunnels initiate **from** nuc (home NAT, dynamic IP) toward ks4's +fixed endpoint; ks4's incus API is never exposed to the internet. + +### Conventions + +- One markdown file per instance in `nuc/`, containing the full + install script (idempotent-ish, run as root on the host), first-run + configuration, and troubleshooting notes. +- Containers are built from `images:ubuntu/24.04` and configured + exclusively through `incus exec`/`incus config` so the doc is the + single source of truth — rebuilding = re-running the script. diff --git a/ks4/incus-copy.md b/ks4/incus-copy.md new file mode 100644 index 0000000..8edeb07 --- /dev/null +++ b/ks4/incus-copy.md @@ -0,0 +1,147 @@ +# incus-copy — ks4 instance replication + +Incremental replication of all ks4 Incus instances via +[`scripts/incus-copy.sh`](../scripts/incus-copy.sh) +(`incus copy --refresh`: ZFS-incremental, produces ready-to-start +replicas instead of tarballs). Two legs replace the historical nightly +push to `ks2` (decommissioning): + +1. **local** — replicas + dumps on a dedicated `backup` zpool on ks4's + second disk (`sdb5`), survives `sda` death +2. **off-site** — replicas pulled by **nuc** into pool `ks4backup` + (dataset `usb4t/backup/ks4`), survives losing ks4 entirely + +## The script + +``` +incus-copy.sh -d [-m pull|push|relay] [-s ] # push local → remote +incus-copy.sh -r [-m pull|push|relay] [-s ] # pull remote → local +incus-copy.sh -p -s # local → local project +``` + +- Copies **all** instances, running and stopped — a stopped instance's + filesystem never changes, so its refresh transfers nothing after the + first copy. +- `flock` on `/run/lock/incus-copy.lock` — overlapping cron runs abort + instead of racing the same instance. +- After each successful copy it sets `boot.autostart=false` on the + **replica** — otherwise the copy inherits autostart and would start + itself on the backup host after a reboot (or, for the local-project + leg, fight the live instance for its static IP on the same bridge). +- Exits non-zero if any instance failed (cron/monitoring can alert); + per-instance failures go to stderr, progress to stdout. +- Snapshot history is the **source's** job — set on ks4, e.g.: + `incus profile set default snapshots.schedule="0 3 * * *" snapshots.expiry=7d`. + `--refresh-exclude-older` keeps expired snapshots from being re-sent. +- Consistency: copies of running instances are crash-consistent; + databases may need recovery on restore. Snapshot history mitigates, + and [`incus-backup.sh`](../scripts/incus-backup.sh) (proper + `mariadb-dump`s + selected paths, manifest-driven) covers the DBs. +- **VMs**: `--refresh` re-sends a VM's **full block volume** every run + unless source and replica share a common snapshot — containers diff + cheaply regardless (verified on nuc: homeassistant re-sent 3.3 GiB, + containers only KBs). Set `snapshots.schedule` on sources *before* + enabling the crons, especially for the WAN leg. + +## Leg 1 — local backup pool on ks4 (sdb5) + +One-time setup (root on ks4). `sdb1-4` mirror the OS (mdraid); `sdb5` +was unused: + +```sh +# find sdb's serial: ls -l /dev/disk/by-id/ | grep sdb5 +zpool create -m none backup /dev/disk/by-id/ata-HGST_HUS726T6TALE6L1_-part5 +zfs create backup/incus # incus-managed (replicas) +zfs create -o mountpoint=/backup backup/dumps # incus-backup.sh output +incus storage create backup zfs source=backup/incus +# replicas live in their own project so names don't collide with the +# live instances; it shares the default project's profiles +incus project create backup -c features.images=false -c features.profiles=false +``` + +Cron (root on ks4) — replaces both ks2 jobs: + +```cron +0 1 * * * /root/scripts/incus-copy.sh -p backup -s backup >> /var/log/incus-copy.log 2>&1 +0 4 * * * /root/scripts/incus-backup.sh -f /root/scripts/incus-backup.db -s data >> /var/log/incus-backup.log 2>&1 +``` + +(`incus-backup.sh` without `-d` now writes locally to +`/backup/$(hostname -s)/` — the `backup/dumps` dataset.) + +⚠️ Replicas in the `backup` project must stay **stopped** — they keep +the live containers' static `192.168.1.x` addresses. + +## Leg 2 — off-site pull from nuc + +Storage pool on nuc (done 2026-08-09): `ks4backup`, backed by the +dataset `usb4t/backup/ks4` on the USB 4 TB pool (quota on `usb4t/backup` +removed 2026-08-09 — full replica set is ~1.75 TiB): + +```sh +incus storage create ks4backup zfs source=usb4t/backup/ks4 +``` + +Replicas live only on the USB drive — if it fails, only backups are +lost; nuc's own instances (pool `data` on the SSD) are unaffected. + +**Direction: nuc pulls, through the WireGuard tunnel.** Verified +2026-08-09: ks4's API listens on wildcard `:8443` (so it answers on +`192.168.1.1`, the `incusbr0` host address) but is **firewalled from +the internet** — the VPN path keeps it that way, needs no inbound port +at home, and doesn't care that nuc's public IP is dynamic. The +`wireguard` container (`192.168.1.18`, `wg0` `10.8.0.1/24`) is exposed +via a proxy device on public UDP `51845`. + +Setup (✅ **done 2026-08-09**, verified end-to-end with +`incus list ks4:` from nuc): + +- **wireguard container** (ks4): forwards + masquerades wg0→eth0 + (pre-existing); nuc added as peer `10.8.0.20/32` + (`wg set wg0 peer 31Tlgloc… allowed-ips 10.8.0.20/32` + + `wg-quick save wg0`). +- **ufw** (ks4): `ufw allow in on incusbr0 from 192.168.1.18 to any + port 8443 proto tcp` — the API stays firewalled from the internet + and the connection arrives masqueraded as the WG container. +- **nuc**: `/etc/wireguard/wg-ks4.conf` (`wg-quick@wg-ks4` enabled; + peer = container pubkey `TVs6d7…`, endpoint `193.70.35.17:51845`, + `AllowedIPs = 10.8.0.0/24, 192.168.1.1/32`, keepalive 25s) and + `incus remote add ks4 https://192.168.1.1:8443 --accept-certificate + --token '…'` (fingerprint cross-checked against the token). + +In pull mode all control + migration traffic flows over that single +API connection, so nothing else needs routing. Copying the `wireguard` +container over its own tunnel is fine (crash-consistent, tiny, no +interruption); if the tunnel is down the cron job fails loudly instead +of hanging. + +Then cron (root on nuc) — stagger after ks4's local leg: + +```cron +30 2 * * * /root/scripts/incus-copy.sh -r ks4 -s ks4backup >> /var/log/incus-copy.log 2>&1 +``` + +The **initial seed is ~1.75 TiB** over the home downlink — run the +first `incus-copy.sh -r ks4 -s ks4backup` manually (tmux), enable the +cron once it completes. + +## Cutover checklist (then kill ks2) + +1. First full cycle of all three jobs clean (logs above). +2. Restore test: on nuc, start a small replica (e.g. `freshrss`) with + its NIC detached, check app data, then stop it. +3. Remove both ks2 cron lines on ks4, `incus remote remove ks2`, + cancel the server (`164.132.173.57` = ks2, rsync target of the old + 4 AM job). + +## Restore + +```sh +# from nuc (off-site replica): +incus copy ks4: --mode push +# from the local backup project (sda replaced, pool data rebuilt): +incus copy --project backup --target-project default -s data +``` + +Remember replicas have `boot.autostart=false`; re-enable after a real +failover, and re-check it after copying back to ks4. diff --git a/ks4/install.md b/ks4/install.md new file mode 100644 index 0000000..433ec87 --- /dev/null +++ b/ks4/install.md @@ -0,0 +1,232 @@ +# ks4 — prod server setup & rebuild procedure + +Prod Incus host at OVH (`ks4.lutran.fr` → `193.70.35.17`, SSH on port +**2233**). This is a scan-derived snapshot of how the box is currently +laid out plus the steps to rebuild the host layer. Per-instance rebuild +docs are still TODO — see [Instances](#instances). + +> Scanned 2026-08-09 as `julien` (in groups `sudo`, `incus`) with +> passwordless `sudo incus …` only. Anything needing root beyond incus +> (`zpool status`, host firewall dumps, container internals) is marked +> **verify** below. + +## Hardware / hosting + +- OVH dedicated server, hostname `ns3061243`, up ~90 days at scan time +- CPU: Intel Xeon D-1521 @ 2.40 GHz (4c/8t) +- RAM: 31 GiB (+1 GiB swap) +- Disks: 2× 6 TB HGST `HUS726T6TALE6L1` (`sda`, `sdb`) + - `sda1/2/3` + `sdb1/2/3` → **mdraid RAID1** mirrors: + `md1` → `/boot/efi` (vfat), `md2` → `/boot` (ext4), + `md3` → `/` (ext4, 40 GB) + - `sda4` / `sdb4` → swap (512 MB each) + - `sda5` (5.4 TB) → ZFS pool `data` (all Incus instance disks), + single vdev `ata-HGST_…_V9K2UX7L-part5`, `ONLINE`, weekly scrub + (Sundays, last clean run 2026-08-09) + - `sdb5` (5.4 TB) → **unused** (no fstype) +- Network: `eno3` up, `193.70.35.17/24`, gw `193.70.35.254`, + IPv6 `2001:41d0:303:711::1/128`. `eno4` down. +- OS: Debian 13 (trixie), kernel `6.12.86+deb13-amd64` + +### ⚠️ Data pool is NOT mirrored + +Confirmed via `zpool status`: the OS (`md1/2/3`) is RAID1 across both +disks, but the ZFS `data` pool is a **single vdev on `sda5`**. If `sda` +dies, every instance disk is lost even though the OS survives on `sdb`. +Decision (2026-08-09): instead of mirroring, `sdb5` hosts a dedicated +`backup` zpool receiving nightly replicas + dumps, with a second +off-site replica on nuc — see [incus-copy.md](incus-copy.md). + +(`zpool status` also notes some pool features are disabled — an +optional `zpool upgrade data` would enable them, at the cost of +compatibility with older ZFS.) + +## Software layer + +- **Incus** from the Zabbly stable repo (`pkgs.zabbly.com/incus/stable`, + suite `trixie`), packages `incus`, `incus-base`, `incus-client`, + `incus-ui-canonical`, version `1:7.2-debian13-…`. +- ZFS `2.4.3-1~bpo13+1` (`zfs-dkms`, `zfsutils-linux`, `zfs-zed`). +- Incus daemon: `core.https_address: :8443` (web UI reachable; TLS auth), + firewall driver **nftables**, storage driver **zfs**, not clustered. + +Repo + install (matches nuc): +```sh +mkdir -p /etc/apt/keyrings +curl -fsSL https://pkgs.zabbly.com/key.asc -o /etc/apt/keyrings/zabbly.asc +cat > /etc/apt/sources.list.d/zabbly-incus-stable.sources < `~/incus-compose` on the host is a Go project (own GitLab CI) — a +> docker-compose-style deployer for Incus that julien develops. The +> `incus-compose-images` project is its image store. Not required to run +> the existing containers. + +## Instances + +All in the `default` project, on the `default` profile +(`eth0` → `incusbr0`, `root` → pool `data`), unprivileged +(`security.nesting=true`), with daily snapshots +(`snapshots.schedule=@daily`, expiry 1w / 4w manual, ~7 kept). + +**Running:** + +| Name | IP | Role (inferred) | Base image | Notes | +|---|---|---|---|---| +| gateway | .2 | **nginx reverse proxy / TLS** — HTTP(S) ingress | Ubuntu 18.04 | proxy 80/443, PROXY protocol | +| mail | .3 | **mail server** (Postfix/Dovecot stack) | Debian 9.9 (Apache/MariaDB/PHP) | proxy 25/587/993; ~20 GB | +| nextcloud | .4 | **Nextcloud** | Debian stretch | ~726 GB data | +| freshrss | .5 | **FreshRSS** reader | — | dedicated `root` disk device | +| git | .6 | **Git server** (Gitea/GitLab-class) | Ubuntu 18.04 | SSH proxy host:2244→:22 | +| seafile | .7 | **Seafile** file sync | — | ~950 GB, dedicated `root` disk | +| databap | .8 | app (role unconfirmed — **verify**) | Ubuntu 18.04 | | +| spot | .9 | app (role unconfirmed — **verify**; joal/torrent?) | Debian 9.9 | ~25 GB | +| bitwarden | .15 | **Bitwarden / Vaultwarden** | Ubuntu 18.04 | ~23 GB | +| solar | .17 | app (solar/PV monitoring? — **verify**) | Debian 9.9 | | +| wireguard | .18 | **WireGuard VPN** | Debian trixie (rebuilt 2025-11-28) | proxy udp:51845; wg0 10.8.0.1/24 | +| login | .19 | **SSO / auth** (runs Docker inside) | Debian 9.9 | `docker0`+bridge; nesting, syscall intercepts | +| outline | .20 | **Outline** wiki (runs Docker inside) | Debian 9.9 | `docker0`+bridge; nesting, syscall intercepts | + +**Stopped (legacy / occasional):** `catc`, `mythoughts`, `qcm`, +`wedding`, `wow` — no snapshots, kept around; roles not documented. + +> Most base images are old (Ubuntu 18.04 / Debian 9 "stretch", both EOL). +> Treat these as pets to migrate, not reproduce. Per-instance install +> docs (one `.md` each, per repo convention) still need to be written — +> that requires shelling into each container, which the current +> incus-only sudo grant doesn't allow. + +## Backups + +> **Migration in progress** (2026-08-09): the `ks2` destination below is +> being replaced by a local `backup` zpool on `sdb5` + off-site pull +> from nuc — plan and setup in [incus-copy.md](incus-copy.md). The +> cron blocks here describe what runs **today**; update on cutover. + +Two **root** cron jobs (this repo's [`scripts/`](../scripts/) are the +copies that run here) — the primary safety net for the non-mirrored +`data` pool: + +```cron +0 1 * * * /root/scripts/incus-copy.sh -d ks2 -m push +0 4 * * * /root/scripts/incus-backup.sh -d 164.132.173.57 -u rsyncbackup \ + -i /root/.ssh/id_rsyncbackup -p 2233 \ + -f /root/scripts/incus-backup.db -s data +``` + +- **01:00 — `incus-copy.sh` → `ks2`**: differential `incus copy … \ + --refresh --refresh-exclude-older --mode push` of every *running* + instance to remote incus host `ks2` (a near-live replica). `ks2` must + exist as an `incus remote`. +- **04:00 — `incus-backup.sh` → `164.132.173.57`**: rsync-over-ssh (user + `rsyncbackup`, key `/root/.ssh/id_rsyncbackup`, port 2233) of the incus + local+global DB dumps, each container's `backup.yaml`, MariaDB dumps + (`incus exec … mariadb-dump`), and selected rootfs paths into + `/backup/ks4/` on the remote. **Which** containers/DBs/paths are + backed up is driven by the JSON manifest + `/root/scripts/incus-backup.db` — keep it current when adding a + service. + +Both scripts run as root (they use `incus exec` / `incus admin sql`), +not through the `julien` incus-only sudo grant. Logs: +`/var/log/incus-copy.log`, `/var/log/incus-backup.log`. + +Manual export of a single instance: +```sh +sudo incus export /path/to/.tar.gz # --instance-only to skip snapshots +``` + +## Host rebuild outline + +OVH reinstalls provide Debian; the ks4-specific layer is: + +1. **OS**: Debian 13, mdraid RAID1 (`md1`/`md2`/`md3`) across both disks + for `/boot/efi`, `/boot`, `/` — OVH's guided partitioning, or restore + the layout above. Leave `sda5` (and ideally `sdb5`) for ZFS. +2. **SSH**: move sshd to port **2233**; restore + `/root/.ssh/authorized_keys` (incl. `id_rsa_claude.pub` from this + repo) and the `julien` account (groups `sudo`, `incus`) + + `/etc/sudoers.d/incus`. +3. **ZFS pool**: + ```sh + zpool create data /dev/disk/by-id/ + # recommended: mirror it (see warning above) + zpool attach data /dev/disk/by-id/ + ``` +4. **Incus**: install from Zabbly (above), then `incus admin init` + with pool `data` and bridge `incusbr0` (`192.168.1.1/24`, NAT): + ```sh + cat < eth0 ipv4.address 192.168.1.x`), + and re-add the proxy devices from the + [ingress table](#networking--public-ingress), e.g.: + ```sh + incus config device add gateway https proxy \ + listen=tcp:0.0.0.0:443 connect=tcp:127.0.0.1:443 proxy_protocol=true + ``` +6. **Re-enable** daily snapshots on each instance, restore root's + crontab + `/root/scripts/` + `/root/.ssh/id_rsyncbackup`, add the + `ks2` incus remote, and confirm both backup jobs run (see + [Backups](#backups)). diff --git a/nuc/jellyfin-client.md b/nuc/jellyfin-client.md new file mode 100644 index 0000000..50092f4 --- /dev/null +++ b/nuc/jellyfin-client.md @@ -0,0 +1,260 @@ +# jellyfin-client + +HTPC kiosk **client** in a privileged Incus container on `nuc`: +Jellyfin Media Player fullscreen inside `cage` (Wayland kiosk compositor) +rendering directly to HDMI via DRM/KMS — no X server or desktop on the host. +Also hosts the Spotify Connect endpoint (go-librespot, see below). + +- Image: `images:ubuntu/24.04`, IP: `192.168.0.6` (LAN bridge) +- Video: Intel Alder Lake-N iGPU → HDMI (`gpu` device, full card access) +- Audio: Pioneer USB audio (`08e4:0176`), ALSA card `Device`, set as ALSA + default via `/etc/asound.conf` +- Input: Logitech Unifying receiver (K400) via `/dev/input` bind + host udev + database bind; FR (AZERTY) keymap +- Why privileged: hotplug-following directory bind-mounts of `/dev/snd` and + `/dev/input`. Unprivileged alternative: one `unix-char` device per node + (doesn't follow card renumbering on replug). + +## How the tricky parts work + +Container-specific pitfalls that cost debugging time — all handled by the +install script: + +1. **seatd must not bind a VT** (`SEATD_VTBOUND=0` drop-in): with VT binding + it tries to open the host's active tty, which isn't in the container, and + cage hangs forever ("running", `Tasks: 0`, black screen). +2. **libinput needs a udev database**: it only accepts input devices carrying + `ID_INPUT*` properties. The container can't run its own udevd (sysfs + uevent writes are silently denied by the Incus AppArmor profile — an + explicit deny rule that `raw.apparmor` cannot override), so the **host's** + `/run/udev` is bind-mounted read-only to `/opt/host-udev` and a + `run-udev.mount` unit re-binds it onto `/run/udev` at boot (a direct bind + would be shadowed by systemd's `/run` tmpfs). `WLR_LIBINPUT_NO_DEVICES=1` + additionally keeps cage alive when no input device is present. +3. **ALSA default must be pinned**: mpv opens device `default`, which maps to + card 0 (Intel HDA, HDMI-only pcm devices 3/7/8/9) → open fails and mpv + silently falls back to the **null** output (video OK, no sound). + `/etc/asound.conf` pins the default to the Pioneer by card *name*. +4. The kiosk service must NOT use `TTYPath`/`StandardInput=tty` — it hides + all cage/JMP errors on an invisible tty. Log to the journal. + +## Install script + +```bash +#!/usr/bin/env bash +set -euxo pipefail + +CNAME="${CNAME:-jellyfin-client}" +IMAGE="${IMAGE:-images:ubuntu/24.04}" +JMP_VER="${JMP_VER:-1.12.0}" # https://github.com/jellyfin/jellyfin-desktop/releases + +incus launch "$IMAGE" "$CNAME" -c security.privileged=true + +for i in $(seq 1 30); do + incus exec "$CNAME" -- getent hosts github.com >/dev/null 2>&1 && break + sleep 2 +done + +incus config set "$CNAME" environment.DEBIAN_FRONTEND=noninteractive + +# --- Host devices ------------------------------------------------------------- +# iGPU (card + render nodes). gid=44 = "video" group inside the container. +incus config device add "$CNAME" gpu gpu gid=44 + +# Sound (Pioneer USB audio -> ALSA card) and input devices (keyboard/remote). +# Directory bind-mounts follow hotplug events, so the USB card can be +# re-plugged without restarting the container. No VT/tty devices needed: +# seatd runs with SEATD_VTBOUND=0 (see below). +# +# The host udev database is bound to /opt/host-udev; a mount unit inside the +# container re-binds it to /run/udev at boot (binding /run/udev directly gets +# shadowed when systemd mounts its own tmpfs on /run). libinput only accepts +# input devices that carry ID_INPUT* properties from a udev database, and the +# container cannot run its own udevd (sysfs uevent writes are blocked by the +# Incus AppArmor profile), so it reads the host's database instead. +RAW_LXC="$(cat <<'EOF' +lxc.cgroup2.devices.allow = c 116:* rwm +lxc.cgroup2.devices.allow = c 13:* rwm +lxc.mount.entry = /dev/snd dev/snd none bind,optional,create=dir +lxc.mount.entry = /dev/input dev/input none bind,optional,create=dir +lxc.mount.entry = /run/udev opt/host-udev none bind,ro,optional,create=dir +EOF +)" +incus config set "$CNAME" raw.lxc="$RAW_LXC" +incus restart "$CNAME" # raw.lxc mount entries apply at container start +sleep 5 + +# --- Packages ------------------------------------------------------------------ +incus exec "$CNAME" -- apt-get update +incus exec "$CNAME" -- apt-get upgrade -y +incus exec "$CNAME" -- apt-get install -y --no-install-recommends software-properties-common curl ca-certificates +incus exec "$CNAME" -- add-apt-repository -y universe +incus exec "$CNAME" -- add-apt-repository -y multiverse +incus exec "$CNAME" -- apt-get install -y --no-install-recommends \ + cage seatd dbus dbus-user-session qtwayland5 \ + intel-media-va-driver-non-free vainfo mesa-utils-bin \ + alsa-utils + +# Jellyfin Media Player (repo renamed to jellyfin-desktop upstream) +incus exec "$CNAME" -- bash -c "curl -fLo /tmp/jmp.deb \ + https://github.com/jellyfin/jellyfin-desktop/releases/download/v${JMP_VER}/jellyfin-media-player_${JMP_VER}-noble.deb" +incus exec "$CNAME" -- apt-get install -y /tmp/jmp.deb + +# --- Default audio output = Pioneer ------------------------------------------- +# mpv opens ALSA device "default", which otherwise maps to card 0 (Intel HDA, +# HDMI-only pcm devices 3/7/8/9 -> open fails with ENOENT and mpv silently +# falls back to the null output). Pin the default to the Pioneer by card NAME +# so it survives card renumbering. "!" is required to override the compound +# definition in alsa.conf. +incus exec "$CNAME" -- bash -c 'cat > /etc/asound.conf </dev/null 2>&1 || useradd -m -G video,render,input,audio kiosk' + +# In a container seatd must NOT bind the seat to a VT (there is no usable VT; +# it would try to open the host's active tty and hang the compositor forever). +incus exec "$CNAME" -- mkdir -p /etc/systemd/system/seatd.service.d +incus exec "$CNAME" -- bash -c 'cat > /etc/systemd/system/seatd.service.d/novt.conf < /etc/systemd/system/run-udev.mount < /etc/systemd/system/jellyfin-kiosk.service </dev/null 2>&1 || useradd -r -m -d /var/lib/go-librespot -G audio spotify +mkdir -p /var/lib/go-librespot/config +cat > /var/lib/go-librespot/config/config.yml < /etc/systemd/system/go-librespot.service </dev/null 2>&1 && break + sleep 2 +done + +incus config set "$CNAME" environment.DEBIAN_FRONTEND=noninteractive +incus config set "$CNAME" environment.DEBCONF_NONINTERACTIVE_SEEN=true + +# --- Jellyfin from the official repository ----------------------------------- +incus exec "$CNAME" -- apt-get update +incus exec "$CNAME" -- apt-get upgrade -y +incus exec "$CNAME" -- apt-get install -y --no-install-recommends curl gnupg ca-certificates +incus exec "$CNAME" -- mkdir -p /etc/apt/keyrings +incus exec "$CNAME" -- bash -c 'curl -fsSL https://repo.jellyfin.org/jellyfin_team.gpg.key | gpg --dearmor -o /etc/apt/keyrings/jellyfin.gpg' +incus exec "$CNAME" -- bash -c 'cat > /etc/apt/sources.list.d/jellyfin.sources <:8096 to run the setup wizard." +``` + +## First-run configuration + +1. Run the setup wizard; add libraries pointing at `/media/...`. +2. Dashboard → Playback → Transcoding: + - Hardware acceleration: **Intel QuickSync (QSV)** (fallback: VA-API), + device `/dev/dri/renderD128`. + - Enable hardware decoding for the codecs you use; Alder Lake-N does + H.264/HEVC/VP9/AV1 decode and H.264/HEVC encode. + - Enable "Low-Power" encoders only if GuC/HuC is loaded (see below). + +## Troubleshooting / notes + +- **`Failed to setup device mount "media": idmapping abilities are required + but aren't supported on system`** — the media source is on a filesystem + without idmapped-mount support (CIFS/NFS). Either move the media to a + local fs, or drop the shift (`incus config device set jellyfin-server + media shift=false`; on CIFS files are world-readable synthetic ownership, + enough for a read-only library). Hot-applying `shift` on a running + container fails — stop it first. +- **QSV "low-power" encode fails** — Alder Lake-N needs GuC/HuC firmware + submission. On the **host**: + `echo 'options i915 enable_guc=3' > /etc/modprobe.d/i915.conf`, + `update-initramfs -u`, reboot; verify with `dmesg | grep -i 'guc\|huc'`. + Or just untick the low-power options. +- GPU check: `incus exec jellyfin-server -- ls -l /dev/dri`. +- Pin a version with `apt-get install jellyfin=+ubu2404` for + reproducibility; keep major versions in sync with the client's JMP. +- To bake a reusable image instead: run the script, then + `incus publish jellyfin-server --alias jellyfin-server-`. diff --git a/nuc/nuc-install.md b/nuc/nuc-install.md new file mode 100644 index 0000000..08d51e9 --- /dev/null +++ b/nuc/nuc-install.md @@ -0,0 +1,261 @@ +# nuc — bare-metal reinstall procedure + +How to rebuild the Incus host from scratch if `/dev/sda` (512 GB SSD, +`XINCUU-512GB`) has to be replaced. + +## Hardware + +- NUC-class mini PC, Intel Alder Lake-N (iGPU `i915`, HDMI to projector/TV) +- `sda`: 512 GB SSD — OS **and** ZFS pool `data` (all Incus instance disks) +- `sdb`: 4 TB WD Red (USB enclosure) — ZFS pool `usb4t`: + `usb4t/backup` → `/backup` (incus exports for nuc + ks4, 1 TB quota), + `usb4t/media` → `/srv/media` (media library) +- USB: Pioneer USB audio (`08e4:0176`), Logitech Unifying receiver (K400), + CSCTEK USB Audio and HID +- NIC: `enp1s0` (static `192.168.0.3/24`, gw `192.168.0.1`) + +## ⚠️ What dies with sda + +The ZFS pool `data` lives on `sda5` → **all instance root disks are lost** +with the OS. The USB pool `usb4t` (backups + media) survives — all +instances are replicated onto it with +[`scripts/incus-copy.sh`](../scripts/incus-copy.sh) (deployed at +`/root/scripts/` on nuc; set up 2026-08-09): + +```sh +# one-time setup (already done; re-run only on a fresh install) +incus storage create nucbackup zfs source=usb4t/backup/nuc +incus project create backup -c features.images=false -c features.profiles=false + +# refresh the replicas (all instances, stopped ones included) +/root/scripts/incus-copy.sh -p backup -s nucbackup +``` + +Runs nightly via `/etc/cron.d/incus-copy` at **03:30** (30 min after +the profile-scheduled 03:00 instance snapshots, so VM refreshes stay +incremental), logging to `/var/log/incus-copy.log` (logrotate: +`/etc/logrotate.d/incus-copy`). Note the script's `flock` is global: +if a future 02:30 ks4 pull overruns past 03:30, the local run aborts +loudly for that night instead of overlapping. + +Replicas sit **stopped** in project `backup` with `boot.autostart=false` +(the script enforces it). Containers refresh in seconds; the +homeassistant **VM re-sends its full volume** unless the source has +snapshots (`snapshots.schedule`) to diff against. + +`jellyfin-server` / `jellyfin-client` are also rebuildable from +[jellyfin-server.md](jellyfin-server.md) / +[jellyfin-client.md](jellyfin-client.md); the media library lives on +`usb4t/media` (originals on the NAS `//192.168.0.10/Shared`). + +## 1. Install Debian 13 (trixie) + +Netinst ISO, manual partitioning of the new sda (GPT/UEFI): + +| Part | Size | Type / FS | Mount | +|------|---------|-----------|-------------| +| sda1 | ~1 GB | EFI vfat | `/boot/efi` | +| sda2 | ~2 GB | ext4 | `/boot` | +| sda3 | ~47 GB | ext4 | `/` | +| sda4 | ~1 GB | swap | — | +| sda5 | rest (~427 GB) | **leave unformatted** (ZFS later) | — | + +- Tasks: only "SSH server" + standard utilities (no desktop). +- Sources: `main contrib non-free-firmware` (contrib is required for + `zfs-dkms`). + +## 2. Base system + +```sh +apt update && apt full-upgrade -y +apt install -y \ + firmware-intel-graphics firmware-iwlwifi firmware-realtek \ + firmware-sof-signed intel-microcode \ + linux-headers-amd64 zfs-dkms zfsutils-linux zfs-zed \ + cifs-utils curl vim htop ripgrep sysstat dmidecode pciutils usbutils +``` + +Static network — `/etc/network/interfaces` (ifupdown): + +``` +source /etc/network/interfaces.d/* + +auto lo +iface lo inet loopback + +allow-hotplug enp1s0 +iface enp1s0 inet static + address 192.168.0.3 + netmask 255.255.255.0 + gateway 192.168.0.1 + dns-nameservers 192.168.0.1 1.1.1.1 +``` + +Restore `/root/.ssh/authorized_keys` (3 keys; one is `id_rsa_claude.pub` +from this repo). + +## 3. ZFS pools + +```sh +# instance pool on the new SSD +ls -l /dev/disk/by-id/ | grep sda5 +zpool create data /dev/disk/by-id/-part5 + +# USB pool (backups + media) survived — just import it +zpool import usb4t +``` + +(If the old sda still works and only the OS was reinstalled elsewhere: +`zpool import data` instead — the instances survive.) + +`usb4t` reference (as created 2026-08: whole disk, `ashift=12`, +`compression=zstd`, `atime=off`, `xattr=sa`, `acltype=posixacl`; +datasets `usb4t/backup` → `/backup` with `quota=1T`, subdatasets +`nuc`/`ks4`, and `usb4t/media` → `/srv/media`). + +## 4. Incus (Zabbly stable repo) + +```sh +mkdir -p /etc/apt/keyrings +curl -fsSL https://pkgs.zabbly.com/key.asc -o /etc/apt/keyrings/zabbly.asc +cat > /etc/apt/sources.list.d/zabbly-incus-stable.sources < /etc/sudoers.d/incus <<'EOF' +## Allow harmless incus commands to be called through sudo +## without a password. +## +## CAUTION: Any syntax error introduced here will break sudo. +## Always edit/validate with: visudo -cf /etc/sudoers.d/incus +## +## Note: "incus config *" also covers config set/edit and device add, +## which can escalate (e.g. security.privileged, host disk mounts). +## Trim to "config show/get" if strict read-only is wanted. + +## Cmnd alias specification +Cmnd_Alias C_INCUS = \ + /usr/bin/incus config, /usr/bin/incus config *, \ + /usr/bin/incus list, /usr/bin/incus list *, \ + /usr/bin/incus info, /usr/bin/incus info *, \ + /usr/bin/incus version, \ + /usr/bin/incus top, \ + /usr/bin/incus monitor, /usr/bin/incus monitor *, \ + /usr/bin/incus snapshot list, /usr/bin/incus snapshot list *, \ + /usr/bin/incus image list, /usr/bin/incus image list *, \ + /usr/bin/incus image info *, \ + /usr/bin/incus profile list, /usr/bin/incus profile show *, \ + /usr/bin/incus project list, /usr/bin/incus project info *, \ + /usr/bin/incus network list, /usr/bin/incus network list *, \ + /usr/bin/incus network show *, /usr/bin/incus network info *, \ + /usr/bin/incus storage list, /usr/bin/incus storage show *, \ + /usr/bin/incus storage info *, \ + /usr/bin/incus storage volume list, /usr/bin/incus storage volume list *, \ + /usr/bin/incus operation list, /usr/bin/incus operation show *, \ + /usr/bin/incus remote list, \ + /usr/bin/incus warning list, /usr/bin/incus warning show * + +## allow julien to use harmless incus commands without a password +julien ALL = (root) NOPASSWD: C_INCUS +EOF +chmod 0440 /etc/sudoers.d/incus +visudo -cf /etc/sudoers.d/incus # must print "parsed OK" +``` + +## 5. Restore instances + +The replicas live on `usb4t/backup/nuc` and survived. Re-register them +with the fresh Incus, then copy back onto the rebuilt `data` pool: + +```sh +# recreate the containers' home first (section 4), then: +incus project create backup -c features.images=false -c features.profiles=false +incus admin recover # point it at pool nucbackup (zfs, source=usb4t/backup/nuc) + +# copy each replica back to the default project / SSD pool: +incus copy blocky blocky --project backup --target-project default -s data +incus config set blocky boot.autostart=true +incus start blocky +# … same for privoxy, homeassistant, jellyfin-* +``` + +(jellyfin-* can alternatively be rebuilt from +[jellyfin-server.md](jellyfin-server.md) / +[jellyfin-client.md](jellyfin-client.md); `/srv/media` is already there +once `usb4t` is imported.) + +## 6. Nightly automation + +| When | What | Where | +|-------|------|-------| +| 03:00 | instance snapshots (`snapshots.schedule` on the default profile, expiry 7d) | incus | +| 03:30 | replicate all instances to the USB pool (`incus-copy.sh -p backup -s nucbackup`) | `/etc/cron.d/incus-copy` → `/var/log/incus-copy.log` | +| 05:00 | apt dist-upgrade all running containers (`incus-container-upgrade.sh`; VMs and non-apt containers skipped; jellyfin pinned to the 10.11 series in-container) | `/etc/cron.d/incus-container-upgrade` → `/var/log/incus-container-upgrade.log` | + +The ordering is deliberate: snapshot → backup → upgrade, so a broken +upgrade is always one snapshot-restore away and the replicas predate it. +Both logs rotate monthly (`/etc/logrotate.d/incus-*`). + +## 7. Post-install checklist + +- [ ] `zpool status` healthy (both `data` and `usb4t`), `incus list` + shows expected instances +- [ ] incus ordered after ZFS mounts (instances bind-mount `/srv/media`): + `/etc/systemd/system/incus.service.d/after-zfs.conf` with + `[Unit]` / `After=zfs-mount.service zfs.target` +- [ ] `boot.autostart=true` on blocky, privoxy, jellyfin-* +- [ ] Host boots to `multi-user.target`, nothing grabs the GPU + (required by the jellyfin-client kiosk) +- [ ] Jellyfin web at `http://192.168.0.5:8096`, kiosk UI on HDMI, + sound on the Pioneer, "Pioneer A-70" visible in Spotify Connect +- [ ] LAN DNS: clients use blocky at `192.168.0.254` (host itself uses + `192.168.0.1` + `1.1.1.1` to avoid a bootstrap loop) +- [ ] Optional (only if QSV low-power encoders are wanted): + `echo 'options i915 enable_guc=3' > /etc/modprobe.d/i915.conf + && update-initramfs -u` diff --git a/nuc/transmission-bt.md b/nuc/transmission-bt.md new file mode 100644 index 0000000..cfa17fb --- /dev/null +++ b/nuc/transmission-bt.md @@ -0,0 +1,146 @@ +# transmission-bt + +BitTorrent client in an unprivileged Incus container on `nuc`, with an +**always-on VPN**: all peer traffic exits via ks4's public IP through a +WireGuard tunnel to the `wireguard` container on ks4. Kill switch by +construction — `eth0` has **no default route**, so with the tunnel down +the container simply has no path to the internet. + +- Image: `images:ubuntu/24.04`, IP: `192.168.0.7` (macvlan, static via netplan) +- Web UI: `http://192.168.0.7:9091` — no auth + (`rpc-authentication-required: false`); access control is the RPC + whitelist (`192.168.0.*` only) +- Egress: WG peer `10.8.0.21` → `193.70.35.17:51845`, `AllowedIPs 0.0.0.0/0` + (verified: `curl ifconfig.me` from the container returns ks4's IP) +- Downloads: `/media/downloads` (= `usb4t/media`, same dataset Jellyfin + reads); in-progress files in `/media/.incomplete` so Jellyfin never + scans partials +- `transmission-daemon` is `BindsTo=wg-quick@wg0.service` and binds + peer traffic to `10.8.0.21` — three independent layers against leaks + (no default route, unit binding, socket binding) + +## Anti-leak design + +1. netplan gives `eth0` only: LAN `/24` (web UI + DNS via blocky) and a + `/32` host route to the WG endpoint via the home gateway. +2. `wg-quick` full-tunnel mode adds its fwmark policy routing + + iptables anti-leak rule (`iptables` package required — its absence + makes `wg-quick` fail with `iptables-restore: command not found`). +3. LAN traffic keeps working thanks to wg-quick's + `suppress_prefixlength 0` rule (connected routes win over the + tunnel's default). + +## Install script + +Run as root on the Incus host. Requires the peer added on ks4 (below). + +```bash +#!/usr/bin/env bash +set -euxo pipefail + +CNAME="${CNAME:-transmission-bt}" +IMAGE="${IMAGE:-images:ubuntu/24.04}" + +incus launch "$IMAGE" "$CNAME" +sleep 8 +incus config set "$CNAME" environment.DEBIAN_FRONTEND=noninteractive + +# static LAN config, NO default route (kill switch), /32 to the WG endpoint +incus exec "$CNAME" -- bash -c 'cat > /etc/netplan/10-lxc.yaml < /etc/wireguard/wg0.key +wg pubkey < /etc/wireguard/wg0.key +cat > /etc/wireguard/wg0.conf < /etc/systemd/system/transmission-daemon.service.d/vpn.conf < allowed-ips 10.8.0.21/32 +incus exec wireguard -- wg-quick save wg0 +``` + +## Verification + +```sh +incus exec transmission-bt -- wg show wg0 latest-handshakes # non-zero timestamp +incus exec transmission-bt -- curl -s https://ifconfig.me # must print 193.70.35.17 +incus exec transmission-bt -- bash -c "ping -c1 -W2 8.8.8.8 || echo kill-switch OK" # with wg0 down +# web UI must be tested from a LAN machine — the macvlan quirk means the +# nuc host itself cannot reach 192.168.0.7 +``` + +## Notes + +- No inbound peer port is forwarded (would need a proxy device on ks4 + + DNAT through the tunnel); torrents work fine outbound-only, just + connect to fewer peers. +- Jellyfin sees finished downloads under `/media/downloads` — add it as + a library folder or move files into the movie/show trees. +- The image server check can make `incus launch` hang on slow WAN — + launching from the cached image fingerprint (`incus image list`) + bypasses it.