julien/doc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

nuc: media over NFS from nas, on-demand role, i915 and input traps

Browse Source 
nuc keeps only what needs its iGPU. blocky, privoxy and transmission-bt
moved to nas so the box can be powered off when not watching Jellyfin or
using the Spotify kiosk.

- /srv/media is now an NFSv4 mount from nas; jellyfin-server reads it
  with shift=false (idmapped mounts are unsupported on NFS, as the
  container doc already noted for CIFS) and readonly=true
- replication to nas is a systemd timer with Persistent=true, not cron —
  an on-demand host misses its 03:30 window and cron cannot catch up

Two failures documented in full, both diagnosed from the wrong layer
first:

- booting with the TV connected and powered on kills the i915 probe
  (drm_WARN_ON in intel_modeset_setup_hw_state), so /dev/dri never
  appears, snd_hda_intel deferred-probes forever holding the PCI device
  lock, and incusd blocks in sriov_numvfs_show — no container starts at
  all, including LAN DNS. Identical on 6.12.107 and 6.12.105.
- the kiosk input gid mismatch is real but was NOT the cause of the
  2026-08-30 outage (flat K400 batteries were); seatd opens input devices
  as root, so kiosk group membership is not on that path. Records the
  one-line raw capture that settles hardware-vs-software immediately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Julien Lutran
2026-08-30 23:53:30 +02:00
co-authored by Claude Opus 5
parent 66c7bca28f
commit 203728674f
4 changed files with 225 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
nuc/README.md
+32 -18
View File
@@ -1,31 +1,45 @@
# Homelab: nuc
Incus host on the LAN.
Incus host on the LAN — **on-demand**: since 2026-08-30 it only needs to
run when watching Jellyfin or using the Spotify Connect kiosk. Everything
always-on (blocky/DNS, privoxy, transmission-bt) moved to
[`nas`](../nas/README.md), which is why nuc can now be powered off.
⚠️ Powering nuc off has backup consequences — its 03:30 replication to
nas only runs while it is up. See [nas-install.md](../nas/nas-install.md) §9.
- Debian 13, Intel Alder Lake-N (iGPU `i915`, shared by both Jellyfin containers) —
bare-metal reinstall: [nuc-install.md](nuc-install.md)
- Instances are bridged onto the LAN (192.168.0.0/24)
- USB 4 TB WD Red: ZFS pool `usb4t` — `usb4t/backup` → `/backup`
(incus exports, `nuc/` + `ks4/` subdatasets, 1 TB quota) and
`usb4t/media` → `/srv/media` (media library, shared into containers
via `shift=true` disk devices; works because ZFS ≥ 2.2 supports
idmapped mounts)
- ⚠️ The USB enclosure drops off the bus periodically and suspends the
pool, which silently breaks the nightly replication — symptoms,
recovery and fixes: [usb4t-dropouts.md](usb4t-dropouts.md)
- Backups: all local instances replicated to the USB pool
(`/root/scripts/incus-copy.sh -p backup -s nucbackup`; replicas
stopped, autostart off) — see
[nuc-install.md](nuc-install.md); ks4 replicas pulled into
pool `ks4backup` — see [ks4/incus-copy.md](../ks4/incus-copy.md)
- Storage: ZFS pool `data` on the SSD (all instance disks). The USB
4 TB and its pool **left nuc on 2026-08-30** — the enclosure's JMicron
bridge was suspending the pool ([usb4t-dropouts.md](usb4t-dropouts.md));
the disk now sits on direct SATA in [`nas`](../nas/nas-install.md) as
pool `tank`.
- Media library: `/srv/media` is now an **NFSv4 mount from nas**
(`192.168.0.4:/media`, read-only). `jellyfin-server` stays here for
the iGPU and reads it with `shift=false`
([jellyfin-server.md](jellyfin-server.md)).
- Backups: nuc and nas **cross-replicate** — nuc pushes all its
instances to `nas:nucbackup` at 03:30, nas pushes its own to
`nuc:nasbackup` at 04:00, so neither host's instances depend on that
host surviving ([nas/nas-install.md](../nas/nas-install.md) §9).
The ks4 pull leg moved to nas as well
([ks2/nas-seed.md](../ks2/nas-seed.md)), so nuc no longer runs a
WireGuard tunnel.
- ⚠️ **Never reboot nuc with the display powered on** — the i915 probe
dies and takes every container with it (including LAN DNS). Symptoms
and fix: [jellyfin-client.md](jellyfin-client.md#-never-boot-nuc-with-the-display-active-2026-08-30)
## Instances
| Name | IP | Doc | Features |
|---|---|---|---|
| [jellyfin-server](jellyfin-server.md) | 192.168.0.5 | ✅ | unprivileged, autostart; iGPU render node (`gpu` device, render gid) for QSV/VAAPI transcoding; `/srv/media` disk device (`shift=true`); proxy device → host :8096 |
| [jellyfin-server](jellyfin-server.md) | 192.168.0.5 | ✅ | unprivileged, autostart; iGPU render node (`gpu` device, render gid) for QSV/VAAPI transcoding; `/srv/media` disk device (NFS from nas, `shift=false`, read-only); proxy device → host :8096 |
| [jellyfin-client](jellyfin-client.md) | 192.168.0.6 | ✅ | **privileged**, autostart; full iGPU (`gpu` device, gid 44) → HDMI kiosk (cage + Jellyfin Media Player); custom `raw.lxc` (bind `/dev/snd`, `/dev/input`, host `/run/udev`); Pioneer USB audio as ALSA default; FR keymap; go-librespot Spotify Connect ("Pioneer A-70") |
| [transmission-bt](transmission-bt.md) | 192.168.0.7 | ✅ | unprivileged, autostart; always-on WireGuard full tunnel → ks4 (egress = 193.70.35.17, kill switch: no default route); `/srv/media` disk device (`shift=true`), downloads to `/media/downloads`; web UI :9091 (LAN only) |
| blocky | 192.168.0.254 | — | unprivileged, autostart; DNS ad-blocker |
| privoxy | 192.168.0.11 | — | unprivileged, autostart; filtering HTTP proxy |
| homeassistant | (stopped) | — | **virtual machine**, 50 GiB root disk on pool `data` |
Moved to [`nas`](../nas/README.md) on 2026-08-30: `transmission-bt`
(next to the media dataset it writes to), plus `blocky` and `privoxy`
so LAN DNS and the proxy survive nuc being shut down.
nuc/jellyfin-client.md
+145 -1
View File
@@ -116,6 +116,15 @@ EOF'
# --- Kiosk user + seat management ----------------------------------------------
incus exec "$CNAME" -- bash -c 'id kiosk >/dev/null 2>&1 || useradd -m -G video,render,input,audio kiosk'
# ⚠️ Group names are NOT enough. The host (Debian) and the container
# (Ubuntu) allocate dynamic system gids independently, so the container's
# `input` group does not necessarily have the same gid as the group that
# owns /dev/input/* on the host. Bind the kiosk user to the *numeric*
# host gid, whatever it is called inside:
HOST_INPUT_GID="$(stat -c %g /dev/input/event0)"
incus exec "$CNAME" -- usermod -aG "$HOST_INPUT_GID" kiosk
incus exec "$CNAME" -- id kiosk # must list $HOST_INPUT_GID
# In a container seatd must NOT bind the seat to a VT (there is no usable VT;
# it would try to open the host's active tty and hang the compositor forever).
incus exec "$CNAME" -- mkdir -p /etc/systemd/system/seatd.service.d
@@ -245,10 +254,17 @@ it works regardless of which USB port the dongle lands on.
`/etc/udev/rules.d/99-jellyfin-kiosk-recover.rules`:
```
# Logitech Unifying receiver (re)plugged -> recover the jellyfin-client kiosk
# A Logitech receiver was (re)plugged -> recover the jellyfin-client kiosk.
# c52b = Unifying receiver (K400); c539 = Lightspeed receiver (G603).
ACTION=="add", SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", ATTR{idVendor}=="046d", ATTR{idProduct}=="c52b", RUN+="/usr/bin/systemctl --no-block start jellyfin-kiosk-recover.service"
ACTION=="add", SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", ATTR{idVendor}=="046d", ATTR{idProduct}=="c539", RUN+="/usr/bin/systemctl --no-block start jellyfin-kiosk-recover.service"
```
⚠️ The match is per product ID, so **a receiver not listed here will not
auto-recover** — the kiosk must be restarted by hand after plugging it in
(`incus exec jellyfin-client -- systemctl restart jellyfin-kiosk`). Add
the new id here when introducing different input hardware.
`/etc/systemd/system/jellyfin-kiosk-recover.service` — oneshot, so a burst of
udev events during one plug merges into a single restart (natural debounce):
@@ -285,6 +301,134 @@ udevadm control --reload-rules && systemctl daemon-reload
journalctl -t jellyfin-kiosk-recover -f
```
## ⚠️ Never boot nuc with the display active (2026-08-30)
**Symptom:** after a reboot, *no* Incus container starts. `incus list`
answers but `incus start <anything>` hangs forever, `systemctl status
incus` sits in `activating (start-post)`, and LAN DNS is down because
blocky never came up. Nothing in the incus logs explains it.
**Cause — nothing to do with incus.** If the TV/projector is connected
**and powered on** when nuc boots, firmware hands i915 an already-lit
pipe. The driver's state readback then trips a series of warnings and
the probe never completes:
```
drm_WARN_ON(!pll_active) intel_ddi.c:4019 intel_ddi_get_clock
drm_WARN_ON(p0 == 0 || p1 == 0 || p2 == 0) intel_dpll_mgr.c:2878
drm_WARN_ON(pixel_rate == 0) skl_watermark.c:1729
```
(all inside `intel_modeset_setup_hw_state` ← `intel_display_driver_probe_nogem`)
The cascade:
1. i915 probe dies → **`/dev/dri` never appears** (no GPU at all)
2. `snd_hda_intel` waits forever for i915's audio component →
permanent **deferred probe** holding the PCI device lock on `0000:00:1f.3`
3. incusd reads that device's `sriov_numvfs` while enumerating
resources → blocks in **D state** → the daemon never signals ready,
so nothing autostarts and every `incus start` hangs
**Reproduced identically on 6.12.107 and 6.12.105** — it is the display
path, not a kernel regression. Do not waste time pinning kernels.
**Diagnosis, in order:**
```sh
ls /dev/dri/ # empty = i915 probe failed
cat /sys/kernel/debug/devices_deferred # snd_hda_intel entry = the deadlock
ps -eLo pid,tid,stat,wchan:26,comm | awk '$3 ~ /D/' # incusd in sriov_numvfs_show
dmesg -T | grep -E 'drm_WARN_ON|deferred probe pending'
```
**Fix:** disconnect HDMI (or power the display fully off — not standby),
reboot, then **hotplug the cable back in**. Connecting after boot goes
through normal connector detection instead of firmware state readback
and works fine.
In-place recovery is *not* possible: `modprobe -r i915` fails (module in
use by the wedged probe) and `modprobe i915` times out. A reboot is the
only way out.
**After hotplugging, restart the kiosk** — `cage` started with zero
outputs and will not pick the display up on its own:
```sh
incus exec jellyfin-client -- systemctl restart jellyfin-kiosk
cat /sys/class/drm/card0-HDMI-A-2/status # expect: connected
```
Note the HDA controller this wedges is only used for **HDMI audio**,
which this setup does not use — audio goes to the Pioneer USB DAC via
`/etc/asound.conf`. It is pure collateral damage, but it takes the whole
host down with it.
## Input gid mismatch — latent, fix it anyway (2026-08-30)
> ⚠️ **This was not the cause of the 2026-08-30 outage.** That turned out
> to be a flat/switched-off K400 — a G603 on the same port and the same
> `event0` worked immediately. The mismatch below is real and worth
> correcting, but with `LIBSEAT_BACKEND=seatd` it is **seatd (running as
> root) that opens input devices** and passes the fd to cage, so the
> kiosk user's group membership is not on the critical path for input.
> Fix it for the direct-open fallback path, not as a debugging lead.
>
> **Before suspecting software, prove the hardware emits anything:**
> ```sh
> timeout 60 cat /dev/input/eventN | wc -c # press keys; 0 bytes = nothing reached the kernel
> ```
> That one check would have saved an hour.
>
> **Dead K400 batteries are invisible from the host.** This K400 exposes
> no `hidpp_battery_*` node under `/sys/class/power_supply/`, so charge
> cannot be read. Worse, the receiver still lists the keyboard as a paired
> peer (`0003:046D:4024.*` under the `C52B` receiver) whether or not it is
> awake, and `/dev/input/event0` plus a `Logitech K400` entry in
> `/proc/bus/input/devices` are present either way — so every software
> check looks perfectly healthy. **Zero bytes from the raw capture is the
> only signal.** Swapping in a different receiver on the same port is the
> quickest A/B confirmation.
**Symptom (if it ever does bite):** kiosk renders but input does nothing,
with no errors — `WLR_LIBINPUT_NO_DEVICES=1` keeps cage alive rather than
failing loudly.
**Cause:** `/dev/input/*` is `crw-rw---- root:<host input gid>`. The
install script adds `kiosk` to the group *named* `input` inside the
container, but Debian (host) and Ubuntu (container) allocate dynamic
system gids independently:
```
host /dev/input/event0 gid 996 -> "input"
container "input" group gid 995 <- kiosk was here
container gid 996 -> "systemd-timesync"
```
So the kiosk user was in the wrong group and could not open any input
device. The udev side was fine — check it first to rule it out:
`cat /run/udev/data/c13:64` should show `E:ID_INPUT=1` etc.
**Diagnose:**
```sh
stat -c '%n %a %u:%g' /dev/input/event0 # host gid
incus exec jellyfin-client -- id kiosk # does it include that gid?
# open() test — do NOT use `head`/`cat`, reading an event device blocks
# with no pending events and looks like a permission failure:
incus exec jellyfin-client -- su -s /bin/bash kiosk -c 'exec 3< /dev/input/event0 && echo OPEN_OK'
```
**Fix** (persists in the container's `/etc/group`):
```sh
HOST_INPUT_GID="$(stat -c %g /dev/input/event0)"
incus exec jellyfin-client -- usermod -aG "$HOST_INPUT_GID" kiosk
incus exec jellyfin-client -- systemctl restart jellyfin-kiosk
```
Re-check this after any host reinstall — the host's `input` gid is
dynamically allocated and can come back different.
## Troubleshooting
```sh
nuc/jellyfin-server.md
+41 -2
View File
@@ -5,8 +5,10 @@ Jellyfin media **server** in an unprivileged Incus container on `nuc`.
- Image: `images:ubuntu/24.04`, Jellyfin from the official repo (repo.jellyfin.org)
- IP: `192.168.0.5` (LAN bridge) — web UI/API on `http://192.168.0.5:8096`
- iGPU render node (`/dev/dri/renderD128`) passed for QSV/VAAPI hardware transcoding
- Media library: host `/srv/media` (ZFS dataset `usb4t/media`, USB 4 TB)
mounted at `/media` with `shift=true` (needs ZFS ≥ 2.2 for idmapped mounts)
- Media library: host `/srv/media` — since 2026-08-30 an **NFSv4 mount
from `nas`** (`192.168.0.4:/media`, dataset `tank/media`), mounted at
`/media` in the container with **`shift=false`** and `readonly=true`.
See [nas/nas-install.md](../nas/nas-install.md) §8.
- Port 8096 additionally proxied to the host address (`web` proxy device)
## Install script
@@ -77,6 +79,43 @@ incus restart "$CNAME"
echo "Done. Open http://<host-ip>:8096 to run the setup wizard."
```
## Media over NFS (2026-08-30)
The library moved to `nas` when the 4 TB left nuc's USB enclosure. The
container keeps the same path, so everything below still applies — only
the mount underneath `/srv/media` changed.
```sh
# nuc host: /etc/fstab
192.168.0.4:/media /srv/media nfs4 ro,_netdev,soft,timeo=100,retrans=3 0 0
```
`shift=true` **cannot** be used: idmapped mounts are not supported on
NFS (nor CIFS). Per the troubleshooting note below, dropping the shift is
enough for a read-only library — the export uses `all_squash` so files
carry synthetic world-readable ownership:
```sh
incus stop jellyfin-server # shift cannot be hot-applied
incus config device set jellyfin-server media shift=false
incus config device set jellyfin-server media readonly=true
incus start jellyfin-server
```
⚠️ Two traps:
- **Boot ordering.** Add `remote-fs.target` to nuc's
`/etc/systemd/system/incus.service.d/after-zfs.conf`, or the container
starts against an empty mountpoint and Jellyfin shows an empty library
(and may prune the library metadata).
- **`soft` is deliberate.** A hung nas should fail Jellyfin's reads, not
wedge nuc's processes in uninterruptible sleep the way the suspended
`usb4t` pool did ([usb4t-dropouts.md](usb4t-dropouts.md)).
`transmission-bt` is no longer on nuc — it moved to nas and writes to
the dataset locally ([nas/transmission-bt.md](../nas/transmission-bt.md)),
so nuc's mount is read-only and there is exactly one writer.
## First-run configuration
1. Run the setup wizard; add libraries pointing at `/media/...`.
nuc/nuc-install.md
+7
View File
@@ -265,6 +265,13 @@ Both logs rotate monthly (`/etc/logrotate.d/incus-*`).
the install scripts in the per-container docs include it)
- [ ] Host boots to `multi-user.target`, nothing grabs the GPU
(required by the jellyfin-client kiosk)
- [ ] ⚠️ **Reboot with the TV/projector disconnected or powered off.**
Booting with the display active kills the i915 probe, which wedges
`snd_hda_intel` in a deferred probe, which blocks incusd in
`sriov_numvfs_show` — **no container starts at all, including
blocky/DNS**. Hotplug the cable back after boot, then
`incus exec jellyfin-client -- systemctl restart jellyfin-kiosk`.
Full diagnosis: [jellyfin-client.md](jellyfin-client.md)
- [ ] Jellyfin web at `http://192.168.0.5:8096`, kiosk UI on HDMI,
sound on the Pioneer, "Pioneer A-70" visible in Spotify Connect
- [ ] LAN DNS: clients use blocky at `192.168.0.254` (host itself uses