Both hosts were powered off and brought back from a cold start, which is the first real test of everything built on 2026-08-30. nas: both pools imported from /etc/zfs/zpool.cache, all instances autostarted, NFS exports republished, 0 failed units. nuc: /dev/dri present with 0 i915 warnings (booted with the display unplugged), NFS auto-remounted, VAAPI transcode at 9.5x realtime. Records the ordering that actually cleared tank's inherited <metadata>:<0x0> and <0x3d>: a scrub alone found 0 errors and repaired 0B but left them, and a plain `zpool clear` afterwards did not drop them — ZFS flushes the persistent error log on a scrub run *after* the clear. That matters beyond tidiness, because while those entries stand `zpool status -x` reports the pool unhealthy forever and zpool-health.sh cannot signal anything new. Two kiosk corrections, both from observed behaviour: - the Pioneer DAC being switched off is the most likely cause of "video, no sound" — asound.conf pins the ALSA default to it by card name, so `default` fails to open outright and mpv falls back to null silently. Adds the one-line aplay check. - hotplugging the display makes cage exit once and Restart=on-failure recovers it ~5s later. Do NOT restart it by hand; check ActiveEnterTimestamp against the hotplug time first. Also flags that the OS mirror is still untested with a disk physically unplugged — it is a guess until then. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
21 KiB
jellyfin-client
HTPC kiosk client in a privileged Incus container on nuc:
Jellyfin Media Player fullscreen inside cage (Wayland kiosk compositor)
rendering directly to HDMI via DRM/KMS — no X server or desktop on the host.
Also hosts the Spotify Connect endpoint (go-librespot, see below).
- Image:
images:ubuntu/24.04, IP:192.168.0.6(LAN bridge) - Video: Intel Alder Lake-N iGPU → HDMI (
gpudevice, full card access) - Audio: Pioneer USB audio (
08e4:0176), ALSA cardDevice, set as ALSA default via/etc/asound.conf - Input: Logitech Unifying receiver (K400) via
/dev/inputbind + host udev database bind; FR (AZERTY) keymap - Why privileged: hotplug-following directory bind-mounts of
/dev/sndand/dev/input. Unprivileged alternative: oneunix-chardevice per node (doesn't follow card renumbering on replug).
How the tricky parts work
Container-specific pitfalls that cost debugging time — all handled by the install script:
- seatd must not bind a VT (
SEATD_VTBOUND=0drop-in): with VT binding it tries to open the host's active tty, which isn't in the container, and cage hangs forever ("running",Tasks: 0, black screen). - libinput needs a udev database: it only accepts input devices carrying
ID_INPUT*properties. The container can't run its own udevd (sysfs uevent writes are silently denied by the Incus AppArmor profile — an explicit deny rule thatraw.apparmorcannot override), so the host's/run/udevis bind-mounted read-only to/opt/host-udevand arun-udev.mountunit re-binds it onto/run/udevat boot (a direct bind would be shadowed by systemd's/runtmpfs).WLR_LIBINPUT_NO_DEVICES=1additionally keeps cage alive when no input device is present. - ALSA default must be pinned: mpv opens device
default, which maps to card 0 (Intel HDA, HDMI-only pcm devices 3/7/8/9) → open fails and mpv silently falls back to the null output (video OK, no sound)./etc/asound.confpins the default to the Pioneer by card name. - The kiosk service must NOT use
TTYPath/StandardInput=tty— it hides all cage/JMP errors on an invisible tty. Log to the journal.
Install script
#!/usr/bin/env bash
set -euxo pipefail
CNAME="${CNAME:-jellyfin-client}"
IMAGE="${IMAGE:-images:ubuntu/24.04}"
JMP_VER="${JMP_VER:-1.12.0}" # https://github.com/jellyfin/jellyfin-desktop/releases
incus launch "$IMAGE" "$CNAME" -c security.privileged=true
for i in $(seq 1 30); do
incus exec "$CNAME" -- getent hosts github.com >/dev/null 2>&1 && break
sleep 2
done
incus config set "$CNAME" environment.DEBIAN_FRONTEND=noninteractive
incus exec "$CNAME" -- timedatectl set-timezone Europe/Paris
# --- Host devices -------------------------------------------------------------
# iGPU (card + render nodes). gid=44 = "video" group inside the container.
incus config device add "$CNAME" gpu gpu gid=44
# Sound (Pioneer USB audio -> ALSA card) and input devices (keyboard/remote).
# Directory bind-mounts follow hotplug events, so the USB card can be
# re-plugged without restarting the container. No VT/tty devices needed:
# seatd runs with SEATD_VTBOUND=0 (see below).
#
# The host udev database is bound to /opt/host-udev; a mount unit inside the
# container re-binds it to /run/udev at boot (binding /run/udev directly gets
# shadowed when systemd mounts its own tmpfs on /run). libinput only accepts
# input devices that carry ID_INPUT* properties from a udev database, and the
# container cannot run its own udevd (sysfs uevent writes are blocked by the
# Incus AppArmor profile), so it reads the host's database instead.
RAW_LXC="$(cat <<'EOF'
lxc.cgroup2.devices.allow = c 116:* rwm
lxc.cgroup2.devices.allow = c 13:* rwm
lxc.mount.entry = /dev/snd dev/snd none bind,optional,create=dir
lxc.mount.entry = /dev/input dev/input none bind,optional,create=dir
lxc.mount.entry = /run/udev opt/host-udev none bind,ro,optional,create=dir
EOF
)"
incus config set "$CNAME" raw.lxc="$RAW_LXC"
incus restart "$CNAME" # raw.lxc mount entries apply at container start
sleep 5
# --- Packages ------------------------------------------------------------------
incus exec "$CNAME" -- apt-get update
incus exec "$CNAME" -- apt-get upgrade -y
incus exec "$CNAME" -- apt-get install -y --no-install-recommends software-properties-common curl ca-certificates
incus exec "$CNAME" -- add-apt-repository -y universe
incus exec "$CNAME" -- add-apt-repository -y multiverse
incus exec "$CNAME" -- apt-get install -y --no-install-recommends \
cage seatd dbus dbus-user-session qtwayland5 \
intel-media-va-driver-non-free vainfo mesa-utils-bin \
alsa-utils
# Jellyfin Media Player (repo renamed to jellyfin-desktop upstream)
incus exec "$CNAME" -- bash -c "curl -fLo /tmp/jmp.deb \
https://github.com/jellyfin/jellyfin-desktop/releases/download/v${JMP_VER}/jellyfin-media-player_${JMP_VER}-noble.deb"
incus exec "$CNAME" -- apt-get install -y /tmp/jmp.deb
# --- Default audio output = Pioneer -------------------------------------------
# mpv opens ALSA device "default", which otherwise maps to card 0 (Intel HDA,
# HDMI-only pcm devices 3/7/8/9 -> open fails with ENOENT and mpv silently
# falls back to the null output). Pin the default to the Pioneer by card NAME
# so it survives card renumbering. "!" is required to override the compound
# definition in alsa.conf.
incus exec "$CNAME" -- bash -c 'cat > /etc/asound.conf <<EOF
# Pioneer USB audio (card name "Device") is the default output
defaults.pcm.!card "Device"
defaults.ctl.!card "Device"
EOF'
# --- Kiosk user + seat management ----------------------------------------------
incus exec "$CNAME" -- bash -c 'id kiosk >/dev/null 2>&1 || useradd -m -G video,render,input,audio kiosk'
# ⚠️ Group names are NOT enough. The host (Debian) and the container
# (Ubuntu) allocate dynamic system gids independently, so the container's
# `input` group does not necessarily have the same gid as the group that
# owns /dev/input/* on the host. Bind the kiosk user to the *numeric*
# host gid, whatever it is called inside:
HOST_INPUT_GID="$(stat -c %g /dev/input/event0)"
incus exec "$CNAME" -- usermod -aG "$HOST_INPUT_GID" kiosk
incus exec "$CNAME" -- id kiosk # must list $HOST_INPUT_GID
# In a container seatd must NOT bind the seat to a VT (there is no usable VT;
# it would try to open the host's active tty and hang the compositor forever).
incus exec "$CNAME" -- mkdir -p /etc/systemd/system/seatd.service.d
incus exec "$CNAME" -- bash -c 'cat > /etc/systemd/system/seatd.service.d/novt.conf <<EOF
[Service]
Environment=SEATD_VTBOUND=0
EOF'
incus exec "$CNAME" -- systemctl daemon-reload
incus exec "$CNAME" -- systemctl enable --now seatd
# Re-bind the host udev database onto /run/udev after systemd sets up /run
incus exec "$CNAME" -- bash -c 'cat > /etc/systemd/system/run-udev.mount <<EOF
[Unit]
Description=Host udev database (read-only bind)
Before=jellyfin-kiosk.service
[Mount]
What=/opt/host-udev
Where=/run/udev
Type=none
Options=bind,ro
[Install]
WantedBy=multi-user.target
EOF'
incus exec "$CNAME" -- systemctl enable run-udev.mount
# --- Autostart service -----------------------------------------------------------
incus exec "$CNAME" -- bash -c 'cat > /etc/systemd/system/jellyfin-kiosk.service <<EOF
[Unit]
Description=Jellyfin Media Player (cage Wayland kiosk)
After=seatd.service systemd-user-sessions.service network-online.target
Wants=seatd.service network-online.target
[Service]
User=kiosk
PAMName=login
Environment=QT_QPA_PLATFORM=wayland
Environment=LIBSEAT_BACKEND=seatd
Environment=WLR_LIBINPUT_NO_DEVICES=1
Environment=XKB_DEFAULT_LAYOUT=fr
ExecStart=/usr/bin/cage -d -- /usr/bin/jellyfinmediaplayer --tv --fullscreen
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF'
incus exec "$CNAME" -- systemctl daemon-reload
incus exec "$CNAME" -- systemctl enable jellyfin-kiosk
incus config set "$CNAME" boot.autostart=true
incus restart "$CNAME"
echo "Done. The Jellyfin Media Player UI should appear on the HDMI output."
echo "Check status with: incus exec $CNAME -- systemctl status jellyfin-kiosk"
First-run configuration
- On the TV, connect JMP to the server:
http://192.168.0.5:8096. - Settings → Audio: "Auto" lands on the Pioneer (ALSA default). Enable AC3/DTS passthrough only if the DAC/amp decodes them (the A-70 doesn't).
- Control: K400 keyboard/touchpad, or any Jellyfin app via "Play On" (JMP announces itself as a remote-control target).
Spotify Connect (go-librespot)
Same container, same Pioneer output. Announces itself as "Pioneer A-70"; zeroconf auth (no credentials stored), Premium account required.
#!/usr/bin/env bash
set -euxo pipefail
CNAME="${CNAME:-jellyfin-client}"
GLS_VER="${GLS_VER:-0.7.4}" # https://github.com/devgianlu/go-librespot/releases
DEVICE_NAME="${DEVICE_NAME:-Pioneer A-70}"
incus exec "$CNAME" -- bash -c "
set -e
curl -fsSL -o /tmp/golibrespot.tar.gz https://github.com/devgianlu/go-librespot/releases/download/v${GLS_VER}/go-librespot_linux_x86_64.tar.gz
tar -xzf /tmp/golibrespot.tar.gz -C /tmp
mv /tmp/go-librespot /usr/local/bin/go-librespot
chmod 755 /usr/local/bin/go-librespot
id spotify >/dev/null 2>&1 || useradd -r -m -d /var/lib/go-librespot -G audio spotify
mkdir -p /var/lib/go-librespot/config
cat > /var/lib/go-librespot/config/config.yml <<EOF
device_name: ${DEVICE_NAME}
device_type: speaker
bitrate: 320
EOF
chown -R spotify:spotify /var/lib/go-librespot
"
incus exec "$CNAME" -- bash -c 'cat > /etc/systemd/system/go-librespot.service <<EOF
[Unit]
Description=Spotify Connect client (go-librespot)
After=network-online.target avahi-daemon.service
Wants=network-online.target avahi-daemon.service
[Service]
User=spotify
Group=spotify
ExecStart=/usr/local/bin/go-librespot --config_dir /var/lib/go-librespot/config
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF'
incus exec "$CNAME" -- systemctl daemon-reload
incus exec "$CNAME" -- systemctl enable --now go-librespot
Input hotplug auto-recovery (host side)
cage only enumerates input devices at startup, and udev hotplug events don't
cross into the container (see tricky-parts #2). So re-plugging the Unifying
receiver leaves the kiosk bound to the old, now-dead device node — no
keyboard/mouse until jellyfin-kiosk is restarted. These three files on the
host (nuc, not the container) automate that: the host does receive the
uevent, so a host udev rule fires on the receiver's USB add and restarts the
container's kiosk. Match is by vendor/product (046d:c52b), not sysfs path, so
it works regardless of which USB port the dongle lands on.
/etc/udev/rules.d/99-jellyfin-kiosk-recover.rules:
# A Logitech receiver was (re)plugged -> recover the jellyfin-client kiosk.
# c52b = Unifying receiver (K400); c539 = Lightspeed receiver (G603).
ACTION=="add", SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", ATTR{idVendor}=="046d", ATTR{idProduct}=="c52b", RUN+="/usr/bin/systemctl --no-block start jellyfin-kiosk-recover.service"
ACTION=="add", SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", ATTR{idVendor}=="046d", ATTR{idProduct}=="c539", RUN+="/usr/bin/systemctl --no-block start jellyfin-kiosk-recover.service"
⚠️ The match is per product ID, so a receiver not listed here will not
auto-recover — the kiosk must be restarted by hand after plugging it in
(incus exec jellyfin-client -- systemctl restart jellyfin-kiosk). Add
the new id here when introducing different input hardware.
/etc/systemd/system/jellyfin-kiosk-recover.service — oneshot, so a burst of
udev events during one plug merges into a single restart (natural debounce):
[Unit]
Description=Restart jellyfin-client kiosk after Logitech receiver hotplug
[Service]
Type=oneshot
ExecStart=/usr/local/bin/jellyfin-kiosk-recover.sh
/usr/local/bin/jellyfin-kiosk-recover.sh — waits for the receiver to enumerate
its paired K400 node, then restarts the kiosk only if the container is up (so
host-boot coldplug events are a no-op):
#!/bin/bash
# Restart the jellyfin-client Wayland kiosk after the Logitech Unifying
# receiver is (re)plugged, so cage/libinput re-enumerates the keyboard/mouse.
sleep 6
if ! incus info jellyfin-client 2>/dev/null | grep -qi 'Status:.*RUNNING'; then
exit 0
fi
incus exec jellyfin-client -- systemctl restart jellyfin-kiosk.service
logger -t jellyfin-kiosk-recover "Restarted jellyfin-kiosk after Logitech receiver hotplug"
Apply, then watch it work by replugging the dongle:
chmod +x /usr/local/bin/jellyfin-kiosk-recover.sh
udevadm control --reload-rules && systemctl daemon-reload
journalctl -t jellyfin-kiosk-recover -f
⚠️ Never boot nuc with the display active (2026-08-30)
Symptom: after a reboot, no Incus container starts. incus list
answers but incus start <anything> hangs forever, systemctl status incus sits in activating (start-post), and LAN DNS is down because
blocky never came up. Nothing in the incus logs explains it.
Cause — nothing to do with incus. If the TV/projector is connected and powered on when nuc boots, firmware hands i915 an already-lit pipe. The driver's state readback then trips a series of warnings and the probe never completes:
drm_WARN_ON(!pll_active) intel_ddi.c:4019 intel_ddi_get_clock
drm_WARN_ON(p0 == 0 || p1 == 0 || p2 == 0) intel_dpll_mgr.c:2878
drm_WARN_ON(pixel_rate == 0) skl_watermark.c:1729
(all inside intel_modeset_setup_hw_state ← intel_display_driver_probe_nogem)
The cascade:
- i915 probe dies →
/dev/drinever appears (no GPU at all) snd_hda_intelwaits forever for i915's audio component → permanent deferred probe holding the PCI device lock on0000:00:1f.3- incusd reads that device's
sriov_numvfswhile enumerating resources → blocks in D state → the daemon never signals ready, so nothing autostarts and everyincus starthangs
Reproduced identically on 6.12.107 and 6.12.105 — it is the display path, not a kernel regression. Do not waste time pinning kernels.
Diagnosis, in order:
ls /dev/dri/ # empty = i915 probe failed
cat /sys/kernel/debug/devices_deferred # snd_hda_intel entry = the deadlock
ps -eLo pid,tid,stat,wchan:26,comm | awk '$3 ~ /D/' # incusd in sriov_numvfs_show
dmesg -T | grep -E 'drm_WARN_ON|deferred probe pending'
Fix: disconnect HDMI (or power the display fully off — not standby), reboot, then hotplug the cable back in. Connecting after boot goes through normal connector detection instead of firmware state readback and works fine.
In-place recovery is not possible: modprobe -r i915 fails (module in
use by the wedged probe) and modprobe i915 times out. A reboot is the
only way out.
After hotplugging, restart the kiosk — cage started with zero
outputs and will not pick the display up on its own:
incus exec jellyfin-client -- systemctl restart jellyfin-kiosk
cat /sys/class/drm/card0-HDMI-A-2/status # expect: connected
Note the HDA controller this wedges is only used for HDMI audio,
which this setup does not use — audio goes to the Pioneer USB DAC via
/etc/asound.conf. It is pure collateral damage, but it takes the whole
host down with it.
Input gid mismatch — latent, fix it anyway (2026-08-30)
⚠️ This was not the cause of the 2026-08-30 outage. That turned out to be a flat/switched-off K400 — a G603 on the same port and the same
event0worked immediately. The mismatch below is real and worth correcting, but withLIBSEAT_BACKEND=seatdit is seatd (running as root) that opens input devices and passes the fd to cage, so the kiosk user's group membership is not on the critical path for input. Fix it for the direct-open fallback path, not as a debugging lead.Before suspecting software, prove the hardware emits anything:
timeout 60 cat /dev/input/eventN | wc -c # press keys; 0 bytes = nothing reached the kernelThat one check would have saved an hour.
Dead K400 batteries are invisible from the host. This K400 exposes no
hidpp_battery_*node under/sys/class/power_supply/, so charge cannot be read. Worse, the receiver still lists the keyboard as a paired peer (0003:046D:4024.*under theC52Breceiver) whether or not it is awake, and/dev/input/event0plus aLogitech K400entry in/proc/bus/input/devicesare present either way — so every software check looks perfectly healthy. Zero bytes from the raw capture is the only signal. Swapping in a different receiver on the same port is the quickest A/B confirmation.
Symptom (if it ever does bite): kiosk renders but input does nothing,
with no errors — WLR_LIBINPUT_NO_DEVICES=1 keeps cage alive rather than
failing loudly.
Cause: /dev/input/* is crw-rw---- root:<host input gid>. The
install script adds kiosk to the group named input inside the
container, but Debian (host) and Ubuntu (container) allocate dynamic
system gids independently:
host /dev/input/event0 gid 996 -> "input"
container "input" group gid 995 <- kiosk was here
container gid 996 -> "systemd-timesync"
So the kiosk user was in the wrong group and could not open any input
device. The udev side was fine — check it first to rule it out:
cat /run/udev/data/c13:64 should show E:ID_INPUT=1 etc.
Diagnose:
stat -c '%n %a %u:%g' /dev/input/event0 # host gid
incus exec jellyfin-client -- id kiosk # does it include that gid?
# open() test — do NOT use `head`/`cat`, reading an event device blocks
# with no pending events and looks like a permission failure:
incus exec jellyfin-client -- su -s /bin/bash kiosk -c 'exec 3< /dev/input/event0 && echo OPEN_OK'
Fix (persists in the container's /etc/group):
HOST_INPUT_GID="$(stat -c %g /dev/input/event0)"
incus exec jellyfin-client -- usermod -aG "$HOST_INPUT_GID" kiosk
incus exec jellyfin-client -- systemctl restart jellyfin-kiosk
Re-check this after any host reinstall — the host's input gid is
dynamically allocated and can come back different.
Troubleshooting
incus exec jellyfin-client -- systemctl status seatd jellyfin-kiosk go-librespot
incus exec jellyfin-client -- journalctl -u jellyfin-kiosk -b
incus exec jellyfin-client -- su -s /bin/bash kiosk -c vainfo # GPU/VAAPI
incus exec jellyfin-client -- aplay -l # ALSA cards
incus exec jellyfin-client -- su -s /bin/bash kiosk -c "aplay -D default /usr/share/sounds/alsa/Front_Center.wav"
incus exec jellyfin-client -- udevadm info /dev/input/event0 # udev db visible?
- cage "running" but stuck,
Tasks: 0, black screen → seatd VT binding (see "How the tricky parts work" #1). Note:Tasks: 0alone is normal —PAMName=loginmoves the process into a logind session scope. - "libinput initialization failed, no input devices" → udev db not
visible (#2); check
run-udev.mountis active. - Video OK, no sound; JMP log shows
AO: [null]→ ALSA default broken (#3); check/etc/asound.confand the card name inaplay -l. Most common cause: the Pioneer DAC is simply switched off./etc/asound.confpins the default to it by card name (Device), so with the amp off the name does not exist anddefaultfails to open — mpv then falls back to null silently: picture, no sound, no error. One-line check before blaming anything else:incus exec jellyfin-client -- su -s /bin/bash kiosk -c 'aplay -D default -d 1 /usr/share/sounds/alsa/Front_Center.wav'audio open error: No such device= amp is off. Power it on; no restart needed, JMP opens the device per playback. - Display hotplugged after boot → cage exits once, then recovers by
itself. If the kiosk started with no outputs, plugging the HDMI in
makes cage fail (
Failed with result 'exit-code'); the unit'sRestart=on-failure/RestartSec=5restarts it ~5 s later, this time with the display present. Do not restart it by hand — checksystemctl show jellyfin-kiosk -p ActiveEnterTimestampfirst and only intervene if the timestamp predates the hotplug. Verified 2026-08-31. - Keyboard plugged in after boot isn't seen — the host udev db is live
through the bind, but udev hotplug events don't cross the container's
network namespace, so cage only enumerates at startup. Re-plugging the
Unifying receiver is now handled automatically by the host-side recovery
rule (see "Input hotplug auto-recovery"); manual fallback for other input
hardware:
incus exec jellyfin-client -- systemctl restart jellyfin-kiosk. - cage can't open card0 — something on the host holds DRM master; the
host must boot to
multi-user.targetwith no display manager. - Jellyfin + Spotify at the same time — the Pioneer PCM is exclusive;
the second opener gets "device busy". Add an ALSA
dmixconfig if simultaneous mixing is ever needed. - JMP log:
/home/kiosk/.local/share/jellyfinmediaplayer/logs/jellyfinmediaplayer.log.