9 Commits
Author SHA1 Message Date
Julien LutranandClaude Opus 5 ceff4ec0d0 doc: record the real ks4 pull bottleneck — source disk, not the link
c8d759c claimed the ~125 Mbit/s seed rate was ks4's OVH uplink. That was
wrong, and measuring it says so: ks4 uploads at 609 Mbit/s, downloads at
900, and nas pulls 670 from OVH's network. WireGuard is not it either —
zero UdpRcvbufErrors, wg-crypt kworkers at ~2%, both hosts ~85% idle.

The limit is `data` sitting on sdb5, one 7200 rpm HGST 6 TB disk: during
the send it does 109 r/s at 14 MB/s with ~131 KB requests and a queue
depth of ~1.0, which is the random-IOPS ceiling of a single HDD reading a
fragmented 1.42 TiB dataset. 14 MB/s is ~112 Mbit/s on the wire, exactly
what we see, while the network sits idle.

Parallelism is the only lever and it works by disk queue depth, not
bandwidth: a second concurrent copy takes sdb from 14 to 21 MB/s and the
request size from 131 KB to 514 KB, lifting the tunnel from 113 to 153
Mbit/s. Recorded, along with the decision to keep the seed sequential —
one-off pass, incremental refreshes after, and incus-copy.sh is shared by
all three legs.

Also corrects the set size to the actual 1.42 TiB (~29 h, not ~31 h).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 14:47:28 +02:00
Julien LutranandClaude Opus 5 c8d759ce9e doc: build the ks4 -> nas backup pull leg
FTTH is up, so the leg nas-seed.md had been holding since 2026-08-30 is
now built: wg-ks4 on nas (10.8.0.22), nas peered on ks4's wireguard
container, incus remote over the tunnel, 05:00 cron, and the first pass
seeding under a systemd-run unit.

Three corrections the runbook needed, all found by running it:

- nas had no wireguard-tools at all. transmission-bt carries its own
  tunnel inside the container, so the host never needed them.
- Every ks4 instance has an instance-level eth0 pinned to incusbr0 with
  a static 192.168.1.x, so each copy failed in under a second with
  "Cannot use manually specified ipv4.address when using unmanaged
  parent bridge". nas now runs a managed incusbr0 on 192.168.1.254/24 —
  deliberately not .1, which must keep resolving over wg-ks4.
- The seed command was missing -p backup, which the doc's own
  verification step already assumed.

Also records the measured rate: ~125 Mbit/s, ks4's OVH uplink rather
than the home downlink, so ~31 h for the first pass — during which the
shared incus-copy lock suppresses the 04:00 nasbackup job.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 13:37:56 +02:00
Julien LutranandClaude Opus 5 7e6b8f2848 doc: make every LAN instance static, record the FTTH fallout
Follow-up to a7f1ac6. The Archer C7's DHCP reservations did not carry
over to the FTTH box, so anything still on DHCP was one lease renew away
from moving — as blocky already demonstrated by taking LAN DNS with it.
jellyfin-server and jellyfin-client are now static too; homeassistant
stays dynamic on purpose (it never had a reservation).

Records three things that cost time to find:

- jellyfin-client can never use netplan: the kiosk raw.lxc bind-mounts
  the host /run/udev read-only, so netplan generate fails and config
  silently does not regenerate at boot. It uses systemd-networkd now.
- LAPTOP719974 and patate also lost their reserved addresses.
- ks4 cannot serve as an IPv6 exit: it has a global v6 address and a
  default v6 route but no working v6 egress, so extending the torrent
  tunnel to ::/0 is not an option. IPv6 stays disabled in the container.

Also notes the C7 + LTE box kept as fallback: only the gateway differs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:33:17 +02:00
Julien LutranandClaude Opus 5 a7f1ac691e doc: FTTH migration — gateway is now 192.168.0.1
The Archer C7 at .2 is gone; the FTTH box at .1 is the gateway. Every
static holdout (nas host, nuc host, privoxy, transmission-bt) pointed at
the dead .2 and had no internet — the reported symptom was privoxy.

Also record two things the migration broke that were not obvious:

- DHCP reservations did not carry over. blocky held .254 by reservation
  on the C7; a lease renew on the FTTH box moved it and took LAN DNS
  down. It is static now. jellyfin-* are still DHCP on stale leases.
- The FTTH box advertises native IPv6. transmission-bt's tunnel is
  AllowedIPs = 0.0.0.0/0, so v6 egressed around the kill switch on the
  home address. IPv6 is now disabled in that container.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:08:42 +02:00
Julien LutranandClaude Fable 5 a5f63c7095 nas: schedule container apt upgrades at 06:00
nas ran none: the script was deployed but never scheduled, while nuc
(lower stakes) upgraded nightly. Kept clear of the future 05:00 ks4
pull.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 19:00:00 +02:00
Julien LutranandClaude Opus 5 9468f30f92 jellyfin: scan on download completion — inotify cannot work over NFS
Finished downloads stopped appearing in Jellyfin after the 2026-08-30
storage move, and everything looked healthy: the file was on nas, visible
through NFS inside the container, and /media/downloads is a configured
library. The cause is that Jellyfin watches libraries with inotify, which
only reports changes made through the local mount — transmission now
writes on nas while jellyfin-server reads over NFS on nuc, so no event
ever reaches it. EnableRealtimeMonitor is true and SupportsLibraryMonitor
reports true, which is why it looks fine. Previously both shared one local
dataset on nuc and it worked.

transmission now calls Jellyfin's /Library/Refresh via script-torrent-done.
The hook always exits 0 and never blocks (transmission runs it
synchronously; a hanging hook stalls the daemon), and both the success and
missing-key paths are tested. nuc is usually powered off, so a failed
request is expected and logged rather than treated as an error — the
scheduled scan catches up.

Also records that nuc's mount is read-only, so reorganising downloads into
movies/tv-shows must now happen on nas, and the ordered checklist for
"my download is not in Jellyfin" — the first three checks all passed when
this was hit, which is what made it confusing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 20:21:36 +02:00
Julien LutranandClaude Fable 5 1721d7b05d docs: schedules live in root crontab, not /etc/cron.d or timers
Standardised 2026-08-31 on regular crontabs: nuc's push moved off its
systemd timer (units kept disabled on disk), and the cron.d files on
nuc and nas were folded into root's crontab. Notes the consequence
accepted for nuc: a night with the box powered off is skipped rather
than caught up after boot.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-31 10:49:29 +02:00
Julien LutranandClaude Opus 5 81d22c5daf nas+nuc: cold boot verified; record what clearing the metadata errors took
Both hosts were powered off and brought back from a cold start, which is
the first real test of everything built on 2026-08-30.

nas: both pools imported from /etc/zfs/zpool.cache, all instances
autostarted, NFS exports republished, 0 failed units. nuc: /dev/dri
present with 0 i915 warnings (booted with the display unplugged), NFS
auto-remounted, VAAPI transcode at 9.5x realtime.

Records the ordering that actually cleared tank's inherited
<metadata>:<0x0> and <0x3d>: a scrub alone found 0 errors and repaired 0B
but left them, and a plain `zpool clear` afterwards did not drop them —
ZFS flushes the persistent error log on a scrub run *after* the clear.
That matters beyond tidiness, because while those entries stand
`zpool status -x` reports the pool unhealthy forever and zpool-health.sh
cannot signal anything new.

Two kiosk corrections, both from observed behaviour:

- the Pioneer DAC being switched off is the most likely cause of
  "video, no sound" — asound.conf pins the ALSA default to it by card
  name, so `default` fails to open outright and mpv falls back to null
  silently. Adds the one-line aplay check.
- hotplugging the display makes cage exit once and Restart=on-failure
  recovers it ~5s later. Do NOT restart it by hand; check
  ActiveEnterTimestamp against the hotplug time first.

Also flags that the OS mirror is still untested with a disk physically
unplugged — it is a guess until then.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 10:16:44 +02:00
Julien LutranandClaude Opus 5 58dcaa5d41 nas: new storage host on A1SAi-2750F — 4 TB off USB onto SATA
Builds the box that ends the usb4t dropouts: the JMicron bridge was the
least reliable device in the setup and it held the intended off-site copy
of ks4. The 4 TB now sits on direct SATA as pool `tank`.

- OS on mdraid RAID1 across two 120 GB SSDs (Intel 330 + Toshiba Q300),
  both ESPs bootable; `incus` ZFS mirror on their tails, ~22% left
  unallocated as over-provisioning
- media at /export/media, exported read-only over NFSv4 to nuc
- transmission-bt moves here (its WireGuard tunnel is in-container, so
  ks4 needed no change) and writes to the dataset locally
- backup pools nucbackup / ks4backup / nasbackup
- monitoring live: msmtp (submission+auth, verified 250), zed with
  NOTIFY_DATA, zpool-health.sh every 15 min, smartd on all three disks

Traps recorded because none of them point at their own cause: booting
with the display active kills the i915 probe and wedges incus; d-i picks
grub-pc vs grub-efi from how the installer booted; `incus storage create`
hangs forever on a mountpoint=none dataset; the BMC is deliberately never
cabled, so there is no out-of-band console.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-30 23:53:30 +02:00