7 Commits
Author SHA1 Message Date
Julien LutranandClaude Opus 5 ceff4ec0d0 doc: record the real ks4 pull bottleneck — source disk, not the link
c8d759c claimed the ~125 Mbit/s seed rate was ks4's OVH uplink. That was
wrong, and measuring it says so: ks4 uploads at 609 Mbit/s, downloads at
900, and nas pulls 670 from OVH's network. WireGuard is not it either —
zero UdpRcvbufErrors, wg-crypt kworkers at ~2%, both hosts ~85% idle.

The limit is `data` sitting on sdb5, one 7200 rpm HGST 6 TB disk: during
the send it does 109 r/s at 14 MB/s with ~131 KB requests and a queue
depth of ~1.0, which is the random-IOPS ceiling of a single HDD reading a
fragmented 1.42 TiB dataset. 14 MB/s is ~112 Mbit/s on the wire, exactly
what we see, while the network sits idle.

Parallelism is the only lever and it works by disk queue depth, not
bandwidth: a second concurrent copy takes sdb from 14 to 21 MB/s and the
request size from 131 KB to 514 KB, lifting the tunnel from 113 to 153
Mbit/s. Recorded, along with the decision to keep the seed sequential —
one-off pass, incremental refreshes after, and incus-copy.sh is shared by
all three legs.

Also corrects the set size to the actual 1.42 TiB (~29 h, not ~31 h).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 14:47:28 +02:00
Julien LutranandClaude Opus 5 c8d759ce9e doc: build the ks4 -> nas backup pull leg
FTTH is up, so the leg nas-seed.md had been holding since 2026-08-30 is
now built: wg-ks4 on nas (10.8.0.22), nas peered on ks4's wireguard
container, incus remote over the tunnel, 05:00 cron, and the first pass
seeding under a systemd-run unit.

Three corrections the runbook needed, all found by running it:

- nas had no wireguard-tools at all. transmission-bt carries its own
  tunnel inside the container, so the host never needed them.
- Every ks4 instance has an instance-level eth0 pinned to incusbr0 with
  a static 192.168.1.x, so each copy failed in under a second with
  "Cannot use manually specified ipv4.address when using unmanaged
  parent bridge". nas now runs a managed incusbr0 on 192.168.1.254/24 —
  deliberately not .1, which must keep resolving over wg-ks4.
- The seed command was missing -p backup, which the doc's own
  verification step already assumed.

Also records the measured rate: ~125 Mbit/s, ks4's OVH uplink rather
than the home downlink, so ~31 h for the first pass — during which the
shared incus-copy lock suppresses the 04:00 nasbackup job.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 13:37:56 +02:00
Julien LutranandClaude Opus 5 a7f1ac691e doc: FTTH migration — gateway is now 192.168.0.1
The Archer C7 at .2 is gone; the FTTH box at .1 is the gateway. Every
static holdout (nas host, nuc host, privoxy, transmission-bt) pointed at
the dead .2 and had no internet — the reported symptom was privoxy.

Also record two things the migration broke that were not obvious:

- DHCP reservations did not carry over. blocky held .254 by reservation
  on the C7; a lease renew on the FTTH box moved it and took LAN DNS
  down. It is static now. jellyfin-* are still DHCP on stale leases.
- The FTTH box advertises native IPv6. transmission-bt's tunnel is
  AllowedIPs = 0.0.0.0/0, so v6 egressed around the kill switch on the
  home address. IPv6 is now disabled in that container.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:08:42 +02:00
Julien LutranandClaude Fable 5 a5f63c7095 nas: schedule container apt upgrades at 06:00
nas ran none: the script was deployed but never scheduled, while nuc
(lower stakes) upgraded nightly. Kept clear of the future 05:00 ks4
pull.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 19:00:00 +02:00
Julien LutranandClaude Fable 5 1721d7b05d docs: schedules live in root crontab, not /etc/cron.d or timers
Standardised 2026-08-31 on regular crontabs: nuc's push moved off its
systemd timer (units kept disabled on disk), and the cron.d files on
nuc and nas were folded into root's crontab. Notes the consequence
accepted for nuc: a night with the box powered off is skipped rather
than caught up after boot.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-31 10:49:29 +02:00
Julien LutranandClaude Opus 5 81d22c5daf nas+nuc: cold boot verified; record what clearing the metadata errors took
Both hosts were powered off and brought back from a cold start, which is
the first real test of everything built on 2026-08-30.

nas: both pools imported from /etc/zfs/zpool.cache, all instances
autostarted, NFS exports republished, 0 failed units. nuc: /dev/dri
present with 0 i915 warnings (booted with the display unplugged), NFS
auto-remounted, VAAPI transcode at 9.5x realtime.

Records the ordering that actually cleared tank's inherited
<metadata>:<0x0> and <0x3d>: a scrub alone found 0 errors and repaired 0B
but left them, and a plain `zpool clear` afterwards did not drop them —
ZFS flushes the persistent error log on a scrub run *after* the clear.
That matters beyond tidiness, because while those entries stand
`zpool status -x` reports the pool unhealthy forever and zpool-health.sh
cannot signal anything new.

Two kiosk corrections, both from observed behaviour:

- the Pioneer DAC being switched off is the most likely cause of
  "video, no sound" — asound.conf pins the ALSA default to it by card
  name, so `default` fails to open outright and mpv falls back to null
  silently. Adds the one-line aplay check.
- hotplugging the display makes cage exit once and Restart=on-failure
  recovers it ~5s later. Do NOT restart it by hand; check
  ActiveEnterTimestamp against the hotplug time first.

Also flags that the OS mirror is still untested with a disk physically
unplugged — it is a guess until then.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 10:16:44 +02:00
Julien LutranandClaude Opus 5 58dcaa5d41 nas: new storage host on A1SAi-2750F — 4 TB off USB onto SATA
Builds the box that ends the usb4t dropouts: the JMicron bridge was the
least reliable device in the setup and it held the intended off-site copy
of ks4. The 4 TB now sits on direct SATA as pool `tank`.

- OS on mdraid RAID1 across two 120 GB SSDs (Intel 330 + Toshiba Q300),
  both ESPs bootable; `incus` ZFS mirror on their tails, ~22% left
  unallocated as over-provisioning
- media at /export/media, exported read-only over NFSv4 to nuc
- transmission-bt moves here (its WireGuard tunnel is in-container, so
  ks4 needed no change) and writes to the dataset locally
- backup pools nucbackup / ks4backup / nasbackup
- monitoring live: msmtp (submission+auth, verified 250), zed with
  NOTIFY_DATA, zpool-health.sh every 15 min, smartd on all three disks

Traps recorded because none of them point at their own cause: booting
with the display active kills the i915 probe and wedges incus; d-i picks
grub-pc vs grub-efi from how the installer booted; `incus storage create`
hangs forever on a mountpoint=none dataset; the BMC is deliberately never
cabled, so there is no out-of-band console.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-30 23:53:30 +02:00