Unplugging the NAS for science
⚠ This is not a live capture: it is a condensed reconstruction, assembled after the fact from the session's real transcript. The prompts and stop messages are those of the actual session; timing is compressed and the long monitoring loops are cut. Hostnames sanitized.
k3s workloads left Running with their NFS storage dead underneath — zombies
nothing was reporting. The session adds liveness probes everywhere, then
verifies them the only honest way: by pulling the NAS’s plug. Three times.
What makes the recording interesting is the progression: the first cut shows
three probes still lying (a /ready served from memory, ls answered by the
NFS attribute cache). The second one wakes a bug the old configuration had
been masking — Loki’s WAL crash loop, regenerated on every restart by the NFS
silly-rename. And in the middle of the repair, the permission classifier
blocks the mv on the data volume, during the outage: the guard-rail doing
exactly its job, at the exact moment you wish it would keep quiet.
The third cut finds nothing. That was the goal.