2.0 KiB
name, description, metadata
| name | description | metadata | ||
|---|---|---|---|---|
| container-storage-on-home | Rootful podman and docker storage live on the SATA LVM via fstab bind mounts since 2026-10-29; how the move was done and the two traps it hit |
|
Since 2026-10-29, /var/lib/containers and /var/lib/docker are bind mounts
of /home/var-lib/containers and /home/var-lib/docker (fstab,
x-systemd.requires-mounts-for=/home). The NVMe is kept for models. Paths and
configs are unchanged, so podman, docker and restic see the same paths.
Scripts and logs from the move: ~/podman-move-2026-10-29/ (cutover2.sh,
fixcopy.sh, mounttest.py).
Why: Connor wants the only fast SSD free for LLM weights.
How to apply — two traps, both hit on the first attempt:
- Copied overlay layers do not mount on a different filesystem. Overlay
indexis on by default in this kernel (/sys/module/overlay/parameters/index= Y). Each container's upperdiffroot carriestrusted.overlay.origin, and each layer'swork/indexcarriestrusted.overlay.upper; both are file handles from the old filesystem. Everypodman startfailed withoverlayfs: failed to verify upper root origin. Fix: remove that xattr and deletework/indexafter the final rsync (fixcopy.sh). podman startrun from a transient systemd unit with default KillMode puts conmon in that unit's cgroup, and systemd kills every conmon when the unit ends. This produced the stale-runningstate from CLAUDE.md item 3a for all 51 containers. Always usesystemd-run -p KillMode=process.
Also: mount(8) from util-linux 2.42 fails any overlay with more than 8
lowerdirs, with no kernel message, even on healthy storage. Test overlay mounts
with mount(2) directly (mounttest.py), and run a control against known-good
layers before trusting a failure. And any podman call between the final rsync
and an rsync verify touches storage.lock and db.sql and fails the verify.
Jellyfin must start before Home Assistant; both bind udp/1900 and whichever starts second fails.