dotfiles/common/claude/memory/-home-connor/nas-dual-homed-wifi-path.md

name, description, metadata

name	description	metadata
nas-dual-homed-wifi-path	NAS is dual-homed on 10.0.0.0/24 and all its traffic silently used WiFi instead of the 2.5GbE wire; this was the real cause of slow NFS	type project

The NAS has both interfaces live on the SAME subnet:

enp3s0  10.0.0.3/24   wired, 2500 Mb/s full duplex
wlo1    10.0.0.42/24  WiFi, about -61 dBm

Two independent faults made all NAS traffic take the WiFi path:

1. Forward (mainframe -> NAS). With arp_ignore=0 the NAS answered ARP for 10.0.0.3 from wlo1, so mainframe cached 10.0.0.3 -> f8:3d:c6:2f:b6:68, the WiFi MAC. Packets addressed to the wired IP were delivered over WiFi. Wired MAC is a0:ad:9f:88:d5:fa — compare against ip neigh show 10.0.0.3.
2. Return (NAS -> mainframe). NetworkManager installed NO 10.0.0.0/24 route for enp3s0 at all, only dev wlo1 ... metric 600, so every reply went out over WiFi regardless of how the request arrived.

Fixing only one of the two barely helps; both must be right. Measured on 2026-10-02, small-file creates over NFS:

WiFi path (both wrong)     456 ms/file,  ~4 MB/s
ARP fixed only             330 ms/file
both directions wired       60 ms/file,  ~38 MB/s

Runtime fix applied (NOT persisted — lost on NAS reboot or NM restart): sysctl -w net.ipv4.conf.all.arp_ignore=1 net.ipv4.conf.all.arp_announce=2 and ip route replace 10.0.0.0/24 dev enp3s0 proto kernel scope link src 10.0.0.3 metric 100. Persisting it means an NM ipv4.route-metric on the wired profile below wifi's 600, plus the sysctls in /etc/sysctl.d/. The cleanest answer is probably to stop the stationary NAS joining WiFi at all.

Even fully wired, NFS is ~60 ms per small-file create against an SSD's ~0.01 ms. The array is LVM RAID1 over a Seagate ST8000DM004, which is SMR, so small synchronous writes stay slow; local dd on the NAS gets 169 MB/s sequential. Do not put build trees or git working copies on it.

CORRECTION to nfs-pnfs-scsi-layout-wedge: the pNFS SCSI fence failure was real and did wedge the mount, but it was not the cause of the chronic slowness — it was most likely a consequence of timeouts over the flaky WiFi link. The original reasoning was unsound because the pNFS fault was found first and fully explained the symptoms in front of me, so I stopped looking; the giveaway I initially read past was ping to the NAS swinging 0.5 ms to 180 ms, which no amount of NFS state can cause.