to-do/FINDINGS.md

# Findings

Empirical results that settle open questions in PLAN.md. Each entry records how
it was measured, so it can be re-checked when something changes.

## SFTPGo WebDAV does not honor `If-Match` on `PUT`

PLAN.md section 7 flagged this as "to verify in phase 1". Verified 2026-09-09
against the live endpoint.

```
PUT (create)                        -> 201
HEAD                                -> etag "18d3c65559dd89a43"
PUT with correct If-Match           -> 201
PUT with If-Match "deadbeef0000..." -> 201   <- should have been 412
GET                                 -> body from the bogus-If-Match write
```

The precondition is ignored, not evaluated. **The client must use
HEAD-compare-then-`PUT`**, which is the fallback PLAN.md already anticipates.
The remaining race window is milliseconds wide for a single user with one phone.

## ETag is exactly `hex(mtime_nanos) + hex(size)`

Not a hash. Measured by comparing the WebDAV ETag against `stat` over ssh on the
same file:

```
HEAD etag decoded   1788991618.521424162
disk mtime          1788991618.521424162   identical to the nanosecond
```

Two consequences:

- Rewriting identical content still produces a new ETag. Content equality has to
  be checked by comparing bytes, never by comparing ETags.
- An ETag says only "the file was written at time T and is N bytes". That is
  sufficient for change detection, which is all the sync design asks of it.

## The ETag returned by `PUT` is never the file's actual ETag

Reproduced 3/3 before the clock fix, and 3/3 after it. Do not cache the `PUT`
response ETag as the new base. **`HEAD` after `PUT`** to learn the authoritative
value, or the next sync will diagnose a phantom remote change on every save.

SFTPGo computes the response ETag roughly a millisecond before the file's final
mtime settles, and ETag is derived from mtime, so the two can never agree.

```
after the clock fix:
trial 1: PUT="18d3c711836b5e748"  HEAD="18d3c7118361820b8"   gap 0.646 ms
trial 2: PUT="18d3c711b9297f978"  HEAD="18d3c711b91648318"   gap 1.259 ms
trial 3: PUT="18d3c711cb79bac48"  HEAD="18d3c711cb62efba8"   gap 1.494 ms
```

Worth recording how this was nearly misdiagnosed. Before the NAS clock was
fixed the same gap measured about 113 **seconds**, which looked like a serious
protocol defect. It was the same one-millisecond write-path race the whole time,
with a broken clock amplifying it by five orders of magnitude. Fixing the clock
shrank the symptom by 100000x without changing the underlying cause, and without
changing what the client has to do about it.

## The NAS clock was ~124 seconds fast, now fixed

`/home/connor/docs` on mainframe is `nas.rcjohnstone.com:/docs` over NFSv4. NFS
stamps mtimes with the *server's* clock, so every file written to that tree, by
any means, was landing with an mtime about two minutes in the future.

Isolated to the filesystem, with SFTPGo entirely out of the path:

```
$ ssh mainframe
now:             1788991551.498123146
touch .probe-ssh.md
ssh-touch mtime: 1788991665.036420629    <- +113.5s, written locally over ssh
fs type:         nfs
```

The offset was measured at a stable 124.125s over a 193s window. It was not
drifting at a measurable rate; the apparent growth between two early spot checks
was inside the ssh round-trip noise of those checks.

`nas` is Arch, and had no time daemon at all: `systemd-timesyncd` was installed
but disabled, and nothing else was present. Fixed by installing `chrony` and
enabling it. The shipped Arch `chrony.conf` already carries the two settings
this case needed, so it was left alone rather than rewritten:

- `makestep 1.0 3`, which steps a large initial offset instead of spending 25
  minutes slewing it away.
- `rtcsync`, which disciplines the hardware clock. This mattered here, since the
  RTC was fast by the same amount and would otherwise have reintroduced the
  offset on the next boot.

`systemd-timesyncd` was masked so it cannot later race chronyd.

```
Sep 09 18:18:29 nas chronyd[1588]: System clock was stepped by -123.824615 seconds
```

The step matches the independently measured 124.125s offset to within the
measurement noise, which is the confirmation that the two figures describe the
same thing. Verified afterwards, end to end:

```
mainframe now : 1788992343.649697683
NFS file mtime: 1788992343.949308287
mtime minus clock: +0.300 s   (was +113.5, and the 0.3 is the gap
                               between `date` and `touch` in one command)
```

Once disciplined, chrony measured the hardware clock at **2.643 ppm**, about
0.23 s/day, which is a perfectly good crystal. So this was never a drift problem.
At that rate 124 seconds is roughly a year and a half of accumulation, which is
simply how long the machine had been running with nothing to set its clock.

The NAS was serving NFS but running no containers and no databases, so stepping
the clock backward by two minutes was safe. One harmless after-effect: files
written to that tree before the fix carry mtimes up to ~124s in the future, and
stay that way until they are next rewritten.

This project never depended on the fix. Rollover selects the previous note by
**filename sort**, never by mtime, and the sync design compares ETags for
inequality only, never against a clock. It is recorded because the drift
silently affected everything else on that mount, `rsync --update` and
incremental backups being the obvious victims.

## Reconciliation, phase 0, completed 2026-09-09

| File | Resolution |
|---|---|
| `2024-10-21.md` | conflict copy was a strict superset, taken wholesale |
| `2024-11-12.md` | differed only in one checkbox, kept the `[x]` |
| `2025-01-17.md` | local was the superset, kept as-is |
| `2025-01-16.md` | genuine divergence, union merged |

Mainframe's copy was byte-identical to the local conflict file in both divergent
cases, so each was a two-way merge, not a three-way one.

All four `.sync-conflict-` files deleted. 228 files remain, all `YYYY-MM-DD.md`.
Laptop pushed to mainframe; both sides now hash identically:

```
laptop     228 files  b9a39c9cdf0e0ba49080c6ba03629679
mainframe  228 files  b9a39c9cdf0e0ba49080c6ba03629679
```

Backups taken first: `~/todo-notes-mainframe-backup-20260909.tar.gz` on
mainframe, and a matching archive in the session scratchpad on the laptop.