connor/to-do
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Give the laptop client the base tracking the phone already has

Browse Source 
Without a record of the last agreement, the client cannot tell "I have
unsaved edits" from "I am simply behind", and reported the second as the
first every time the phone pushed, leaving a .remote- file in the notes
directory each time. It now keeps the pristine text and its ETag under
XDG_STATE_HOME and merges three ways, matching the phone.

Two bugs found while testing that, both of which manufacture conflicts
rather than lose data, and both of which were invisible until a real edit
happened on each device at once.

diff3 was the wrong merge tool. It reports a conflict even when both sides
make the identical change, and ticking the same box on both devices is an
ordinary thing to do. git merge-file handles that cleanly and agrees with
the Android merge about what counts as a genuine conflict.

The merge result was not being recorded as the new base, so one sync later
the base was a version behind and an already merged edit looked like a
fresh change from both sides. Every point where the two are known to agree
now records that agreement.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PU5ZFfQFtDTFqqhvMTWdGH
This commit is contained in:
Connor Johnstone
2026-09-09 20:50:37 -04:00
co-authored by Claude Opus 5
parent dd23ca67cd
commit 2bf910757a
2 changed files with 122 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files
FINDINGS.md
+43
View File
@@ -309,6 +309,49 @@ tap in that sample happened to be in the unscrolled top of the document, where
the two coordinate systems coincide. The evidence was consistent with both
explanations, and only looked decisive.
## diff3 calls an identical change on both sides a conflict
The laptop client needed a three way merge, and `diff3 -m mine base theirs` is
the obvious Unix answer. It is the wrong one here.
```
base: - [ ] a mine: - [x] a theirs: - [x] a
$ diff3 -m mine base theirs
<<<<<<< base
- [ ] a
=======
- [x] a
>>>>>>> theirs
exit=1
```
Both sides made the *same* edit and it still reports a conflict. Ticking the
same box on the phone and the laptop is an entirely ordinary thing to do here,
so this would have manufactured conflicts during normal use.
`git merge-file -p mine base theirs` resolves that case cleanly, and resolves
edits to different boxes cleanly, and still conflicts on genuinely competing
edits to adjacent lines. That last behaviour matches what the Android merge
does, so the two clients agree about what a conflict is, which matters more than
either being clever.
## A merge is only as good as the base it merges from
The laptop client merged correctly and then failed to record the result as the
new base. One sync later the base was a version behind, and an edit that had
already been merged in looked like a fresh change from both sides, which
produced a conflict out of nothing.
The symptom appeared far from the cause: the false conflict was reported on a
line neither side had touched in that round. The rule the code now follows is
that any point where local and server are known to agree records that agreement,
whether the agreement came from adopting, from merging to something identical to
the server, or from a successful push.
Worth stating plainly because it applies to both clients: a three way merge is
only correct if the base really is the last common ancestor. A stale base does
not fail loudly, it silently invents conflicts.
## Reconciliation, phase 0, completed 2026-09-09
| File | Resolution |
client/todo
+79 -49
View File
@@ -1,14 +1,20 @@
#!/bin/zsh
# Open today's note, syncing it through WebDAV around the edit.
# Open today's note, reconciling it with the server around the edit.
#
# The server rolls the note forward at 00:01, so normally this just fetches a
# file that already exists. Everything here is arranged so that losing the
# network degrades to editing the local copy, never to losing an edit.
# The server rolls the note forward at 00:01, so normally this fetches a file
# that already exists. Everything is arranged so that losing the network
# degrades to editing the local copy, never to losing an edit.
#
# Alongside the note, a pristine copy of the text last agreed with the server is
# kept, with its ETag. Without it there is no way to tell "I have unsaved edits"
# from "I am simply behind", and the second case gets misreported as the first
# every time the phone pushes.
emulate -L zsh
set -o errexit -o nounset -o pipefail
NOTES=${TODO_NOTES:-$HOME/docs/notes/todo}
STATE=${TODO_STATE:-${XDG_STATE_HOME:-$HOME/.local/state}/todo}
REMOTE=${TODO_REMOTE:-https://files.rcjohnstone.com/docs/notes/todo}
DAV_USER=${TODO_USER:-connor}
RBW_ENTRY=${TODO_RBW_ENTRY:-SFTPGo}
@@ -16,16 +22,18 @@ PRUNE=${TODO_PRUNE:-$HOME/.local/lib/todo/prune.py}
# zshenv exports EDITOR on every invocation, including this script's own, so
# `EDITOR=x todo` cannot work. TODO_EDITOR is the override that does.
EDIT=${TODO_EDITOR:-${EDITOR:-nvim}}
AGENT=todo-cli/1.0
AGENT=todo-cli/2.0
today=$(date --iso-8601)
current=$NOTES/$today.md
base=$STATE/base/$today.md
base_etag=$STATE/base/$today.etag
mkdir -p $NOTES
mkdir -p $NOTES $STATE/base
work=$(mktemp -d); trap 'rm -rf $work' EXIT
note() { print -u2 -- "todo: $*" }
die() { print -u2 -- "todo: $*"; exit 1 }
note() { print -u2 -- "todo: $*" }
die() { print -u2 -- "todo: $*"; exit 1 }
# Password stays in a variable and reaches curl over stdin. Passing it as -u
# would put it in argv, where any other user's `ps` can read it.
@@ -42,26 +50,60 @@ dav() {
etag_of() { grep -i '^etag:' $1 2>/dev/null | tr -d '\r' | awk '{print $2}' }
adopt() { # text-file etag
[[ $1 == $current ]] || cp $1 $current
cp $1 $base
print -r -- ${2:-} > $base_etag
}
set_aside() { # text-file -> prints the filename used
local name=$today.sync-conflict-$(date +%Y%m%d-%H%M%S)-${HOST:u}.md
cp $1 $NOTES/$name
dav -X PUT --data-binary @$1 -o /dev/null $REMOTE/$name >/dev/null 2>&1 || true
print -r -- $name
}
# ---------------------------------------------------------------- pull
base_etag=""
have_remote=0
remote_hash=""
if [[ -z $dav_pass ]]; then
note "no credential from rbw, working offline"
else
code=$(dav -o $work/pull -D $work/pull.h -w '%{http_code}' \
$REMOTE/$today.md) || code=000
code=$(dav -o $work/theirs -D $work/h -w '%{http_code}' $REMOTE/$today.md) || code=000
case $code in
200)
base_etag=$(etag_of $work/pull.h)
have_remote=1
if [[ -e $current ]] && ! cmp -s $current $work/pull; then
note "local and remote both changed since last sync, keeping local"
note "remote copy saved alongside it for comparison"
cp $work/pull $NOTES/$today.remote-$(date +%H%M%S).md
their_etag=$(etag_of $work/h)
remote_hash=$(md5sum < $work/theirs)
if [[ ! -e $current ]]; then
adopt $work/theirs $their_etag
elif [[ ! -e $base ]]; then
# No record of a previous agreement, so nothing to merge from.
if cmp -s $current $work/theirs; then
adopt $work/theirs $their_etag
else
note "no sync history, yours saved as $(set_aside $current)"
adopt $work/theirs $their_etag
fi
elif [[ -e $base_etag && -n $their_etag && $their_etag == $(<$base_etag) ]]; then
: # server has not moved since the last agreement
elif git merge-file -p $current $base $work/theirs > $work/merged 2>/dev/null; then
# git rather than diff3. diff3 calls it a conflict even when
# both sides made the same change, and both devices ticking the
# same box is an ordinary thing to happen here.
cp $work/merged $current
if cmp -s $current $work/theirs; then
# Nothing local to contribute, so this is the new agreement.
adopt $work/theirs $their_etag
else
note "merged in changes from the server"
fi
else
cp $work/pull $current
# Both sides changed the same lines. The server's copy becomes
# the note and the local one goes beside it, which is the
# convention already in this directory.
note "conflict, yours saved as $(set_aside $current)"
adopt $work/theirs $their_etag
fi
;;
404)
@@ -74,12 +116,8 @@ else
# earn a four hour ban that no whitelist covers.
die "credentials rejected. fix the $RBW_ENTRY entry, do not re-run blindly"
;;
000)
note "server unreachable, working offline"
;;
*)
note "server returned $code, working offline"
;;
000) note "server unreachable, working offline" ;;
*) note "server returned $code, working offline" ;;
esac
fi
@@ -88,7 +126,7 @@ fi
if [[ ! -e $current ]]; then
# Match the date glob exactly. A bare `ls | tail -1` can select a
# .sync-conflict- sibling, which is how divergent notes spread before.
previous=(${~NOTES}/[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9].md(N))
previous=($NOTES/[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9].md(N))
if (( ${#previous} )); then
src=${previous[-1]}
if [[ -x $PRUNE ]]; then
@@ -97,8 +135,8 @@ if [[ ! -e $current ]]; then
note "prune not found at $PRUNE, carrying the note forward unpruned"
cp $src $work/new
fi
# Only line 1. A global replace would rewrite any date in the body,
# so "meeting moved to 2026-09-08" would silently become today.
# Only line 1. A global replace would rewrite any date in the body, so
# "meeting moved to 2026-09-08" would silently become today.
sed -i "1s/.*/# $today/" $work/new
cp $work/new $current
note "created $today.md from ${src:t}"
@@ -108,34 +146,26 @@ if [[ ! -e $current ]]; then
fi
fi
before=$(md5sum < $current)
# ---------------------------------------------------------------- edit
$EDIT $current
# ---------------------------------------------------------------- push
[[ $(md5sum < $current) == $before ]] && { note "unchanged, nothing to push"; exit 0 }
[[ $(md5sum < $current) == $remote_hash ]] && { note "in step with the server"; exit 0 }
[[ -n $dav_pass ]] || { note "no credential, leaving changes local"; exit 0 }
# SFTPGo returns ETags but ignores If-Match on PUT, so the precondition has to
# be checked here instead. The race window is milliseconds wide for one person.
if (( have_remote )); then
dav -I -D $work/head.h -o /dev/null $REMOTE/$today.md >/dev/null 2>&1 || true
now_etag=$(etag_of $work/head.h)
if [[ -n $now_etag && -n $base_etag && $now_etag != $base_etag ]]; then
sibling=$NOTES/$today.sync-conflict-$(date +%Y%m%d-%H%M%S)-${HOST:u}.md
cp $current $sibling
die "remote changed while you were editing. yours saved as ${sibling:t}, nothing overwritten"
fi
fi
code=$(dav -X PUT --data-binary @$current -o /dev/null -w '%{http_code}' \
$REMOTE/$today.md) || code=000
code=$(dav -X PUT --data-binary @$current -o /dev/null -w '%{http_code}' $REMOTE/$today.md) ||
code=000
case $code in
200|201|204) note "pushed $today.md" ;;
401) die "credentials rejected on push. changes are safe locally" ;;
000) note "push failed, server unreachable. changes are safe locally" ;;
*) note "push returned $code. changes are safe locally" ;;
200|201|204)
# The ETag a PUT returns is not the file's own, so it is read back
# rather than believed. See FINDINGS.md.
dav -I -D $work/ph -o /dev/null $REMOTE/$today.md >/dev/null 2>&1 || true
adopt $current "$(etag_of $work/ph)"
note "pushed $today.md"
;;
401) die "credentials rejected on push. changes are safe locally" ;;
000) note "push failed, server unreachable. changes are safe locally" ;;
*) note "push returned $code. changes are safe locally" ;;
esac