Connor pointed out he may use the site from a laptop away from home. The gated
/ui/feedback/ path was worse than just inconvenient: fetch() follows redirects,
so an expired Authelia session would go 302 -> login page -> HTTP 200, and the
UI would mark the card labelled having recorded nothing. Silent data loss on the
one signal the system learns from.
Both surfaces now post to the same HMAC-signed public endpoint, with
`redirect: 'error'` so a redirect fails loudly instead of looking like success.
The gated route is gone -- one mechanism, not two.
/config shows what every block actually matches right now, and a box to try a
filter before saving it: describe what you want caught, see what it catches and
what sits just below the line, move the threshold, then save it as a veto or a
demote. That boundary view is the point -- a crypto filter at 0.55 catches 131
clusters including "Security updates for Monday", and at 0.62 catches 4, which
you can only know by looking.
Saves use tomlkit rather than tomllib plus a dump, because profile.toml carries
the reasoning behind every constant and a comment-stripping round-trip would
quietly destroy the record of why each number is what it is. Verified: 97
top-level comment lines in, 97 out.
The config mount had to stop being :ro, and ~/config/news2 is now owned
10001:1000 group-writable so the container can save while hand-editing still
works. `:U` would have fixed the container and locked the user out.
Pipeline: fetch -> normalise -> embed (CPU) -> cluster. `news poll` runs a
cycle; `news sources` and `news clusters` inspect the result. Replay and the
live service share run_cycle so tuning and production cannot diverge -- the old
system's replay harness hardcoded default params and could never sweep the
thresholds it existed to tune.
Calibrated, not guessed:
- Cluster threshold 0.82, from reading real output. 0.70 merged "Solheim Cup
capsules" with "FIBA Women's World Cup Glance"; 0.82 gives clean groups (76
items on the AfD Saxony-Anhalt election, 27 on the Nepal tunnel rescue). The
old system produced 515 clusters of which 514 were singletons.
- Corroboration counts DISTINCT buckets, not feeds, so ten outlets reprinting
one wire story is one confirmation rather than ten.
Removed after measurement: a SimHash near-duplicate stage. It gave +1.2 bits of
separation on titles and -0.2 with summaries -- news duplicates are paraphrases,
which is the same lexical trap that sank the old Jaccard clustering.
Sources: all 39 fetched against the live web, not assumed. Dropped CBC (every
feed URL times out from this host) and Tech Policy Press (serves HTML at every
documented feed path). Fixed WVXU (index.rss is an empty stub; local-news.rss
works). Reddit moved to .rss because .json now 403s without OAuth, and its
intervals raised because it 429s after three quick requests. Deutsche Welle is
back: it ships RSS 1.0/RDF, which feedparser reads and the old hand-rolled
quick-xml parser could not.
Ground-up replacement for news_triage, whose signal quality failed because it
had no representation of what its user likes -- three keywords fed to BM25 plus
a two-term substring blocklist.
profile/ holds the taste model as prose and TOML: 11 interest blocks, 3 veto
blocks, 5 importance blocks, each a set of probe sentences to be embedded.
The [scoring] constants are fitted, not guessed. CALIBRATION.md records the
measurements against all 3,885 items in the v0.2 database and 40 real feedback
labels, including two findings that changed the design:
- compare blocks by z-score within block, never raw cosine; raw cosine let
geopolitics win 78% of items and buried known-liked items at median rank 1246
of 3885 (293 after normalizing)
- veto needs a margin over the interest signal, not a threshold; a plain 0.55
cutoff falsely killed "Belgian mayor blocks club from hosting Israeli opponent
in Europa League", which the old substring blocklist also killed
connor