Normalise per group, and make the notification quota subject-based

Following Connor's point that corpus prevalence reflects how much an outlet
publishes rather than what deserves a notification: allocation should track
stated weights, not corpus composition in EITHER direction. Penalising a topic
for being prevalent is a bug; rewarding it would just amplify whoever publishes
most. This rules out the "reserve slots proportional to corpus share" model that
the previous commit's findings were pointing toward.

Two fixes:

1. Normalisation moves from per-block to per-GROUP. Taking the max across blocks
   gives a subject one draw per block, so foreign affairs -- six of 23 blocks --
   beat everything else by having more chances. It held 70% of the digest while
   software-systems, a quarter of the corpus, held none. Collapsing to the group
   before normalising prices that in. Digest conflict share 13 -> 9, space 2 ->
   4, and us-national appears at all.

2. The per-group quota now keys on the INTEREST group for both lanes. Importance
   blocks describe a kind of importance, not a subject, so two stories on one
   subject could each take a slot via different importance blocks -- which is
   what happened: both importance alerts were Middle East, one via
   war-escalation and one via economic-break.

Interest alerts went from five-of-five Middle East to Ohio redistricting,
political theory and tech policy.

Also demoted prerelease/nightly build entries. The first alert software-systems
ever won was "Nvim development (prerelease) build", which matches the block
correctly and is worthless as a notification.

Documented that switching normalizer requires re-tuning [routing] thresholds:
they are in z-score units, and leaving them at 2.0 under "top-anchored"
silences the interest lane entirely.
This commit is contained in:
connor committed 2026-09-08 16:19:24 -04:00
1 parent d4631ec13d
commit b852840729
4 files changed
+70 -10

No files matched your search

+9
View File
@@ -50,6 +50,15 @@ syndication_threshold = 0.97
# Compare blocks by z-score WITHIN each block, never by raw cosine: cosine
# magnitude is not comparable across blocks, and a plain max let one block win
# 78% of all items. See CALIBRATION.md.
# "zscore-per-block" or "top-anchored". Both normalise per GROUP (see `group`
# in profile.toml), not per block: taking the max across blocks gives a subject
# one draw per block, so a topic split into five blocks beats one covered by a
# single block purely by having more chances.
#
# NOTE the thresholds in [routing] are in z-score units. "top-anchored" puts
# scores on a different scale (1.0 = that group's 99th percentile), so switching
# normalizer REQUIRES re-tuning them -- verified: leaving them at 2.0 silences
# the interest lane completely.
normalize = "zscore-per-block"
# Veto is not a plain similarity threshold. A story can be genuinely adjacent to