Stop one subject taking every notification slot

Connor noticed 5 of 5 alerts were Middle East. Measuring showed that was not
proportionate: Middle East blocks are ~11% of the eligible corpus while
software-systems alone is 25%, yet Middle East took 35% of the top 20 and
software took none.

Two mechanisms, both measured:

1. Five conflict-* blocks could each claim their own alert slot. Interest blocks
   now carry a `group`, and routing enforces max_alerts_per_group_per_day, so
   related blocks share one quota. This alone moved the mix from 5/5 Middle East
   to three distinct groups.

2. Z-scoring subtracts each block's MEAN, which silently encodes how much of the
   corpus that block matches. software-systems has the HIGHEST mean raw
   similarity of any block (0.537) and the LOWEST max z (2.51);
   conflict-mideast-israel has a lower mean (0.460) and reaches 3.47. A topic is
   penalised for being well covered in your own feed list.

Added `normalize = "top-anchored"` as an alternative that rescales each block by
its own upper tail instead of its mean, leaving `weight` to express preference
rather than corpus composition.

Also ruled out a plausible-sounding cause rather than assuming it: HN and
Lobsters items are title-only and much shorter than news articles, so terse
titles seemed likely to match prose probes poorly. Measured the opposite --
title-only HN items score 0.618 against the software block, ABOVE Phoronix/LWN
with summaries (0.610) and Ars Technica (0.540). Short text is not the problem;
prevalence is.
This commit is contained in:
connor committed 2026-09-08 16:17:13 -04:00
1 parent 23966a2d63
commit d4631ec13d
6 files changed
+95 -10

No files matched your search

+3
View File
@@ -72,6 +72,9 @@ corroboration_depth_weight = 0.2
# Daily budgets. Deliberately small: the point of the system is a handful of
# well-chosen interruptions, not a feed.
interest_per_day = 3
# Max alerts per quota group per day (see `group` in profile.toml). Stops one
# subject taking every slot.
max_alerts_per_group_per_day = 1
important_per_day = 2
# Hard cap on waking the reader during quiet hours, inside important_per_day.
important_per_night = 1
+21 -5
View File
@@ -10,6 +10,10 @@ schema_version = 2
# =========================================================== interests =======
# `group` shares one notification quota across related blocks. Without it the
# five conflict-* blocks could each claim an alert slot and fill a day with the
# same subject -- which is what happened: 5 of 5 alerts were Middle East.
#
# --- foreign affairs, split from armed conflict ------------------------------
# Connor: "my actual war interest is low, but unfortunately all wars are
# important." The click history looked war-heavy only because world-news feeds
@@ -30,7 +34,8 @@ probes = [
# The A-E ordering Connor gave, encoded as weights. Even the lowest band still
# reaches him -- but through importance, on major developments only.
[[interest]]
id = "conflict-anti-imperial" # (A) top of his ordering
id = "conflict-anti-imperial"
group = "conflict" # (A) top of his ordering
weight = 0.9
probes = [
"An armed liberation movement fights a colonial or occupying power for self-determination.",
@@ -39,7 +44,8 @@ probes = [
]
[[interest]]
id = "conflict-us-involved" # (B)
id = "conflict-us-involved"
group = "conflict" # (B)
weight = 0.85
probes = [
"US forces carry out strikes and the administration defends the legal basis for the war.",
@@ -47,7 +53,8 @@ probes = [
]
[[interest]]
id = "conflict-mideast-israel" # (C)
id = "conflict-mideast-israel"
group = "conflict" # (C)
weight = 0.8
probes = [
"Israeli forces strike targets in Gaza, Lebanon or Syria amid continuing escalation.",
@@ -56,7 +63,8 @@ probes = [
]
[[interest]]
id = "conflict-undercovered" # (D)
id = "conflict-undercovered"
group = "conflict" # (D)
weight = 0.75
probes = [
"Fighting intensifies in Sudan as famine spreads and aid convoys are blocked.",
@@ -65,7 +73,8 @@ probes = [
]
[[interest]]
id = "conflict-general" # (E) -- low interest, still important
id = "conflict-general"
group = "conflict" # (E) -- low interest, still important
weight = 0.45
probes = [
"Front-line fighting continues as both sides exchange drone and artillery strikes.",
@@ -137,6 +146,7 @@ probes = [
[[interest]]
id = "labor-inequality"
group = "left-politics"
weight = 0.85
probes = [
"Workers vote to unionize after a long organizing campaign; the employer contests the result.",
@@ -148,6 +158,7 @@ probes = [
[[interest]]
id = "left-movements"
group = "left-politics"
weight = 0.85
probes = [
"A socialist party wins a decisive election, shifting the governing coalition leftward.",
@@ -157,6 +168,7 @@ probes = [
[[interest]]
id = "anticolonial-sovereignty"
group = "left-politics"
weight = 0.8
probes = [
"Indigenous communities win recognition of land rights against extractive industry.",
@@ -176,6 +188,7 @@ probes = [
[[interest]]
id = "political-theory"
group = "left-politics"
weight = 0.7
probes = [
"An essay argues about socialism, imperialism and political economy, drawing on history rather than reporting a news event.",
@@ -204,6 +217,7 @@ probes = [
[[interest]]
id = "cincinnati"
group = "local"
weight = 0.9
local = true
probes = [
@@ -216,6 +230,7 @@ probes = [
[[interest]]
id = "kentucky-louisville"
group = "local"
weight = 0.6
local = true
probes = [
@@ -226,6 +241,7 @@ probes = [
# coverage alongside Cincinnati city government.
[[interest]]
id = "ohio-state-politics"
group = "local"
weight = 0.75
local = true
probes = [