Seed the rewrite: profile, calibration, and repo scaffolding
Ground-up replacement for news_triage, whose signal quality failed because it had no representation of what its user likes -- three keywords fed to BM25 plus a two-term substring blocklist. profile/ holds the taste model as prose and TOML: 11 interest blocks, 3 veto blocks, 5 importance blocks, each a set of probe sentences to be embedded. The [scoring] constants are fitted, not guessed. CALIBRATION.md records the measurements against all 3,885 items in the v0.2 database and 40 real feedback labels, including two findings that changed the design: - compare blocks by z-score within block, never raw cosine; raw cosine let geopolitics win 78% of items and buried known-liked items at median rank 1246 of 3885 (293 after normalizing) - veto needs a margin over the interest signal, not a threshold; a plain 0.55 cutoff falsely killed "Belgian mayor blocks club from hosting Israeli opponent in Europa League", which the old substring blocklist also killed
This commit is contained in:
commit
c89a723603
6 files changed
+551
No files matched your search
@@ -0,0 +1,108 @@
|
||||
# Connor's news profile
|
||||
|
||||
The thing the previous system never had. `news_triage` v0.2's entire model of taste was
|
||||
three keywords and two blocked substrings; everything else was ranking machinery with
|
||||
nothing meaningful to rank. This file is the replacement, and `profile.toml` is its
|
||||
machine-readable form.
|
||||
|
||||
Written 2026-09-08, from a structured interview plus evidence from 32 real feedback
|
||||
clicks in the v0.2 database.
|
||||
|
||||
## How this is used
|
||||
|
||||
Every `description` and `probe` below is embedded (bge-small-en-v1.5, CPU) into a vector.
|
||||
Item relevance is the **max** cosine similarity across probes, never the mean — averaging
|
||||
"Middle East geopolitics" with "grid-scale energy storage" produces a vector near neither.
|
||||
Feedback then moves these vectors over time (Rocchio); the written text is the cold-start
|
||||
seed so the system is useful on day one instead of after 200 clicks.
|
||||
|
||||
Vetoes work the same way, as *regions* rather than strings. This is the whole point: the old
|
||||
system could not express "sports" as a concept, only as the literal characters `s-p-o-r-t-s`,
|
||||
which is why 18 of 32 feedback clicks were Connor hand-rejecting AP baseball recaps that a
|
||||
`blocklist = ["sports"]` never matched.
|
||||
|
||||
## Interests
|
||||
|
||||
Ordered roughly by strength. Weights are multipliers on the similarity score.
|
||||
|
||||
1. **Geopolitics and conflict** (1.0) — the strongest and most consistent signal. Middle East
|
||||
above all, plus great-power dynamics: China's expanding global reach, US foreign policy,
|
||||
war and escalation, sanctions, the UN.
|
||||
2. **Tech and science policy** (0.9) — AI regulation, platform governance, surveillance and
|
||||
privacy law, research funding. Policy *about* technology.
|
||||
3. **Labor, inequality and social policy** (0.9) — the "democratic socialism" of the old
|
||||
config, read properly: unions, strikes, wages, housing, healthcare, education, welfare
|
||||
states.
|
||||
4. **Software and systems, deep** (0.9) — Connor runs a self-hosted inference stack and a
|
||||
substantial homelab. Rust and WASM, systems programming, self-hosting, local model
|
||||
inference, and release notes for things he actually runs. **Eligible for push**, not
|
||||
digest-only — this was an explicit choice.
|
||||
5. **Urbanism and city planning** (0.8) — transit, zoning, housing density, street design,
|
||||
public space. New in this profile; entirely absent from v0.2.
|
||||
6. **Energy, climate and infrastructure** (0.6) — grid buildout, storage, renewables, climate
|
||||
policy. Real but explicitly weaker than the others.
|
||||
7. **Global South** (0.6) — Africa, South Asia, Latin America. Not selected in the interview,
|
||||
but three of the fourteen starred items were Uber exiting Nigeria and Uganda, India's grid
|
||||
and storage crunch, and explosions at a Bolivian military base. Carried at a modest weight
|
||||
on that evidence. **Flagged for review** — drop it if it turns out to be noise.
|
||||
|
||||
### Geography
|
||||
|
||||
- **Cincinnati** — where he lives. Local politics and city stories, at a much lower
|
||||
importance bar than world news.
|
||||
- **Louisville / Kentucky** — where he is from. Still wanted, weaker than Cincinnati.
|
||||
- **Ohio** — state politics.
|
||||
- **US national politics** — federal policy, courts, elections, legislation.
|
||||
- **Italy** and **Mexico** — specific standing country interests.
|
||||
|
||||
## Vetoes
|
||||
|
||||
- **US big-three sports** — professional and college basketball, American football, baseball,
|
||||
and hockey. Game recaps, scores, trades, drafts, salary-cap news, betting lines.
|
||||
- **Royal and celebrity gossip** — the British royal family, red carpets, box office,
|
||||
influencer drama.
|
||||
- **Horoscopes, listicles, product-roundup content.**
|
||||
|
||||
### Explicitly NOT vetoed
|
||||
|
||||
Connor is not anti-sport; he is anti *US-big-three-wire-recap*. These stay:
|
||||
|
||||
- **Rock climbing, chess, the Olympics, oddball sports** — actively wanted.
|
||||
- **Soccer** — he is trying to get into it. Match previews and live threads are noise, but
|
||||
the sport is not blocked as a concept. This is the first thing to loosen if soccer feels
|
||||
under-covered.
|
||||
- **Sports governance and politics** — FIFA/UEFA corruption, Olympic host-city displacement,
|
||||
Qatar migrant labour. These read as politics, not sport, and the old blocklist's
|
||||
collisions with them were a bug.
|
||||
|
||||
## The importance lane
|
||||
|
||||
Independent of interest. This is the "I need to know this even though it isn't my topic"
|
||||
channel, and it is half the reason the project exists. Its own small daily budget.
|
||||
|
||||
- War and escalation — new military action between states, major escalation in an ongoing
|
||||
conflict, nuclear or WMD events.
|
||||
- Mass-casualty events and disasters — earthquakes, major attacks, industrial and transport
|
||||
catastrophes.
|
||||
- US federal shocks — constitutional crises, major Supreme Court rulings, shutdowns, sudden
|
||||
sweeping policy moves.
|
||||
- Economic breaks — market crashes, bank failures, currency crises, central bank surprises.
|
||||
- Major US political events — significant election results, deaths of political figures.
|
||||
|
||||
**Vetoes still apply on this lane.** In v0.2 the bypass lane ignored relevance entirely
|
||||
(`gate.rs:302`), so a blocklisted item could still fire at priority 5. That must not recur.
|
||||
|
||||
## Delivery
|
||||
|
||||
- 2–5 pushes per day total, across both lanes. High bar.
|
||||
- The daily digest is a **web page**, not a push payload. v0.2 sent a long markdown blob that
|
||||
ntfy turned into a `.md` attachment, which is miserable to read on a phone. The push is
|
||||
three lines and a link.
|
||||
- ntfy feedback buttons **must work**. They never have — `NEWS_API_BASE_URL` was unset, so
|
||||
they were silently dropped at boot. That is the entire training signal for everything above.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Digest hour, and the exact quiet-hours window.
|
||||
- Whether the importance lane may ever override quiet hours, and at what score.
|
||||
- Whether "Global South" is a real standing interest or an artifact of a small sample.
|
||||
Reference in new issue
Block a user