Commit Graph
2 Commits
Author SHA1 Message Date
connor 863f5e20ac Make the system portable: split config from profile, remove hard-coding
Audit found three things compiled in that shouldn't have been: the User-Agent
carried the author's domain, the cluster threshold and window were module
constants the pipeline never overrode, and the embedding model was pinned --
which blocked the one change most likely to be wanted, swapping in a
multilingual model for native-language sources.

Configuration is now three files with distinct roles: config.toml (machinery),
profile.toml (who the reader is), sources.toml (what to read). All are
relocatable via NEWS_CONFIG_DIR. news/*.py now contains no interest, place name,
feed URL, or personal identifier.

tests/test_config.py keeps it that way: it builds a configuration for an
invented reader and asserts none of the author's identity or preferences reaches
the running code. Writing it immediately found a real bug -- CONFIG_DIR was
resolved at import time, so NEWS_CONFIG_DIR silently did nothing if set after
the module was first imported. Now resolved per instantiation.

Documented the coupling that portability creates: [clustering].threshold and
[scoring].veto_absolute are cosine constants calibrated against one corpus with
one model, so changing [embedding].model invalidates both.
2026-09-08 15:31:43 -04:00
connor c89a723603 Seed the rewrite: profile, calibration, and repo scaffolding
Ground-up replacement for news_triage, whose signal quality failed because it
had no representation of what its user likes -- three keywords fed to BM25 plus
a two-term substring blocklist.

profile/ holds the taste model as prose and TOML: 11 interest blocks, 3 veto
blocks, 5 importance blocks, each a set of probe sentences to be embedded.

The [scoring] constants are fitted, not guessed. CALIBRATION.md records the
measurements against all 3,885 items in the v0.2 database and 40 real feedback
labels, including two findings that changed the design:

- compare blocks by z-score within block, never raw cosine; raw cosine let
  geopolitics win 78% of items and buried known-liked items at median rank 1246
  of 3885 (293 after normalizing)
- veto needs a margin over the interest signal, not a threshold; a plain 0.55
  cutoff falsely killed "Belgian mayor blocks club from hosting Israeli opponent
  in Europa League", which the old substring blocklist also killed
2026-09-08 14:34:56 -04:00