Make the system portable: split config from profile, remove hard-coding
Audit found three things compiled in that shouldn't have been: the User-Agent carried the author's domain, the cluster threshold and window were module constants the pipeline never overrode, and the embedding model was pinned -- which blocked the one change most likely to be wanted, swapping in a multilingual model for native-language sources. Configuration is now three files with distinct roles: config.toml (machinery), profile.toml (who the reader is), sources.toml (what to read). All are relocatable via NEWS_CONFIG_DIR. news/*.py now contains no interest, place name, feed URL, or personal identifier. tests/test_config.py keeps it that way: it builds a configuration for an invented reader and asserts none of the author's identity or preferences reaches the running code. Writing it immediately found a real bug -- CONFIG_DIR was resolved at import time, so NEWS_CONFIG_DIR silently did nothing if set after the module was first imported. Now resolved per instantiation. Documented the coupling that portability creates: [clustering].threshold and [scoring].veto_absolute are cosine constants calibrated against one corpus with one model, so changing [embedding].model invalidates both.
This commit is contained in:
1 parent
6dc3694260
commit
863f5e20ac
8 files changed
+317
-36
No files matched your search
@@ -43,11 +43,36 @@ relevance entirely, so a blocklisted item could still fire at priority 5.
|
||||
|
||||
## Layout
|
||||
|
||||
profile/ the user's taste, as prose and as TOML. Start here.
|
||||
news/ the pipeline: sources, store, embed, cluster, score, route, deliver
|
||||
profile/ all configuration (see below)
|
||||
tests/
|
||||
deploy/
|
||||
|
||||
## Configuration
|
||||
|
||||
Nothing about the reader is compiled in. `news/*.py` contains no interest, no
|
||||
place name, no feed URL and no personal identifier; all of it is data in three
|
||||
TOML files under `profile/` (override the directory with `NEWS_CONFIG_DIR`):
|
||||
|
||||
| file | what it is | who edits it |
|
||||
|---|---|---|
|
||||
| `profile.toml` | who you are: interests, vetoes, demotes, importance, delivery | you, often |
|
||||
| `sources.toml` | what to read, with independence buckets and trust weights | you, occasionally |
|
||||
| `config.toml` | machinery: HTTP, embedding model, clustering and scoring constants | rarely |
|
||||
|
||||
To use this as someone else: replace `profile.toml` and `sources.toml`, and change
|
||||
`user_agent` in `config.toml` to your own contact. That is the whole job.
|
||||
|
||||
`tests/test_config.py` exists to keep this true — it builds a configuration for a
|
||||
different, invented reader and asserts that no part of the author's identity or
|
||||
preferences reaches the running code.
|
||||
|
||||
Two constants deserve a warning. `[clustering].threshold` (0.82) and
|
||||
`[scoring].veto_absolute` (0.60) were calibrated against a specific 3,885-item
|
||||
corpus with a specific embedding model. **Changing `[embedding].model` invalidates
|
||||
both**, because a different model puts cosine similarity on a different scale.
|
||||
Re-run the calibration in `profile/CALIBRATION.md` before trusting them.
|
||||
|
||||
## Status
|
||||
|
||||
Early. `profile/` is written; the pipeline is not yet built.
|
||||
Reference in new issue
Block a user