# news

A personal news aggregator: many sources in, two to five well-chosen push
notifications and one readable daily digest out.

This is a ground-up rewrite of [`news_triage`](../news_triage), which is still
running. The rewrite exists because the old system had no representation of what
its user actually likes — its whole preference model was three keywords fed to
BM25 plus a two-term substring blocklist, and everything else was ranking
machinery with nothing meaningful to rank.

## The core idea

Taste is a set of **vectors**, not a bag of keywords.

Every interest, veto and importance category in [`profile/profile.toml`](profile/profile.toml)
is written as prose and embedded with a small CPU model (`bge-small-en-v1.5`,
~130 MB, ONNX, no torch, no GPU — so it never contends with the llama-swap stack).
An item's score is the max cosine similarity across a block's probes, never the
mean: averaging "Middle East geopolitics" with "grid-scale energy storage" gives
a vector near neither.

This is what lets a topic be expressed as a *region* rather than a string. The
old system could not represent "sports" as a concept, only as the literal
characters `s-p-o-r-t-s`, which is why 18 of its 32 feedback clicks were its user
hand-rejecting baseball recaps that `blocklist = ["sports"]` never matched.

A generative LLM is used only for prose — the daily digest write-up and cluster
headlines — at roughly 1–30 calls/day through LiteLLM, never for per-item scoring.

## Two routing lanes

Notifications are decided on two independent axes, because "something I might
like" and "something I need to know about" are different questions:

- **interest** — high similarity to the profile's interest vectors.
- **importance** — burst velocity, independent-source-bucket count, editorial
  prominence, and aggregator-native scores. Deliberately *not* raw corroboration
  count, which just measures how many wire-reprinting outlets you subscribed to.

Vetoes are absolute on **both** lanes. In the old system the bypass lane ignored
relevance entirely, so a blocklisted item could still fire at priority 5.

## Layout

    news/        the pipeline: sources, store, embed, cluster, score, route, deliver
    profile/     all configuration (see below)
    tests/
    deploy/

## Configuration

Nothing about the reader is compiled in. `news/*.py` contains no interest, no
place name, no feed URL and no personal identifier; all of it is data in three
TOML files under `profile/` (override the directory with `NEWS_CONFIG_DIR`):

| file | what it is | who edits it |
|---|---|---|
| `profile.toml` | who you are: interests, vetoes, demotes, importance, delivery | you, often |
| `sources.toml` | what to read, with independence buckets and trust weights | you, occasionally |
| `config.toml` | machinery: HTTP, embedding model, clustering and scoring constants | rarely |

To use this as someone else: replace `profile.toml` and `sources.toml`, and change
`user_agent` in `config.toml` to your own contact. That is the whole job.

`tests/test_config.py` exists to keep this true — it builds a configuration for a
different, invented reader and asserts that no part of the author's identity or
preferences reaches the running code.

Two constants deserve a warning. `[clustering].threshold` (0.82) and
`[scoring].veto_absolute` (0.60) were calibrated against a specific 3,885-item
corpus with a specific embedding model. **Changing `[embedding].model` invalidates
both**, because a different model puts cosine similarity on a different scale.
Re-run the calibration in `profile/CALIBRATION.md` before trusting them.

## Status

Early. `profile/` is written; the pipeline is not yet built.