Lots of memory changes mostly

2026-10-13 12:36:52 -04:00
1 parent 29192320cd
commit b450004d08
@@ -0,0 +1,2 @@
+- [News rewrite status](news-rewrite-status.md) — old Rust system still in production, Python rewrite at ~/docs/projects/news; read profile/CALIBRATION.md before touching scoring constants
+- [news_triage v0.2 gotchas](news-triage-v02-gotchas.md) — SIGHUP won't reload sources, healthcheck lies, ntfy buttons dead by design
@@ -0,0 +1,29 @@
+---
+name: news-rewrite-status
+description: State of the news aggregator rewrite - old system still running, new one at ~/docs/projects/news, design rationale lives in profile/CALIBRATION.md
+metadata:
+  type: project
+---
+
+Two systems exist. **`~/docs/projects/news_triage`** (Rust, v0.2, container
+`connor_news-backend_1`, config `~/config/news/config.toml`) is **still running in
+production** and serving Connor's notifications. **`~/docs/projects/news`** is the
+greenfield Python rewrite, started 2026-09-08. Keep the old one running until cutover;
+Connor chose **no data import** into the new schema.
+
+The rewrite exists because signal quality failed, and the root cause was that the old
+system had no representation of taste — three keywords fed to BM25 plus a two-term
+substring blocklist. The fix is embeddings: interests, vetoes, and importance categories
+written as prose in `profile/profile.toml`, embedded on CPU with bge-small-en-v1.5.
+
+**Before changing any scoring constant, read `profile/CALIBRATION.md` in that repo.** The
+constants were fitted against 3,885 real items and 40 real feedback labels, and two of the
+findings are counterintuitive enough to be re-derived wrongly: blocks must be compared by
+z-score *within block* (raw cosine gave geopolitics 78% of all items), and veto must be a
+margin over the interest signal rather than a similarity threshold (a plain cutoff killed a
+Middle East story about a mayor barring an Israeli football club).
+
+Phases 0 and 1 are done. Phase 2 onward (ingest/embed/cluster, then scoring, delivery, web
+UI) is not built. Plan file: `~/.claude/plans/alright-so-i-d-like-tranquil-sunset.md`.
+
+Related: [[news-triage-v02-gotchas]]
@@ -0,0 +1,28 @@
+---
+name: news-triage-v02-gotchas
+description: Operational traps in the running news_triage v0.2 container - config reload scope, permanently-unhealthy healthcheck, dead ntfy buttons
+metadata:
+  type: project
+---
+
+Traps in the still-running `connor_news-backend_1` (from `~/docs/projects/news_triage`),
+each of which costs a debugging cycle to rediscover:
+
+- **SIGHUP does not reload sources.** `watch_sighup` swaps the config, and per-cycle reads
+  pick up `[topics]`, `[notify]`, `[clustering]` and `[llm]`. But `spawn_source_loop` takes
+  an *owned* `Source` and the source list is enumerated once at startup in `build_runtime`,
+  so adding, removing, or disabling a feed needs `sudo podman restart`, not a HUP.
+- **The container reports `unhealthy` forever and is fine.** The compose healthcheck is
+  `CMD curl -f localhost:3000/healthz` and the runtime image ships no curl.
+- **ntfy feedback buttons have never worked.** `NEWS_API_BASE_URL` is unset, so they are
+  dropped at boot with a single WARN. ntfy executes those actions from the phone, so it
+  needs a publicly reachable URL. Deliberately left unfixed — `/api/feedback` has no auth,
+  and with no data import into the rewrite the labels have no carry-over value.
+- **The bypass lane ignores relevance entirely** (`gate.rs:302`): it fires priority-5 push
+  on `source_count >= 2` OR p99 importance, skipping quiet hours, the token bucket, and the
+  blocklist. `bypass_ceiling_per_day` is the only brake on it.
+
+Config is bind-mounted, so per the usual rule an edit needs `sudo podman restart` —
+`podman-compose up -d` does nothing when only the mounted file changed.
+
+Related: [[news-rewrite-status]]