Rebalance the digest: rank by interest, cap per group, fix HN maturation
Connor: too much world news, very little software/aerospace/urbanism/Cincinnati. Measuring found three separate causes. 1. The digest ranked by max(interest, importance), and importance is largely a "big world news" detector -- 22 of 28 slots were entered on the importance axis. A personal reading list ordered by global importance is the wrong objective. The bulk of the digest is now ranked by INTEREST, with a bounded allowance (digest_importance_slots = 6) so a major event still makes the record on a busy day. 2. No per-group cap, so conflict and diplomacy took 18 of 28 slots. digest_max_per_group = 4, with unused capacity flowing back to the best remaining items so a quiet day still fills. 3. HN maturation could never happen. Items came only from `search_by_date`, which returns posts at 2-3 points; once one ages out of the newest window its score is never refetched, so it can never clear the bar and HN's best content was structurally invisible. 45 software clusters were sitting in "aggregator-immature". Added the front page via the `search` endpoint, which returns the crowd's verdict (measured: median 117 points, max 912) and whose re-polling is what actually supplies maturation. hn_min_score 150 -> 100, since a higher bar rejects most of the front page. Digest went from 8 groups (conflict 10, diplomacy 8, software 0) to 13 groups with a far flatter distribution, and the software slot is now "Benchmarking Qwen3.8 27B quantizations" -- the local-inference material that was asked for. Process note: the first two attempts at this silently did nothing. I used str.replace() without asserting the anchor matched, so a stale anchor wrote the file back unchanged while reporting success, and I measured "no change" twice before checking the file. Edits here now assert before writing.
This commit is contained in:
1 parent
83bc36aa79
commit
7f402f22fe
5 files changed
+91
-21
No files matched your search
+11
-1
@@ -97,12 +97,22 @@ digest_floor = 0.5
|
||||
# A floor alone yielded 198 items a day. The digest is capped by count and fed
|
||||
# strongest-first, so the cap is always spent on the best candidates.
|
||||
digest_max_items = 25
|
||||
# Max items per topic group in the digest, so no single subject crowds it out.
|
||||
# Measured before this existed: conflict and diplomacy took 18 of 28 slots while
|
||||
# software-systems took 0, despite being 141 of the eligible clusters.
|
||||
digest_max_per_group = 4
|
||||
# Slots reserved for the day's most important stories regardless of topic, so a
|
||||
# major event stays in the record even when your own interests are busy. The
|
||||
# remaining slots are ranked by interest.
|
||||
digest_importance_slots = 6
|
||||
# Serendipity slots are filled from items BELOW this interest score, so they are
|
||||
# genuinely outside the profile rather than near-misses.
|
||||
serendipity_max_interest = 0.5
|
||||
|
||||
# Aggregator maturation: an HN post at 3 points is noise, at 300 it is news.
|
||||
hn_min_score = 150
|
||||
# 100, not 150: the HN front page has a median of ~117 points, so a higher bar
|
||||
# would reject most of what the crowd has actually promoted.
|
||||
hn_min_score = 100
|
||||
|
||||
# Nothing older than this may generate a notification. Feeds return large
|
||||
# archives on first fetch -- one run pulled 88 items going back five months --
|
||||
|
||||
@@ -118,6 +118,20 @@ kind = "hn"
|
||||
bucket = "hacker-news"
|
||||
poll_interval_secs = 1800
|
||||
|
||||
# The front page, which is the only way HN scores ever mature. Items fetched by
|
||||
# `search_by_date` arrive at 2-3 points and, once they age out of that window,
|
||||
# are never refetched -- so on their own they can never clear the maturation
|
||||
# bar and HN's best content stays permanently invisible. Re-polling the front
|
||||
# page refreshes the score of whatever the crowd has actually promoted.
|
||||
[[source]]
|
||||
name = "Hacker News front page"
|
||||
url = "https://hn.algolia.com/api/v1/search"
|
||||
kind = "hn"
|
||||
query = "front_page"
|
||||
bucket = "hacker-news"
|
||||
page_size = 50
|
||||
poll_interval_secs = 1800
|
||||
|
||||
[[source]]
|
||||
name = "Lobsters"
|
||||
url = "https://lobste.rs/rss"
|
||||
|
||||
Reference in new issue
Block a user