connor/news
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
7f402f22fe5836d98e67e3fbbeb1077d5449b8ed
news/profile/sources.toml
T
connor 7f402f22fe Rebalance the digest: rank by interest, cap per group, fix HN maturation 
Connor: too much world news, very little software/aerospace/urbanism/Cincinnati.
Measuring found three separate causes.

1. The digest ranked by max(interest, importance), and importance is largely a
   "big world news" detector -- 22 of 28 slots were entered on the importance
   axis. A personal reading list ordered by global importance is the wrong
   objective. The bulk of the digest is now ranked by INTEREST, with a bounded
   allowance (digest_importance_slots = 6) so a major event still makes the
   record on a busy day.

2. No per-group cap, so conflict and diplomacy took 18 of 28 slots.
   digest_max_per_group = 4, with unused capacity flowing back to the best
   remaining items so a quiet day still fills.

3. HN maturation could never happen. Items came only from `search_by_date`,
   which returns posts at 2-3 points; once one ages out of the newest window its
   score is never refetched, so it can never clear the bar and HN's best content
   was structurally invisible. 45 software clusters were sitting in
   "aggregator-immature". Added the front page via the `search` endpoint, which
   returns the crowd's verdict (measured: median 117 points, max 912) and whose
   re-polling is what actually supplies maturation. hn_min_score 150 -> 100,
   since a higher bar rejects most of the front page.

Digest went from 8 groups (conflict 10, diplomacy 8, software 0) to 13 groups
with a far flatter distribution, and the software slot is now
"Benchmarking Qwen3.8 27B quantizations" -- the local-inference material that
was asked for.

Process note: the first two attempts at this silently did nothing. I used
str.replace() without asserting the anchor matched, so a stale anchor wrote the
file back unchanged while reporting success, and I measured "no change" twice
before checking the file. Edits here now assert before writing.
2026-09-08 17:46:48 -04:00

414 lines
11 KiB
TOML
Raw Blame History

# Sources.
#
# `bucket` is the editorial-independence group, and it is the important field.
# Importance counts DISTINCT buckets, never feeds: two BBC feeds are one bucket,
# and ten outlets reprinting one wire story must not read as ten independent
# confirmations. Getting this wrong is what made the old system fire priority-5
# pushes on anything two feeds happened to carry.
#
# `weight` is editorial trust, carried over from Connor's old config where he
# had deliberately set Al Jazeera highest (1.2) and NYT lowest (0.8). It affects
# importance and tie-breaking, never whether an item is visible at all.
#
# `prominence = 1` marks a publisher's front-page/top-stories feed. Subscribing
# to both it and the section feed is a free editorial-promotion signal; the old
# config dropped BBC's top-news feed and threw that away.
[defaults]
poll_interval_secs = 900
# ------------------------------------------------------------- world / wire --
[[source]]
name = "Al Jazeera"
url = "https://www.aljazeera.com/xml/rss/all.xml"
bucket = "aljazeera"
weight = 1.2
[[source]]
name = "BBC Top Stories"
url = "https://feeds.bbci.co.uk/news/rss.xml"
bucket = "bbc"
prominence = 1
weight = 1.1
[[source]]
name = "BBC World"
url = "https://feeds.bbci.co.uk/news/world/rss.xml"
bucket = "bbc"
weight = 1.1
[[source]]
name = "Guardian World"
url = "https://www.theguardian.com/world/rss"
bucket = "guardian"
[[source]]
name = "NYT World"
url = "https://rss.nytimes.com/services/xml/rss/nyt/World.xml"
bucket = "nyt"
weight = 0.8
[[source]]
name = "NPR News"
url = "https://feeds.npr.org/1001/rss.xml"
bucket = "npr"
[[source]]
name = "France24"
url = "https://www.france24.com/en/rss"
bucket = "france24"
# RSS 1.0/RDF. The old hand-rolled quick-xml parser could not read this format
# and DW was excluded with the note that it "would yield zero items silently".
# feedparser handles RDF, so DW is back.
[[source]]
name = "Deutsche Welle"
url = "https://rss.dw.com/rdf/rss-en-all"
bucket = "dw"
[[source]]
name = "ABC Australia"
url = "https://www.abc.net.au/news/feed/51120/rss.xml"
bucket = "abc-au"
[[source]]
name = "PBS NewsHour"
url = "https://www.pbs.org/newshour/feeds/rss/headlines"
bucket = "pbs"
poll_interval_secs = 1800
# --------------------------------------------------------------- mideast ----
[[source]]
name = "Middle East Eye"
weight = 1.1
url = "https://www.middleeasteye.net/rss"
bucket = "middle-east-eye"
[[source]]
name = "+972 Magazine"
weight = 1.1
url = "https://www.972mag.com/feed/"
bucket = "972mag"
poll_interval_secs = 3600
# ----------------------------------------------------------- tech / policy --
[[source]]
name = "Ars Technica"
url = "https://feeds.arstechnica.com/arstechnica/index"
bucket = "ars-technica"
[[source]]
name = "EFF Deeplinks"
url = "https://www.eff.org/rss/updates.xml"
bucket = "eff"
poll_interval_secs = 3600
# Global-South technology and economics; covers two interest blocks at once.
[[source]]
name = "Rest of World"
url = "https://restofworld.org/feed/latest/"
bucket = "rest-of-world"
poll_interval_secs = 3600
# ------------------------------------------------------ software / systems --
[[source]]
name = "Hacker News"
url = "https://hn.algolia.com/api/v1/search_by_date"
kind = "hn"
bucket = "hacker-news"
poll_interval_secs = 1800
# The front page, which is the only way HN scores ever mature. Items fetched by
# `search_by_date` arrive at 2-3 points and, once they age out of that window,
# are never refetched -- so on their own they can never clear the maturation
# bar and HN's best content stays permanently invisible. Re-polling the front
# page refreshes the score of whatever the crowd has actually promoted.
[[source]]
name = "Hacker News front page"
url = "https://hn.algolia.com/api/v1/search"
kind = "hn"
query = "front_page"
bucket = "hacker-news"
page_size = 50
poll_interval_secs = 1800
[[source]]
name = "Lobsters"
url = "https://lobste.rs/rss"
bucket = "lobsters"
poll_interval_secs = 1800
# Reddit's .json endpoints return 403 without OAuth as of 2026-09; the .rss
# views work unauthenticated but 429 aggressively -- three requests in quick
# succession from one IP were enough to get throttled during validation, and the
# block persisted across serialized retries. Hence the long intervals here.
# The fetcher treats 429 as a state to back off from, not an error.
#
# Cost of the .rss route: the Atom feed carries no score, so these contribute no
# aggregator-maturity signal. Hacker News is the source that supplies it.
[[source]]
name = "r/LocalLLaMA"
url = "https://www.reddit.com/r/LocalLLaMA/new/.rss"
bucket = "reddit"
poll_interval_secs = 5400
[[source]]
name = "r/selfhosted"
url = "https://www.reddit.com/r/selfhosted/new/.rss"
bucket = "reddit"
poll_interval_secs = 7200
[[source]]
name = "r/rust"
url = "https://www.reddit.com/r/rust/new/.rss"
bucket = "reddit"
poll_interval_secs = 7200
# Release feeds for the stack Connor actually runs. GitHub releases are Atom,
# so they need no adapter of their own.
[[source]]
name = "llama.cpp releases"
url = "https://github.com/ggml-org/llama.cpp/releases.atom"
bucket = "github-releases"
poll_interval_secs = 7200
[[source]]
name = "LiteLLM releases"
url = "https://github.com/BerriAI/litellm/releases.atom"
bucket = "github-releases"
poll_interval_secs = 7200
[[source]]
name = "llama-swap releases"
url = "https://github.com/mostlygeek/llama-swap/releases.atom"
bucket = "github-releases"
poll_interval_secs = 7200
[[source]]
name = "Podman releases"
url = "https://github.com/containers/podman/releases.atom"
bucket = "github-releases"
poll_interval_secs = 7200
[[source]]
name = "Rust releases"
url = "https://github.com/rust-lang/rust/releases.atom"
bucket = "github-releases"
poll_interval_secs = 7200
# arXiv announces once per weekday (~00:00 UTC); the feed is legitimately EMPTY
# between announcement windows, which is not a fault. Verified 2026-09-08 that
# the endpoint serves valid RSS -- cs.CL, cs.LG and cs.AI were all empty at the
# time of checking, consistent with being between windows.
[[source]]
name = "arXiv cs.CL"
url = "https://rss.arxiv.org/rss/cs.CL"
bucket = "arxiv"
poll_interval_secs = 21600
# ------------------------------------------------------------------ labor ---
[[source]]
name = "Labor Notes"
url = "https://labornotes.org/feed"
bucket = "labor-notes"
poll_interval_secs = 3600
[[source]]
name = "Jacobin"
url = "https://jacobin.com/feed/"
bucket = "jacobin"
poll_interval_secs = 3600
# --------------------------------------------------------------- urbanism ---
[[source]]
name = "Streetsblog USA"
url = "https://usa.streetsblog.org/feed"
bucket = "streetsblog"
poll_interval_secs = 3600
[[source]]
name = "Strong Towns"
url = "https://www.strongtowns.org/journal?format=rss"
bucket = "strong-towns"
poll_interval_secs = 7200
# --------------------------------------------------------- energy / climate --
[[source]]
name = "Canary Media"
url = "https://www.canarymedia.com/rss"
bucket = "canary-media"
poll_interval_secs = 3600
# --------------------------------------------------------------- US / local --
[[source]]
name = "ProPublica"
url = "https://www.propublica.org/feeds/propublica/main"
bucket = "propublica"
poll_interval_secs = 3600
[[source]]
name = "NPR Politics"
url = "https://feeds.npr.org/1014/rss.xml"
bucket = "npr"
# Ohio Capital Journal and Kentucky Lantern are both States Newsroom affiliates
# and syndicate the same national desk -- they carried 10 byte-identical
# headlines in a single poll, more than any other source pair in the config.
# One bucket, so their shared copy reads as one confirmation rather than two.
# They still contribute genuinely independent state-level reporting; the cost of
# merging is slight under-counting there, which is the safe direction.
[[source]]
name = "Ohio Capital Journal"
url = "https://ohiocapitaljournal.com/feed/"
bucket = "states-newsroom"
poll_interval_secs = 3600
[[source]]
name = "WVXU Cincinnati"
url = "https://www.wvxu.org/local-news.rss"
bucket = "wvxu"
poll_interval_secs = 3600
[[source]]
name = "WCPO Cincinnati"
url = "https://www.wcpo.com/news.rss"
bucket = "wcpo"
poll_interval_secs = 3600
[[source]]
name = "Kentucky Lantern"
url = "https://kentuckylantern.com/feed/"
bucket = "states-newsroom"
poll_interval_secs = 3600
# ------------------------------------------------------------ italy / mexico --
[[source]]
name = "ANSA English"
url = "https://www.ansa.it/english/english_rss.xml"
bucket = "ansa"
poll_interval_secs = 3600
[[source]]
name = "Mexico News Daily"
url = "https://mexiconewsdaily.com/feed/"
bucket = "mexico-news-daily"
poll_interval_secs = 3600
# ============================================================ space / aerospace
# Connor is an aerospace engineer and asked for launches and missions, deep
# technical coverage, and astronomy/planetary science. All verified 2026-09-08.
[[source]]
name = "SpaceNews"
url = "https://spacenews.com/feed/"
bucket = "spacenews"
poll_interval_secs = 3600
[[source]]
name = "Spaceflight Now"
url = "https://spaceflightnow.com/feed/"
bucket = "spaceflight-now"
poll_interval_secs = 3600
[[source]]
name = "NASASpaceflight"
url = "https://www.nasaspaceflight.com/feed/"
bucket = "nasaspaceflight"
poll_interval_secs = 3600
[[source]]
name = "NASA Breaking"
url = "https://www.nasa.gov/rss/dyn/breaking_news.rss"
bucket = "nasa"
poll_interval_secs = 3600
[[source]]
name = "ESA"
url = "https://www.esa.int/rssfeed/Our_Activities/Space_News"
bucket = "esa"
poll_interval_secs = 7200
[[source]]
name = "Ars Technica Science"
url = "https://feeds.arstechnica.com/arstechnica/science"
bucket = "ars-technica"
poll_interval_secs = 3600
[[source]]
name = "Phys.org Space"
url = "https://phys.org/rss-feed/space-news/"
bucket = "phys-org"
poll_interval_secs = 7200
# Deliberately NOT added: nature.rss (mostly "Author Correction:" journal
# metadata, not news) and sciencedaily.com (a university press-release mill,
# 60 items a poll). Both were probed and rejected on the content they returned.
[[source]]
name = "Quanta Magazine"
url = "https://www.quantamagazine.org/feed/"
bucket = "quanta"
poll_interval_secs = 21600
# ================================================================ linux / ricing
# Connor: "Linux news in general. The ricing world/neovim configs/etc"
[[source]]
name = "Phoronix"
url = "https://www.phoronix.com/rss.php"
bucket = "phoronix"
poll_interval_secs = 3600
[[source]]
name = "LWN"
url = "https://lwn.net/headlines/newrss"
bucket = "lwn"
poll_interval_secs = 7200
[[source]]
name = "It's FOSS"
url = "https://news.itsfoss.com/rss/"
bucket = "its-foss"
poll_interval_secs = 7200
[[source]]
name = "r/unixporn"
url = "https://www.reddit.com/r/unixporn/top/.rss"
bucket = "reddit"
poll_interval_secs = 10800
[[source]]
name = "Neovim releases"
url = "https://github.com/neovim/neovim/releases.atom"
bucket = "github-releases"
poll_interval_secs = 21600
# ==================================================== left / anti-colonial =====
[[source]]
name = "African Arguments"
weight = 1.1
url = "https://africanarguments.org/feed/"
bucket = "african-arguments"
poll_interval_secs = 21600
[[source]]
name = "Tribune"
weight = 1.1
url = "https://tribunemag.co.uk/feed"
bucket = "tribune"
poll_interval_secs = 21600
[[source]]
name = "The Intercept"
weight = 1.1
url = "https://theintercept.com/feed/?rss"
bucket = "the-intercept"
poll_interval_secs = 7200
[[source]]
name = "Drop Site News"
weight = 1.1
url = "https://www.dropsitenews.com/feed"
bucket = "drop-site"
poll_interval_secs = 7200
Reference in new issue View Git Blame Copy Permalink