connor/news
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
6dc3694260cb7b81ee2a1518eeb5e9a2d27a83a4
news/profile/sources.toml
T
connor 6dc3694260 Profile v2: split conflict from statecraft, add space, expand vetoes 
Second elicitation round. The significant correction is Connor's own framing:
"my actual war interest is low, but unfortunately all wars are important." The
click history looked war-heavy only because world-news feeds are war-heavy, so
geopolitics-at-weight-1.0 was reading the corpus, not the reader. Statecraft is
now its own block and armed conflict is five weighted bands (anti-imperial,
US-involved, Mideast/Israel, undercovered, general) routed mainly by importance.

New: space-aerospace at 0.95, the highest-weighted block -- Connor is an
aerospace engineer. Plus science-discovery, environment-ecology,
ohio-state-politics, left-movements, anticolonial-sovereignty, far-right-watch,
political-theory. Linux/ricing/neovim folded into software-systems.

Introduces `demote` blocks, distinct from hard vetoes, for categories adjacent to
real interests: crypto/AI hype sits next to AI regulation, culture-war outrage
next to far-right-watch, true crime next to criminal-justice policy. Hard-vetoing
those would take real stories with them -- the lesson from the Europa League
story that both a substring blocklist and a cosine threshold wrongly killed.

Sources 39 -> 56, all fetched and verified: space (SpaceNews, Spaceflight Now,
NASASpaceflight, NASA, ESA, Phys.org), Linux (Phoronix, LWN, It's FOSS,
r/unixporn), left/anticolonial (Intercept, Drop Site, Tribune, African
Arguments). Rejected on inspection: nature.rss is journal metadata ("Author
Correction: ...") and sciencedaily is a press-release mill.

Embedding stays English-only, measured rather than assumed: the same story in
Spanish and English scored 0.758 and 0.600 cosine against a random-pair p99 of
0.634, so one pair was indistinguishable from an unrelated article. Italy and
Mexico are covered by English-language sources instead.

Also fixes clustering performance: centroids are held in memory across a batch
instead of one SELECT plus a numpy rebuild per item, which had taken ~15 minutes
for 1,258 items.
2026-09-08 15:25:56 -04:00

394 lines
10 KiB
TOML
Raw Blame History

# Sources.
#
# `bucket` is the editorial-independence group, and it is the important field.
# Importance counts DISTINCT buckets, never feeds: two BBC feeds are one bucket,
# and ten outlets reprinting one wire story must not read as ten independent
# confirmations. Getting this wrong is what made the old system fire priority-5
# pushes on anything two feeds happened to carry.
#
# `weight` is editorial trust, carried over from Connor's old config where he
# had deliberately set Al Jazeera highest (1.2) and NYT lowest (0.8). It affects
# importance and tie-breaking, never whether an item is visible at all.
#
# `prominence = 1` marks a publisher's front-page/top-stories feed. Subscribing
# to both it and the section feed is a free editorial-promotion signal; the old
# config dropped BBC's top-news feed and threw that away.
[defaults]
poll_interval_secs = 900
# ------------------------------------------------------------- world / wire --
[[source]]
name = "Al Jazeera"
url = "https://www.aljazeera.com/xml/rss/all.xml"
bucket = "aljazeera"
weight = 1.2
[[source]]
name = "BBC Top Stories"
url = "https://feeds.bbci.co.uk/news/rss.xml"
bucket = "bbc"
prominence = 1
weight = 1.1
[[source]]
name = "BBC World"
url = "https://feeds.bbci.co.uk/news/world/rss.xml"
bucket = "bbc"
weight = 1.1
[[source]]
name = "Guardian World"
url = "https://www.theguardian.com/world/rss"
bucket = "guardian"
[[source]]
name = "NYT World"
url = "https://rss.nytimes.com/services/xml/rss/nyt/World.xml"
bucket = "nyt"
weight = 0.8
[[source]]
name = "NPR News"
url = "https://feeds.npr.org/1001/rss.xml"
bucket = "npr"
[[source]]
name = "France24"
url = "https://www.france24.com/en/rss"
bucket = "france24"
# RSS 1.0/RDF. The old hand-rolled quick-xml parser could not read this format
# and DW was excluded with the note that it "would yield zero items silently".
# feedparser handles RDF, so DW is back.
[[source]]
name = "Deutsche Welle"
url = "https://rss.dw.com/rdf/rss-en-all"
bucket = "dw"
[[source]]
name = "ABC Australia"
url = "https://www.abc.net.au/news/feed/51120/rss.xml"
bucket = "abc-au"
[[source]]
name = "PBS NewsHour"
url = "https://www.pbs.org/newshour/feeds/rss/headlines"
bucket = "pbs"
poll_interval_secs = 1800
# --------------------------------------------------------------- mideast ----
[[source]]
name = "Middle East Eye"
weight = 1.1
url = "https://www.middleeasteye.net/rss"
bucket = "middle-east-eye"
[[source]]
name = "+972 Magazine"
weight = 1.1
url = "https://www.972mag.com/feed/"
bucket = "972mag"
poll_interval_secs = 3600
# ----------------------------------------------------------- tech / policy --
[[source]]
name = "Ars Technica"
url = "https://feeds.arstechnica.com/arstechnica/index"
bucket = "ars-technica"
[[source]]
name = "EFF Deeplinks"
url = "https://www.eff.org/rss/updates.xml"
bucket = "eff"
poll_interval_secs = 3600
# Global-South technology and economics; covers two interest blocks at once.
[[source]]
name = "Rest of World"
url = "https://restofworld.org/feed/latest/"
bucket = "rest-of-world"
poll_interval_secs = 3600
# ------------------------------------------------------ software / systems --
[[source]]
name = "Hacker News"
url = "https://hn.algolia.com/api/v1/search_by_date"
kind = "hn"
bucket = "hacker-news"
poll_interval_secs = 1800
[[source]]
name = "Lobsters"
url = "https://lobste.rs/rss"
bucket = "lobsters"
poll_interval_secs = 1800
# Reddit's .json endpoints return 403 without OAuth as of 2026-09; the .rss
# views work unauthenticated but 429 aggressively -- three requests in quick
# succession from one IP were enough to get throttled during validation, and the
# block persisted across serialized retries. Hence the long intervals here.
# The fetcher treats 429 as a state to back off from, not an error.
#
# Cost of the .rss route: the Atom feed carries no score, so these contribute no
# aggregator-maturity signal. Hacker News is the source that supplies it.
[[source]]
name = "r/LocalLLaMA"
url = "https://www.reddit.com/r/LocalLLaMA/new/.rss"
bucket = "reddit"
poll_interval_secs = 5400
[[source]]
name = "r/selfhosted"
url = "https://www.reddit.com/r/selfhosted/new/.rss"
bucket = "reddit"
poll_interval_secs = 7200
[[source]]
name = "r/rust"
url = "https://www.reddit.com/r/rust/new/.rss"
bucket = "reddit"
poll_interval_secs = 7200
# Release feeds for the stack Connor actually runs. GitHub releases are Atom,
# so they need no adapter of their own.
[[source]]
name = "llama.cpp releases"
url = "https://github.com/ggml-org/llama.cpp/releases.atom"
bucket = "github-releases"
poll_interval_secs = 7200
[[source]]
name = "LiteLLM releases"
url = "https://github.com/BerriAI/litellm/releases.atom"
bucket = "github-releases"
poll_interval_secs = 7200
[[source]]
name = "llama-swap releases"
url = "https://github.com/mostlygeek/llama-swap/releases.atom"
bucket = "github-releases"
poll_interval_secs = 7200
[[source]]
name = "Podman releases"
url = "https://github.com/containers/podman/releases.atom"
bucket = "github-releases"
poll_interval_secs = 7200
[[source]]
name = "Rust releases"
url = "https://github.com/rust-lang/rust/releases.atom"
bucket = "github-releases"
poll_interval_secs = 7200
# arXiv announces once per weekday (~00:00 UTC); the feed is legitimately EMPTY
# between announcement windows, which is not a fault. Verified 2026-09-08 that
# the endpoint serves valid RSS -- cs.CL, cs.LG and cs.AI were all empty at the
# time of checking, consistent with being between windows.
[[source]]
name = "arXiv cs.CL"
url = "https://rss.arxiv.org/rss/cs.CL"
bucket = "arxiv"
poll_interval_secs = 21600
# ------------------------------------------------------------------ labor ---
[[source]]
name = "Labor Notes"
url = "https://labornotes.org/feed"
bucket = "labor-notes"
poll_interval_secs = 3600
[[source]]
name = "Jacobin"
url = "https://jacobin.com/feed/"
bucket = "jacobin"
poll_interval_secs = 3600
# --------------------------------------------------------------- urbanism ---
[[source]]
name = "Streetsblog USA"
url = "https://usa.streetsblog.org/feed"
bucket = "streetsblog"
poll_interval_secs = 3600
[[source]]
name = "Strong Towns"
url = "https://www.strongtowns.org/journal?format=rss"
bucket = "strong-towns"
poll_interval_secs = 7200
# --------------------------------------------------------- energy / climate --
[[source]]
name = "Canary Media"
url = "https://www.canarymedia.com/rss"
bucket = "canary-media"
poll_interval_secs = 3600
# --------------------------------------------------------------- US / local --
[[source]]
name = "ProPublica"
url = "https://www.propublica.org/feeds/propublica/main"
bucket = "propublica"
poll_interval_secs = 3600
[[source]]
name = "NPR Politics"
url = "https://feeds.npr.org/1014/rss.xml"
bucket = "npr"
[[source]]
name = "Ohio Capital Journal"
url = "https://ohiocapitaljournal.com/feed/"
bucket = "ohio-capital-journal"
poll_interval_secs = 3600
[[source]]
name = "WVXU Cincinnati"
url = "https://www.wvxu.org/local-news.rss"
bucket = "wvxu"
poll_interval_secs = 3600
[[source]]
name = "WCPO Cincinnati"
url = "https://www.wcpo.com/news.rss"
bucket = "wcpo"
poll_interval_secs = 3600
[[source]]
name = "Kentucky Lantern"
url = "https://kentuckylantern.com/feed/"
bucket = "kentucky-lantern"
poll_interval_secs = 3600
# ------------------------------------------------------------ italy / mexico --
[[source]]
name = "ANSA English"
url = "https://www.ansa.it/english/english_rss.xml"
bucket = "ansa"
poll_interval_secs = 3600
[[source]]
name = "Mexico News Daily"
url = "https://mexiconewsdaily.com/feed/"
bucket = "mexico-news-daily"
poll_interval_secs = 3600
# ============================================================ space / aerospace
# Connor is an aerospace engineer and asked for launches and missions, deep
# technical coverage, and astronomy/planetary science. All verified 2026-09-08.
[[source]]
name = "SpaceNews"
url = "https://spacenews.com/feed/"
bucket = "spacenews"
poll_interval_secs = 3600
[[source]]
name = "Spaceflight Now"
url = "https://spaceflightnow.com/feed/"
bucket = "spaceflight-now"
poll_interval_secs = 3600
[[source]]
name = "NASASpaceflight"
url = "https://www.nasaspaceflight.com/feed/"
bucket = "nasaspaceflight"
poll_interval_secs = 3600
[[source]]
name = "NASA Breaking"
url = "https://www.nasa.gov/rss/dyn/breaking_news.rss"
bucket = "nasa"
poll_interval_secs = 3600
[[source]]
name = "ESA"
url = "https://www.esa.int/rssfeed/Our_Activities/Space_News"
bucket = "esa"
poll_interval_secs = 7200
[[source]]
name = "Ars Technica Science"
url = "https://feeds.arstechnica.com/arstechnica/science"
bucket = "ars-technica"
poll_interval_secs = 3600
[[source]]
name = "Phys.org Space"
url = "https://phys.org/rss-feed/space-news/"
bucket = "phys-org"
poll_interval_secs = 7200
# Deliberately NOT added: nature.rss (mostly "Author Correction:" journal
# metadata, not news) and sciencedaily.com (a university press-release mill,
# 60 items a poll). Both were probed and rejected on the content they returned.
[[source]]
name = "Quanta Magazine"
url = "https://www.quantamagazine.org/feed/"
bucket = "quanta"
poll_interval_secs = 21600
# ================================================================ linux / ricing
# Connor: "Linux news in general. The ricing world/neovim configs/etc"
[[source]]
name = "Phoronix"
url = "https://www.phoronix.com/rss.php"
bucket = "phoronix"
poll_interval_secs = 3600
[[source]]
name = "LWN"
url = "https://lwn.net/headlines/newrss"
bucket = "lwn"
poll_interval_secs = 7200
[[source]]
name = "It's FOSS"
url = "https://news.itsfoss.com/rss/"
bucket = "its-foss"
poll_interval_secs = 7200
[[source]]
name = "r/unixporn"
url = "https://www.reddit.com/r/unixporn/top/.rss"
bucket = "reddit"
poll_interval_secs = 10800
[[source]]
name = "Neovim releases"
url = "https://github.com/neovim/neovim/releases.atom"
bucket = "github-releases"
poll_interval_secs = 21600
# ==================================================== left / anti-colonial =====
[[source]]
name = "African Arguments"
weight = 1.1
url = "https://africanarguments.org/feed/"
bucket = "african-arguments"
poll_interval_secs = 21600
[[source]]
name = "Tribune"
weight = 1.1
url = "https://tribunemag.co.uk/feed"
bucket = "tribune"
poll_interval_secs = 21600
[[source]]
name = "The Intercept"
weight = 1.1
url = "https://theintercept.com/feed/?rss"
bucket = "the-intercept"
poll_interval_secs = 7200
[[source]]
name = "Drop Site News"
weight = 1.1
url = "https://www.dropsitenews.com/feed"
bucket = "drop-site"
poll_interval_secs = 7200
Reference in new issue View Git Blame Copy Permalink