Merge States Newsroom affiliates into one independence bucket

Reading real clustered output caught a config error the tests could not:
Ohio Capital Journal and Kentucky Lantern are both States Newsroom outlets
syndicating the same national desk, and carried 10 byte-identical headlines in a
single poll -- more than any other source pair. Configured as separate buckets
they would have doubled the corroboration signal on every syndicated story,
which is exactly the failure the bucket concept exists to prevent.

Found by checking identical headlines across all source pairs rather than
spot-fixing the one noticed by eye. That sweep also confirmed the buckets
already set deliberately are right: BBC Top/World, Ars Technica/Science and
NPR News/Politics each share a bucket and each showed shared headlines.

Regression test added.
This commit is contained in:
connor committed 2026-09-08 15:44:37 -04:00
1 parent 863f5e20ac
commit d047c58d8a
2 files changed
+25 -2

No files matched your search

+8 -2
View File
@@ -237,10 +237,16 @@ name = "NPR Politics"
url = "https://feeds.npr.org/1014/rss.xml"
bucket = "npr"
# Ohio Capital Journal and Kentucky Lantern are both States Newsroom affiliates
# and syndicate the same national desk -- they carried 10 byte-identical
# headlines in a single poll, more than any other source pair in the config.
# One bucket, so their shared copy reads as one confirmation rather than two.
# They still contribute genuinely independent state-level reporting; the cost of
# merging is slight under-counting there, which is the safe direction.
[[source]]
name = "Ohio Capital Journal"
url = "https://ohiocapitaljournal.com/feed/"
bucket = "ohio-capital-journal"
bucket = "states-newsroom"
poll_interval_secs = 3600
[[source]]
@@ -258,7 +264,7 @@ poll_interval_secs = 3600
[[source]]
name = "Kentucky Lantern"
url = "https://kentuckylantern.com/feed/"
bucket = "kentucky-lantern"
bucket = "states-newsroom"
poll_interval_secs = 3600
# ------------------------------------------------------------ italy / mexico --