Seed the rewrite: profile, calibration, and repo scaffolding
Ground-up replacement for news_triage, whose signal quality failed because it had no representation of what its user likes -- three keywords fed to BM25 plus a two-term substring blocklist. profile/ holds the taste model as prose and TOML: 11 interest blocks, 3 veto blocks, 5 importance blocks, each a set of probe sentences to be embedded. The [scoring] constants are fitted, not guessed. CALIBRATION.md records the measurements against all 3,885 items in the v0.2 database and 40 real feedback labels, including two findings that changed the design: - compare blocks by z-score within block, never raw cosine; raw cosine let geopolitics win 78% of items and buried known-liked items at median rank 1246 of 3885 (293 after normalizing) - veto needs a margin over the interest signal, not a threshold; a plain 0.55 cutoff falsely killed "Belgian mayor blocks club from hosting Israeli opponent in Europa League", which the old substring blocklist also killed
This commit is contained in:
commit
c89a723603
6 files changed
+551
No files matched your search
@@ -0,0 +1,28 @@
|
||||
[project]
|
||||
name = "news"
|
||||
version = "0.1.0"
|
||||
description = "Personal news aggregator with an embedding-based model of taste"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = [
|
||||
"httpx>=0.27", # conditional GET, async polling
|
||||
"feedparser>=6.0", # RSS/Atom incl. RSS 1.0/RDF, which the old hand-rolled quick-xml parser could not read
|
||||
"fastembed>=0.4", # bge-small-en-v1.5 via ONNX on CPU; deliberately not torch/sentence-transformers
|
||||
"numpy>=1.26",
|
||||
"starlette>=0.37",
|
||||
"jinja2>=3.1",
|
||||
"uvicorn>=0.30",
|
||||
"python-multipart>=0.0.9",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
news = "news.cli:main"
|
||||
|
||||
[dependency-groups]
|
||||
dev = ["pytest>=8.0", "pytest-asyncio>=0.23", "respx>=0.21"]
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
Reference in new issue
Block a user