Phase 5: triage feed and the tuning surface
/ is the labelling machine: ranked clusters, one keystroke per judgement (j/k move, i/n label, o open), advancing automatically so a run of judgements is a run of single keys. Feedback volume is the whole point -- the old system collected 32 labels because its buttons did not persist and its UI could only be clicked. Browser feedback posts to /ui/feedback/, deliberately NOT under /api/feedback/, which Caddy routes around Authelia for the ntfy buttons. The browser path stays behind the identity gateway and so needs no signature. /tune drives the same route() the live service calls, so what it shows is what would have been sent. The old replay harness hardcoded its parameters and could never sweep the thresholds it existed to tune. Building it corrected a claim I had already made twice. I said the budgets bind before the thresholds, then wrote that the subject quota binds before the budget. Measured on this corpus, neither is generally true: at the shipped budgets (3 interest, 2 importance) the BUDGET binds and the quota is inert -- moving it 1 to 3 changes nothing. Raise budgets to 8 and the QUOTA takes over, 1/2/3 giving 10/11/13 alerts. The page now says exactly that, with the numbers, rather than a rule of thumb that happened to be wrong.
This commit is contained in:
1 parent
e99b7c9fb7
commit
42c38d759d
4 files changed
+454
-3
No files matched your search
@@ -84,8 +84,19 @@ Re-run the calibration in `profile/CALIBRATION.md` before trusting them.
|
|||||||
`simulate` is the tuning surface. The old system's replay harness hardcoded its
|
`simulate` is the tuning surface. The old system's replay harness hardcoded its
|
||||||
parameters and so could never sweep the thresholds it existed to tune.
|
parameters and so could never sweep the thresholds it existed to tune.
|
||||||
|
|
||||||
|
## Web
|
||||||
|
|
||||||
|
/ triage feed -- ranked, keyboard-driven (j/k move, i/n label, o open)
|
||||||
|
/tune move the thresholds and budgets, see what would have been sent
|
||||||
|
/digest the day's briefing, mobile-first
|
||||||
|
/healthz
|
||||||
|
|
||||||
|
Everything except `/api/feedback/*` sits behind the identity gateway. That one
|
||||||
|
path is exempt because ntfy runs notification buttons from the phone and a
|
||||||
|
native app cannot complete an interactive login; it is HMAC-signed instead.
|
||||||
|
|
||||||
## Status
|
## Status
|
||||||
|
|
||||||
Ingest, embedding, clustering, two-axis scoring and routing are built and
|
Running in production. Ingest, clustering, two-axis scoring, routing, ntfy
|
||||||
running against live feeds. Delivery (ntfy, the digest page) and the web UI are
|
delivery with working feedback buttons, the digest page and the triage UI are
|
||||||
not yet written.
|
all live.
|
||||||
@@ -0,0 +1,153 @@
|
|||||||
|
<!doctype html>
|
||||||
|
<meta charset="utf-8">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||||
|
<title>Triage</title>
|
||||||
|
<style>
|
||||||
|
:root { color-scheme: light dark; --fg:#16181d; --bg:#fdfdfc; --muted:#6b7280;
|
||||||
|
--rule:#e4e4e7; --accent:#7c3aed; --card:#fff; --sel:#f3f0ff;
|
||||||
|
--pos:#15803d; --neg:#b91c1c; --warn:#b45309; }
|
||||||
|
@media (prefers-color-scheme: dark) {
|
||||||
|
:root { --fg:#e8e8ea; --bg:#141416; --muted:#9ca3af; --rule:#2a2a2e;
|
||||||
|
--accent:#a78bfa; --card:#1b1b1f; --sel:#241f36;
|
||||||
|
--pos:#4ade80; --neg:#f87171; --warn:#fbbf24; }
|
||||||
|
}
|
||||||
|
* { box-sizing:border-box; }
|
||||||
|
body { margin:0; background:var(--bg); color:var(--fg);
|
||||||
|
font:15px/1.55 -apple-system, BlinkMacSystemFont, "Segoe UI", system-ui, sans-serif; }
|
||||||
|
main { max-width:52rem; margin:0 auto; padding:1.25rem 1rem 6rem; }
|
||||||
|
header { display:flex; flex-wrap:wrap; gap:.75rem; align-items:baseline;
|
||||||
|
justify-content:space-between; border-bottom:1px solid var(--rule);
|
||||||
|
padding-bottom:.6rem; margin-bottom:1rem; }
|
||||||
|
h1 { font-size:1.15rem; margin:0; letter-spacing:-.01em; }
|
||||||
|
.stats, .filters { color:var(--muted); font-size:.83rem; }
|
||||||
|
.filters a { color:var(--accent); text-decoration:none; margin-left:.6rem; }
|
||||||
|
.filters a.on { text-decoration:underline; }
|
||||||
|
|
||||||
|
article { background:var(--card); border:1px solid var(--rule); border-radius:9px;
|
||||||
|
padding:.7rem .85rem; margin-bottom:.5rem; scroll-margin-top:1rem; }
|
||||||
|
article.sel { border-color:var(--accent); background:var(--sel); }
|
||||||
|
article.done { opacity:.4; }
|
||||||
|
.top { display:flex; gap:.6rem; align-items:baseline; justify-content:space-between; }
|
||||||
|
h2 { font-size:.98rem; line-height:1.35; margin:0 0 .3rem; font-weight:600; }
|
||||||
|
h2 a { color:inherit; text-decoration:none; }
|
||||||
|
h2 a:hover { text-decoration:underline; }
|
||||||
|
.tags { display:flex; flex-wrap:wrap; gap:.4rem; margin:.35rem 0; }
|
||||||
|
.tag { font-size:.7rem; text-transform:uppercase; letter-spacing:.04em;
|
||||||
|
color:var(--muted); border:1px solid var(--rule); border-radius:99px;
|
||||||
|
padding:.05rem .45rem; white-space:nowrap; }
|
||||||
|
.tag.lane-alert\:important { color:var(--neg); border-color:currentColor; }
|
||||||
|
.tag.lane-alert\:interest { color:var(--accent); border-color:currentColor; }
|
||||||
|
.tag.demote { color:var(--warn); border-color:currentColor; }
|
||||||
|
.srcs { list-style:none; padding:0; margin:.35rem 0 0; }
|
||||||
|
.srcs li { font-size:.83rem; margin:.12rem 0; color:var(--muted); }
|
||||||
|
.srcs a { color:inherit; text-decoration:none; border-bottom:1px solid var(--rule); }
|
||||||
|
.acts { display:flex; gap:.4rem; margin-top:.5rem; }
|
||||||
|
button { font:inherit; font-size:.83rem; padding:.22rem .7rem; cursor:pointer;
|
||||||
|
border:1px solid var(--rule); background:transparent; color:inherit;
|
||||||
|
border-radius:6px; }
|
||||||
|
button.yes:hover { border-color:var(--pos); color:var(--pos); }
|
||||||
|
button.no:hover { border-color:var(--neg); color:var(--neg); }
|
||||||
|
button[disabled] { opacity:.45; cursor:default; }
|
||||||
|
kbd { font:inherit; font-size:.72rem; border:1px solid var(--rule);
|
||||||
|
border-bottom-width:2px; border-radius:4px; padding:0 .25rem; color:var(--muted); }
|
||||||
|
.help { position:fixed; bottom:0; left:0; right:0; background:var(--card);
|
||||||
|
border-top:1px solid var(--rule); padding:.45rem 1rem; font-size:.78rem;
|
||||||
|
color:var(--muted); display:flex; gap:1rem; justify-content:center; }
|
||||||
|
.empty { color:var(--muted); font-style:italic; padding:2rem 0; }
|
||||||
|
@media (max-width:34rem) { .help { display:none; } }
|
||||||
|
</style>
|
||||||
|
<main>
|
||||||
|
<header>
|
||||||
|
<h1>Triage</h1>
|
||||||
|
<div class="stats">{{ rows|length }} shown · {{ labelled }} labelled · {{ total }} clusters</div>
|
||||||
|
<div class="filters">show
|
||||||
|
<a class="{{ 'on' if show == 'unlabelled' }}" href="?show=unlabelled">unlabelled</a>
|
||||||
|
<a class="{{ 'on' if show == 'all' }}" href="?show=all">all</a>
|
||||||
|
· <a href="/digest">digest</a>
|
||||||
|
</div>
|
||||||
|
</header>
|
||||||
|
|
||||||
|
{% for r in rows %}
|
||||||
|
<article id="c{{ loop.index0 }}" data-id="{{ r.id }}"
|
||||||
|
class="{{ 'done' if r.labelled }}">
|
||||||
|
<h2><a href="{{ r.sources[0].url if r.sources else '#' }}" target="_blank"
|
||||||
|
rel="noopener">{{ r.headline }}</a></h2>
|
||||||
|
<div class="tags">
|
||||||
|
<span class="tag lane-{{ r.lane }}">{{ r.lane }}</span>
|
||||||
|
<span class="tag">{{ r.topic }} · int {{ '%.2f'|format(r.interest) }}</span>
|
||||||
|
<span class="tag">{{ r.importance_topic }} · imp {{ '%.2f'|format(r.importance) }}</span>
|
||||||
|
<span class="tag">{{ r.origins }} origin{{ '' if r.origins == 1 else 's' }}</span>
|
||||||
|
{% if r.demote %}<span class="tag demote">demoted: {{ r.demote }}</span>{% endif %}
|
||||||
|
</div>
|
||||||
|
<ul class="srcs">
|
||||||
|
{% for s in r.sources %}
|
||||||
|
<li><a href="{{ s.url }}" target="_blank" rel="noopener">{{ s.title }}</a>
|
||||||
|
— {{ s.name }}</li>
|
||||||
|
{% endfor %}
|
||||||
|
</ul>
|
||||||
|
<div class="acts">
|
||||||
|
<button class="yes" data-kind="interested">Interested <kbd>i</kbd></button>
|
||||||
|
<button class="no" data-kind="not_interested">Not interested <kbd>n</kbd></button>
|
||||||
|
</div>
|
||||||
|
</article>
|
||||||
|
{% else %}
|
||||||
|
<p class="empty">Nothing left to label. Try <a href="?show=all">show all</a>.</p>
|
||||||
|
{% endfor %}
|
||||||
|
</main>
|
||||||
|
|
||||||
|
<div class="help">
|
||||||
|
<span><kbd>j</kbd>/<kbd>k</kbd> move</span>
|
||||||
|
<span><kbd>i</kbd> interested</span>
|
||||||
|
<span><kbd>n</kbd> not interested</span>
|
||||||
|
<span><kbd>o</kbd> open</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<script>
|
||||||
|
const cards = [...document.querySelectorAll('article')];
|
||||||
|
let cur = 0;
|
||||||
|
const select = i => {
|
||||||
|
if (!cards.length) return;
|
||||||
|
cur = Math.max(0, Math.min(i, cards.length - 1));
|
||||||
|
cards.forEach(c => c.classList.remove('sel'));
|
||||||
|
cards[cur].classList.add('sel');
|
||||||
|
cards[cur].scrollIntoView({block: 'nearest', behavior: 'smooth'});
|
||||||
|
};
|
||||||
|
|
||||||
|
async function label(card, kind) {
|
||||||
|
const btns = card.querySelectorAll('button');
|
||||||
|
btns.forEach(b => b.disabled = true);
|
||||||
|
try {
|
||||||
|
const r = await fetch(`/ui/feedback/${card.dataset.id}/${kind}`, {method: 'POST'});
|
||||||
|
if (!r.ok) throw new Error(r.status);
|
||||||
|
card.classList.add('done');
|
||||||
|
// Advance, so a run of judgements is one keystroke each.
|
||||||
|
if (cards[cur] === card) select(cur + 1);
|
||||||
|
} catch (e) {
|
||||||
|
btns.forEach(b => b.disabled = false);
|
||||||
|
card.style.borderColor = 'var(--neg)';
|
||||||
|
console.error('feedback failed', e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
cards.forEach((card, i) => {
|
||||||
|
card.addEventListener('click', () => select(i));
|
||||||
|
card.querySelectorAll('button').forEach(b =>
|
||||||
|
b.addEventListener('click', ev => { ev.stopPropagation(); label(card, b.dataset.kind); }));
|
||||||
|
});
|
||||||
|
|
||||||
|
document.addEventListener('keydown', ev => {
|
||||||
|
if (ev.metaKey || ev.ctrlKey || ev.altKey) return;
|
||||||
|
const card = cards[cur];
|
||||||
|
switch (ev.key) {
|
||||||
|
case 'j': select(cur + 1); break;
|
||||||
|
case 'k': select(cur - 1); break;
|
||||||
|
case 'i': if (card) label(card, 'interested'); break;
|
||||||
|
case 'n': if (card) label(card, 'not_interested'); break;
|
||||||
|
case 'o': if (card) card.querySelector('h2 a')?.click(); break;
|
||||||
|
default: return;
|
||||||
|
}
|
||||||
|
ev.preventDefault();
|
||||||
|
});
|
||||||
|
|
||||||
|
select(0);
|
||||||
|
</script>
|
||||||
@@ -0,0 +1,136 @@
|
|||||||
|
<!doctype html>
|
||||||
|
<meta charset="utf-8">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||||
|
<title>Tune</title>
|
||||||
|
<style>
|
||||||
|
:root { color-scheme: light dark; --fg:#16181d; --bg:#fdfdfc; --muted:#6b7280;
|
||||||
|
--rule:#e4e4e7; --accent:#7c3aed; --card:#fff; --neg:#b91c1c; }
|
||||||
|
@media (prefers-color-scheme: dark) {
|
||||||
|
:root { --fg:#e8e8ea; --bg:#141416; --muted:#9ca3af; --rule:#2a2a2e;
|
||||||
|
--accent:#a78bfa; --card:#1b1b1f; --neg:#f87171; }
|
||||||
|
}
|
||||||
|
* { box-sizing:border-box; }
|
||||||
|
body { margin:0; background:var(--bg); color:var(--fg);
|
||||||
|
font:15px/1.55 -apple-system, BlinkMacSystemFont, "Segoe UI", system-ui, sans-serif; }
|
||||||
|
main { max-width:54rem; margin:0 auto; padding:1.25rem 1rem 4rem; }
|
||||||
|
header { display:flex; gap:1rem; align-items:baseline; justify-content:space-between;
|
||||||
|
border-bottom:1px solid var(--rule); padding-bottom:.6rem; margin-bottom:1rem; }
|
||||||
|
h1 { font-size:1.15rem; margin:0; }
|
||||||
|
a { color:var(--accent); }
|
||||||
|
form { background:var(--card); border:1px solid var(--rule); border-radius:9px;
|
||||||
|
padding:.9rem 1rem; margin-bottom:1.1rem; }
|
||||||
|
.grid { display:grid; grid-template-columns:repeat(auto-fit,minmax(13rem,1fr)); gap:.9rem; }
|
||||||
|
label { display:block; font-size:.8rem; color:var(--muted); margin-bottom:.2rem; }
|
||||||
|
input[type=range] { width:100%; accent-color:var(--accent); }
|
||||||
|
input[type=number] { width:5rem; font:inherit; background:transparent; color:inherit;
|
||||||
|
border:1px solid var(--rule); border-radius:5px; padding:.15rem .35rem; }
|
||||||
|
.val { font-variant-numeric:tabular-nums; color:var(--fg); font-size:.85rem; }
|
||||||
|
.headline { display:flex; gap:1.5rem; flex-wrap:wrap; align-items:baseline;
|
||||||
|
margin:.9rem 0 1.2rem; }
|
||||||
|
.big { font-size:1.9rem; font-weight:650; letter-spacing:-.02em; }
|
||||||
|
.big small { font-size:.8rem; font-weight:400; color:var(--muted); margin-left:.3rem; }
|
||||||
|
h2 { font-size:.82rem; text-transform:uppercase; letter-spacing:.05em;
|
||||||
|
color:var(--muted); margin:1.4rem 0 .5rem; }
|
||||||
|
li { list-style:none; padding:.3rem 0; border-bottom:1px solid var(--rule);
|
||||||
|
font-size:.9rem; display:flex; gap:.6rem; }
|
||||||
|
ul { padding:0; margin:0; }
|
||||||
|
.sc { font-variant-numeric:tabular-nums; color:var(--muted); min-width:3rem; }
|
||||||
|
.lane { color:var(--accent); font-size:.72rem; text-transform:uppercase;
|
||||||
|
letter-spacing:.04em; min-width:5.5rem; }
|
||||||
|
.lane.imp { color:var(--neg); }
|
||||||
|
.note { color:var(--muted); font-size:.82rem; }
|
||||||
|
</style>
|
||||||
|
<main>
|
||||||
|
<header>
|
||||||
|
<h1>Tune</h1>
|
||||||
|
<div class="note"><a href="/">triage</a> · <a href="/digest">digest</a></div>
|
||||||
|
</header>
|
||||||
|
|
||||||
|
<form method="get">
|
||||||
|
<div class="grid">
|
||||||
|
<div>
|
||||||
|
<label>interest threshold <span class="val" id="v1">{{ routing.interest_threshold }}</span></label>
|
||||||
|
<input type="range" name="interest_threshold" min="0" max="6" step="0.1"
|
||||||
|
value="{{ routing.interest_threshold }}" oninput="v1.textContent=this.value">
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<label>importance threshold <span class="val" id="v2">{{ routing.important_threshold }}</span></label>
|
||||||
|
<input type="range" name="important_threshold" min="0" max="6" step="0.1"
|
||||||
|
value="{{ routing.important_threshold }}" oninput="v2.textContent=this.value">
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<label>digest floor <span class="val" id="v3">{{ routing.digest_floor }}</span></label>
|
||||||
|
<input type="range" name="digest_floor" min="-2" max="4" step="0.1"
|
||||||
|
value="{{ routing.digest_floor }}" oninput="v3.textContent=this.value">
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<label>interest alerts / day</label>
|
||||||
|
<input type="number" name="interest_per_day" min="0" max="20"
|
||||||
|
value="{{ routing.interest_per_day }}">
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<label>importance alerts / day</label>
|
||||||
|
<input type="number" name="important_per_day" min="0" max="20"
|
||||||
|
value="{{ routing.important_per_day }}">
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<label>digest size</label>
|
||||||
|
<input type="number" name="digest_max_items" min="1" max="100"
|
||||||
|
value="{{ routing.digest_max_items }}">
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<label>max alerts per subject / day</label>
|
||||||
|
<input type="number" name="max_alerts_per_group_per_day" min="1" max="10"
|
||||||
|
value="{{ routing.max_alerts_per_group_per_day }}">
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<p style="margin:.9rem 0 0"><button>Simulate</button>
|
||||||
|
<span class="note">· over the last {{ '%.0f'|format(days * 24) }}h of clusters</span></p>
|
||||||
|
</form>
|
||||||
|
|
||||||
|
<div class="headline">
|
||||||
|
<div><div class="big">{{ '%.1f'|format(per_day) }}<small>notifications/day</small></div></div>
|
||||||
|
<div class="note">
|
||||||
|
{{ counts['alert:important'] }} importance · {{ counts['alert:interest'] }} interest ·
|
||||||
|
{{ counts['digest'] }} digest · {{ counts['drop'] }} dropped
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<h2>Would notify</h2>
|
||||||
|
<ul>
|
||||||
|
{% for a in alerts %}
|
||||||
|
<li><span class="lane {{ 'imp' if 'important' in a.lane }}">{{ a.lane.replace('alert:','') }}</span>
|
||||||
|
<span class="sc">{{ '%.2f'|format(a.score) }}</span>
|
||||||
|
<span>{{ a.headline }}</span></li>
|
||||||
|
{% else %}
|
||||||
|
<li class="note">Nothing clears these thresholds.</li>
|
||||||
|
{% endfor %}
|
||||||
|
</ul>
|
||||||
|
|
||||||
|
<h2>Would go to the digest</h2>
|
||||||
|
<ul>
|
||||||
|
{% for d in digest %}
|
||||||
|
<li><span class="sc">{{ '%.2f'|format(d.score) }}</span>
|
||||||
|
<span>{{ d.headline }}</span>
|
||||||
|
<span class="note">{{ d.reason if d.reason != 'digest' else '' }}</span></li>
|
||||||
|
{% endfor %}
|
||||||
|
</ul>
|
||||||
|
|
||||||
|
<p class="note" style="margin-top:1.5rem">
|
||||||
|
Sliders are a simulation only — nothing here writes to <code>config.toml</code>.
|
||||||
|
</p>
|
||||||
|
<p class="note">
|
||||||
|
Three constraints compete, and which one actually binds is not obvious.
|
||||||
|
Measured on this corpus: at the shipped budgets (3 interest, 2 importance)
|
||||||
|
the <strong>budget</strong> binds and the subject quota is inert — moving the
|
||||||
|
quota from 1 to 3 changes nothing. Raise the budgets to 8 and the
|
||||||
|
<strong>quota</strong> takes over, at which point 1 / 2 / 3 yields 10 / 11 /
|
||||||
|
13 alerts. The <strong>threshold</strong> binds last in both cases: there is
|
||||||
|
almost always more above-threshold news than there are slots, so it works as
|
||||||
|
a quality floor for slow days rather than as a volume control.
|
||||||
|
</p>
|
||||||
|
<p class="note">
|
||||||
|
So to get <em>more</em> notifications, raise the budgets first. To get more
|
||||||
|
<em>variety</em> at the same volume, lower the budgets and raise the quota.
|
||||||
|
</p>
|
||||||
|
</main>
|
||||||
+151
@@ -28,6 +28,22 @@ def _con():
|
|||||||
return store.connect(Config().db_path)
|
return store.connect(Config().db_path)
|
||||||
|
|
||||||
|
|
||||||
|
# Scoring a page load is cheap (matrix ops over ~1,300 clusters), but building
|
||||||
|
# ProfileVectors loads the embedding model. Cached so only the first request
|
||||||
|
# pays for it.
|
||||||
|
_VECTORS = None
|
||||||
|
|
||||||
|
|
||||||
|
def _scored(con, cfg):
|
||||||
|
global _VECTORS
|
||||||
|
from .embed import Embedder
|
||||||
|
from .score import ProfileVectors, score_clusters
|
||||||
|
|
||||||
|
if _VECTORS is None:
|
||||||
|
_VECTORS = ProfileVectors(con, cfg.profile(), Embedder.from_config(cfg))
|
||||||
|
return score_clusters(con, _VECTORS, cfg.scoring, cfg.clustering)
|
||||||
|
|
||||||
|
|
||||||
async def healthz(request):
|
async def healthz(request):
|
||||||
return PlainTextResponse("ok")
|
return PlainTextResponse("ok")
|
||||||
|
|
||||||
@@ -60,6 +76,138 @@ async def feedback(request):
|
|||||||
return JSONResponse({"ok": True, "cluster_id": cluster_id, "kind": kind})
|
return JSONResponse({"ok": True, "cluster_id": cluster_id, "kind": kind})
|
||||||
|
|
||||||
|
|
||||||
|
async def triage(request):
|
||||||
|
"""The labelling machine.
|
||||||
|
|
||||||
|
This is where feedback volume comes from, so it is built to be fast to skim
|
||||||
|
and label: ranked, keyboard-driven, one keystroke per judgement. The old
|
||||||
|
system's UI could only be clicked, and its clicks did not persist.
|
||||||
|
"""
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
from .route import route
|
||||||
|
|
||||||
|
cfg = Config()
|
||||||
|
con = _con()
|
||||||
|
scores = _scored(con, cfg)
|
||||||
|
decisions = {d.cluster_id: d for d in
|
||||||
|
route(con, scores, cfg.routing, cfg.delivery,
|
||||||
|
datetime.now(UTC), cfg.block_groups())}
|
||||||
|
|
||||||
|
labelled = {r["cluster_id"] for r in con.execute("SELECT DISTINCT cluster_id FROM feedback")}
|
||||||
|
show = request.query_params.get("show", "unlabelled")
|
||||||
|
limit = min(int(request.query_params.get("limit", 60)), 200)
|
||||||
|
|
||||||
|
ranked = sorted(scores, key=lambda s: -max(s.interest_score, s.importance_score))
|
||||||
|
rows = []
|
||||||
|
for s in ranked:
|
||||||
|
if s.veto:
|
||||||
|
continue
|
||||||
|
if show == "unlabelled" and s.cluster_id in labelled:
|
||||||
|
continue
|
||||||
|
d = decisions.get(s.cluster_id)
|
||||||
|
members = con.execute(
|
||||||
|
"""SELECT i.title, i.url, so.name FROM cluster_members cm
|
||||||
|
JOIN items i ON i.id = cm.item_id
|
||||||
|
JOIN sources so ON so.id = i.source_id
|
||||||
|
WHERE cm.cluster_id = ? LIMIT 5""", (s.cluster_id,)).fetchall()
|
||||||
|
rows.append({
|
||||||
|
"id": s.cluster_id, "headline": s.lead_title,
|
||||||
|
"topic": s.interest.block, "importance_topic": s.importance.block,
|
||||||
|
"interest": s.interest_score, "importance": s.importance_score,
|
||||||
|
"origins": s.corroboration.origins, "demote": s.demote,
|
||||||
|
"lane": d.lane if d else "?", "reason": d.reason if d else "",
|
||||||
|
"labelled": s.cluster_id in labelled,
|
||||||
|
"sources": [{"name": m["name"], "title": m["title"], "url": m["url"]}
|
||||||
|
for m in members],
|
||||||
|
})
|
||||||
|
if len(rows) >= limit:
|
||||||
|
break
|
||||||
|
|
||||||
|
return TEMPLATES.TemplateResponse(
|
||||||
|
request, "triage.html",
|
||||||
|
{"rows": rows, "show": show, "total": len(scores), "labelled": len(labelled)})
|
||||||
|
|
||||||
|
|
||||||
|
async def web_feedback(request):
|
||||||
|
"""Feedback from the browser.
|
||||||
|
|
||||||
|
Deliberately NOT under /api/feedback/, which Caddy routes around Authelia
|
||||||
|
for the ntfy buttons. This path stays behind the identity gateway, so it
|
||||||
|
needs no signature.
|
||||||
|
"""
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
kind = request.path_params["kind"]
|
||||||
|
if kind not in ("interested", "not_interested"):
|
||||||
|
return JSONResponse({"error": "bad kind"}, status_code=400)
|
||||||
|
con = _con()
|
||||||
|
con.execute(
|
||||||
|
"INSERT INTO feedback (cluster_id, kind, source, created_at) VALUES (?,?,?,?)",
|
||||||
|
(request.path_params["cluster_id"], kind, "web", datetime.now(UTC).isoformat()))
|
||||||
|
return JSONResponse({"ok": True})
|
||||||
|
|
||||||
|
|
||||||
|
async def tune(request):
|
||||||
|
"""Threshold tuning against real history.
|
||||||
|
|
||||||
|
The old system's replay harness hardcoded its parameters, so it could never
|
||||||
|
sweep the thresholds it existed to tune. Here the sliders drive the same
|
||||||
|
`route()` the live service calls, so what you see is what it would have sent.
|
||||||
|
"""
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
from .route import DIGEST, DROP, IMPORTANT, INTEREST, route
|
||||||
|
|
||||||
|
cfg = Config()
|
||||||
|
con = _con()
|
||||||
|
scores = _scored(con, cfg)
|
||||||
|
|
||||||
|
routing = dict(cfg.routing)
|
||||||
|
for key in ("interest_threshold", "important_threshold", "digest_floor"):
|
||||||
|
if (raw := request.query_params.get(key)) is not None:
|
||||||
|
try:
|
||||||
|
routing[key] = float(raw)
|
||||||
|
except ValueError:
|
||||||
|
pass
|
||||||
|
for key in ("interest_per_day", "important_per_day", "digest_max_items",
|
||||||
|
"max_alerts_per_group_per_day"):
|
||||||
|
if (raw := request.query_params.get(key)) is not None:
|
||||||
|
try:
|
||||||
|
routing[key] = int(raw)
|
||||||
|
except ValueError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
decisions = route(con, scores, routing, cfg.delivery, now, cfg.block_groups())
|
||||||
|
by_id = {s.cluster_id: s for s in scores}
|
||||||
|
lanes: dict[str, list] = {}
|
||||||
|
for d in decisions:
|
||||||
|
lanes.setdefault(d.lane, []).append(d)
|
||||||
|
|
||||||
|
days = routing["max_age_hours"] / 24.0
|
||||||
|
counts = {lane: len(lanes.get(lane, [])) for lane in (IMPORTANT, INTEREST, DIGEST, DROP)}
|
||||||
|
alerts = [
|
||||||
|
{"lane": d.lane, "reason": d.reason, "score": d.score,
|
||||||
|
"headline": by_id[d.cluster_id].lead_title,
|
||||||
|
"topic": by_id[d.cluster_id].interest.block}
|
||||||
|
for d in sorted(lanes.get(IMPORTANT, []) + lanes.get(INTEREST, []),
|
||||||
|
key=lambda d: -d.score)
|
||||||
|
]
|
||||||
|
digest = [
|
||||||
|
{"reason": d.reason, "score": d.score,
|
||||||
|
"headline": by_id[d.cluster_id].lead_title,
|
||||||
|
"topic": by_id[d.cluster_id].interest.block}
|
||||||
|
for d in sorted(lanes.get(DIGEST, []), key=lambda d: -d.score)[:25]
|
||||||
|
]
|
||||||
|
|
||||||
|
return TEMPLATES.TemplateResponse(
|
||||||
|
request, "tune.html",
|
||||||
|
{"routing": routing, "counts": counts, "alerts": alerts, "digest": digest,
|
||||||
|
"per_day": (counts[IMPORTANT] + counts[INTEREST]) / days, "days": days,
|
||||||
|
"defaults": cfg.routing})
|
||||||
|
|
||||||
|
|
||||||
async def digest_page(request):
|
async def digest_page(request):
|
||||||
day = request.path_params.get("day")
|
day = request.path_params.get("day")
|
||||||
try:
|
try:
|
||||||
@@ -107,6 +255,9 @@ async def digest_page(request):
|
|||||||
|
|
||||||
|
|
||||||
routes = [
|
routes = [
|
||||||
|
Route("/", triage),
|
||||||
|
Route("/ui/feedback/{cluster_id}/{kind}", web_feedback, methods=["POST"]),
|
||||||
|
Route("/tune", tune),
|
||||||
Route("/healthz", healthz),
|
Route("/healthz", healthz),
|
||||||
Route("/api/feedback/{cluster_id}/{kind}/{token}", feedback, methods=["POST"]),
|
Route("/api/feedback/{cluster_id}/{kind}/{token}", feedback, methods=["POST"]),
|
||||||
Route("/digest", digest_page),
|
Route("/digest", digest_page),
|
||||||
|
|||||||
Reference in new issue
Block a user