182 lines
7.4 KiB
Plaintext
182 lines
7.4 KiB
Plaintext
F3 Final Verification - news-triage
|
|
====================================
|
|
Reviewer: F3 manual end-to-end QA
|
|
Date: 2026-09-01
|
|
Workspace: /home/connor/docs/projects/news
|
|
Scratch environment: /tmp/opencode/news-triage-scratch
|
|
Target evidence file: /home/connor/docs/projects/news/.omo/evidence/f3-news-triage.txt
|
|
|
|
Verdict: REJECT
|
|
|
|
Reason for rejection: A real functional bug was discovered during end-to-end
|
|
verification. When a blocklisted source is present in the configuration, the
|
|
backend serializes NaN relevance values as JSON null in /api/stories. The
|
|
frontend expects `relevance: f64` and fails to deserialize the response,
|
|
rendering the entire story list unusable. This breaks the core user journey.
|
|
|
|
The bug was worked around for testing by removing the blocklisted source from
|
|
the scratch configuration, but the underlying defect remains in the codebase
|
|
and must be fixed before release.
|
|
|
|
Environment
|
|
-----------
|
|
- Container image reused: news-triage:test (built previously; `podman image exists news-triage:test` returned true).
|
|
- Host binary rebuilt during F3: `cargo build -p news-server --locked` succeeded.
|
|
- Mock ntfy server: http://127.0.0.1:31337 (Python http.server logging POSTs to /news-triage).
|
|
- Fixture server: http://127.0.0.1:31338 (Python http.server serving /tmp/opencode/news-triage-scratch/fixtures).
|
|
- UI proxy server: http://127.0.0.1:8080 (serves crates/news-web/dist and proxies /api to backend).
|
|
- Backend container: http://127.0.0.1:3001 (network_mode: host).
|
|
- Database: sqlite::memory: for dry-run tests; container uses mounted news.db.
|
|
|
|
Test fixtures
|
|
-------------
|
|
- /tmp/opencode/news-triage-scratch/fixtures/source-a-rss.xml (RSS, recent-dated stories)
|
|
- /tmp/opencode/news-triage-scratch/fixtures/source-d-rss.xml (RSS)
|
|
- /tmp/opencode/news-triage-scratch/fixtures/source-b-sitemap.xml (news-sitemap)
|
|
- source-c-rss.xml (celebrity gossip) was created but removed from active configs
|
|
after the relevance/NaN bug was discovered.
|
|
|
|
Configurations used
|
|
-------------------
|
|
1. /tmp/opencode/news-triage-scratch/config/config.toml
|
|
- Enabled sources: source-a, source-b, source-d (source-c removed).
|
|
- Used for containerized backend + UI verification.
|
|
|
|
2. /tmp/opencode/news-triage-scratch/config/config-dryrun.toml
|
|
- Default sources (Al Jazeera, BBC, AP) explicitly disabled.
|
|
- Only source-a enabled (http://127.0.0.1:31338/source-a-rss.xml).
|
|
- Used for `news-server --dry-run --once` verification.
|
|
|
|
Backend API verification
|
|
------------------------
|
|
All checks passed against the running container:
|
|
|
|
$ curl -s http://127.0.0.1:3001/healthz
|
|
ok
|
|
|
|
$ curl -s http://127.0.0.1:3001/metrics | head -3
|
|
# HELP news_uptime_seconds Process uptime
|
|
# TYPE news_uptime_seconds gauge
|
|
news_uptime_seconds 123
|
|
|
|
$ curl -s http://127.0.0.1:3001/api/config
|
|
{"sources":[{"name":"Source A","url":"http://127.0.0.1:31338/source-a-rss.xml",...},...]}
|
|
|
|
$ curl -s http://127.0.0.1:3001/api/stories
|
|
[{"id":"...","title":"Major trade deal signed at summit",...},...]
|
|
|
|
$ curl -s -X POST http://127.0.0.1:3001/api/feedback \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"story_id":"test-123","verdict":"interested"}'
|
|
{"status":"recorded"}
|
|
|
|
Ntfy notification verification
|
|
------------------------------
|
|
Mock ntfy log captured 3 bypass-lane POSTs when stories exceeded the bypass
|
|
importance threshold. Each request included:
|
|
- Title matching the story headline
|
|
- Priority: 5
|
|
- Tags: newspaper
|
|
- Authorization: Basic <token>
|
|
|
|
Sample captured request (verbatim from mock-ntfy.log):
|
|
|
|
POST /news-triage HTTP/1.1
|
|
Host: 127.0.0.1:31337
|
|
Authorization: Basic bmV3cy10cmlhZ2U6bXktc2VjcmV0LWFwaS1rZXk=
|
|
Content-Type: application/json
|
|
Content-Length: ...
|
|
|
|
{"title":"Major trade deal signed at summit","message":"...","priority":5,"tags":["newspaper"]}
|
|
|
|
This confirms the notification publisher correctly authenticates and formats
|
|
bypass notifications.
|
|
|
|
UI browser verification
|
|
-----------------------
|
|
- Story list rendered at least one row.
|
|
- Clicked the "Interested" button on a story.
|
|
- Browser dev tools confirmed POST /api/feedback returned 200 with
|
|
{"status":"recorded"}.
|
|
- Screenshot saved: /home/connor/docs/projects/news/.omo/evidence/ui-interested.png
|
|
- After stopping the backend container and reloading the page, the UI showed
|
|
"Could not load stories" with an HTTP 502 from the proxy.
|
|
- Screenshot saved: /home/connor/docs/projects/news/.omo/evidence/ui-error.png
|
|
|
|
Dry-run verification
|
|
--------------------
|
|
Command:
|
|
|
|
$ : > /tmp/opencode/news-triage-scratch/mock-ntfy.log
|
|
$ RUST_LOG=info \
|
|
NEWS_DATABASE_URL=sqlite::memory: \
|
|
NEWS_CONFIG_PATH=/tmp/opencode/news-triage-scratch/config/config-dryrun.toml \
|
|
NTFY_URL=http://127.0.0.1:31337 \
|
|
timeout 30 ./target/debug/news-server --dry-run --once
|
|
|
|
Result:
|
|
- exit=0
|
|
- /tmp/opencode/news-triage-scratch/mock-ntfy.log contained 0 entries.
|
|
- Only the fixture stories were processed (default real sources were disabled
|
|
in config):
|
|
|
|
Major trade deal signed at summit 2.400 1.187 83.333 Digest
|
|
Coastal storm forces evacuations 2.400 0.730 50.000 Suppress:BelowThreshold
|
|
Election officials certify final results 2.400 0.689 16.667 Suppress:BelowThreshold
|
|
|
|
This confirms `--dry-run` does not emit ntfy notifications.
|
|
|
|
Replay determinism verification
|
|
-------------------------------
|
|
Command:
|
|
|
|
$ ./target/debug/news-cli replay crates/news-cli/tests/fixtures/sample-week > replay1b.out
|
|
$ ./target/debug/news-cli replay crates/news-cli/tests/fixtures/sample-week > replay2b.out
|
|
$ cmp replay1b.out replay2b.out && echo identical
|
|
identical
|
|
|
|
Result: byte-identical output across two replay runs.
|
|
|
|
Bug discovered: NaN relevance for blocklisted items
|
|
---------------------------------------------------
|
|
When source-c (celebrity gossip) was enabled, the backend produced a story
|
|
whose title contained blocklist keywords. The relevance scorer returned
|
|
`f64::NEG_INFINITY` or `NaN` for the relevance field. The `/api/stories`
|
|
endpoint serialized this as JSON `null`.
|
|
|
|
Frontend error (from browser console):
|
|
|
|
could not decode response: invalid type: null, expected f64
|
|
|
|
This comes from:
|
|
- Frontend: crates/news-web/src/api.rs defines `StoryRow { relevance: f64 }`.
|
|
- Backend: crates/news-server/src/api/stories.rs returns the raw f64 value
|
|
without replacing NaN/Infinity.
|
|
|
|
Impact: any configuration that includes a blocklisted source renders the
|
|
entire /api/stories response undecodable by the frontend. The UI shows a
|
|
blank or error state and the user cannot interact with stories at all.
|
|
|
|
Workaround used for remaining tests: removed source-c from the scratch
|
|
config. This is NOT a fix; the code still has the defect.
|
|
|
|
Additional observations
|
|
-----------------------
|
|
- Default sources are always seeded as enabled by `SourceRepo::seed_default_sources()`
|
|
(crates/news-store/src/sources.rs). To run an offline test, the operator must
|
|
explicitly disable Al Jazeera, BBC, and AP in config. This is surprising but
|
|
documented behavior.
|
|
- Initial F3 dry-run attempts with an outdated host binary hung silently at
|
|
startup when configured with localhost fixture URLs. Rebuilding with
|
|
`cargo build -p news-server --locked` resolved the hang.
|
|
|
|
Conclusion
|
|
----------
|
|
The notification path, API surface, container health, and dry-run mode all
|
|
function as intended. UI interaction works under the workaround config.
|
|
However, the NaN/null relevance bug is a showstopper for the end-to-end user
|
|
experience whenever a blocklisted source is present. F3 therefore rejects the
|
|
release until the bug is fixed and re-verified.
|
|
|
|
F3 Verdict: REJECT
|