Initial commit -- did OMO do a good job?
This commit is contained in:
@@ -0,0 +1,181 @@
|
||||
F3 Final Verification - news-triage
|
||||
====================================
|
||||
Reviewer: F3 manual end-to-end QA
|
||||
Date: 2026-09-01
|
||||
Workspace: /home/connor/docs/projects/news
|
||||
Scratch environment: /tmp/opencode/news-triage-scratch
|
||||
Target evidence file: /home/connor/docs/projects/news/.omo/evidence/f3-news-triage.txt
|
||||
|
||||
Verdict: REJECT
|
||||
|
||||
Reason for rejection: A real functional bug was discovered during end-to-end
|
||||
verification. When a blocklisted source is present in the configuration, the
|
||||
backend serializes NaN relevance values as JSON null in /api/stories. The
|
||||
frontend expects `relevance: f64` and fails to deserialize the response,
|
||||
rendering the entire story list unusable. This breaks the core user journey.
|
||||
|
||||
The bug was worked around for testing by removing the blocklisted source from
|
||||
the scratch configuration, but the underlying defect remains in the codebase
|
||||
and must be fixed before release.
|
||||
|
||||
Environment
|
||||
-----------
|
||||
- Container image reused: news-triage:test (built previously; `podman image exists news-triage:test` returned true).
|
||||
- Host binary rebuilt during F3: `cargo build -p news-server --locked` succeeded.
|
||||
- Mock ntfy server: http://127.0.0.1:31337 (Python http.server logging POSTs to /news-triage).
|
||||
- Fixture server: http://127.0.0.1:31338 (Python http.server serving /tmp/opencode/news-triage-scratch/fixtures).
|
||||
- UI proxy server: http://127.0.0.1:8080 (serves crates/news-web/dist and proxies /api to backend).
|
||||
- Backend container: http://127.0.0.1:3001 (network_mode: host).
|
||||
- Database: sqlite::memory: for dry-run tests; container uses mounted news.db.
|
||||
|
||||
Test fixtures
|
||||
-------------
|
||||
- /tmp/opencode/news-triage-scratch/fixtures/source-a-rss.xml (RSS, recent-dated stories)
|
||||
- /tmp/opencode/news-triage-scratch/fixtures/source-d-rss.xml (RSS)
|
||||
- /tmp/opencode/news-triage-scratch/fixtures/source-b-sitemap.xml (news-sitemap)
|
||||
- source-c-rss.xml (celebrity gossip) was created but removed from active configs
|
||||
after the relevance/NaN bug was discovered.
|
||||
|
||||
Configurations used
|
||||
-------------------
|
||||
1. /tmp/opencode/news-triage-scratch/config/config.toml
|
||||
- Enabled sources: source-a, source-b, source-d (source-c removed).
|
||||
- Used for containerized backend + UI verification.
|
||||
|
||||
2. /tmp/opencode/news-triage-scratch/config/config-dryrun.toml
|
||||
- Default sources (Al Jazeera, BBC, AP) explicitly disabled.
|
||||
- Only source-a enabled (http://127.0.0.1:31338/source-a-rss.xml).
|
||||
- Used for `news-server --dry-run --once` verification.
|
||||
|
||||
Backend API verification
|
||||
------------------------
|
||||
All checks passed against the running container:
|
||||
|
||||
$ curl -s http://127.0.0.1:3001/healthz
|
||||
ok
|
||||
|
||||
$ curl -s http://127.0.0.1:3001/metrics | head -3
|
||||
# HELP news_uptime_seconds Process uptime
|
||||
# TYPE news_uptime_seconds gauge
|
||||
news_uptime_seconds 123
|
||||
|
||||
$ curl -s http://127.0.0.1:3001/api/config
|
||||
{"sources":[{"name":"Source A","url":"http://127.0.0.1:31338/source-a-rss.xml",...},...]}
|
||||
|
||||
$ curl -s http://127.0.0.1:3001/api/stories
|
||||
[{"id":"...","title":"Major trade deal signed at summit",...},...]
|
||||
|
||||
$ curl -s -X POST http://127.0.0.1:3001/api/feedback \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"story_id":"test-123","verdict":"interested"}'
|
||||
{"status":"recorded"}
|
||||
|
||||
Ntfy notification verification
|
||||
------------------------------
|
||||
Mock ntfy log captured 3 bypass-lane POSTs when stories exceeded the bypass
|
||||
importance threshold. Each request included:
|
||||
- Title matching the story headline
|
||||
- Priority: 5
|
||||
- Tags: newspaper
|
||||
- Authorization: Basic <token>
|
||||
|
||||
Sample captured request (verbatim from mock-ntfy.log):
|
||||
|
||||
POST /news-triage HTTP/1.1
|
||||
Host: 127.0.0.1:31337
|
||||
Authorization: Basic bmV3cy10cmlhZ2U6bXktc2VjcmV0LWFwaS1rZXk=
|
||||
Content-Type: application/json
|
||||
Content-Length: ...
|
||||
|
||||
{"title":"Major trade deal signed at summit","message":"...","priority":5,"tags":["newspaper"]}
|
||||
|
||||
This confirms the notification publisher correctly authenticates and formats
|
||||
bypass notifications.
|
||||
|
||||
UI browser verification
|
||||
-----------------------
|
||||
- Story list rendered at least one row.
|
||||
- Clicked the "Interested" button on a story.
|
||||
- Browser dev tools confirmed POST /api/feedback returned 200 with
|
||||
{"status":"recorded"}.
|
||||
- Screenshot saved: /home/connor/docs/projects/news/.omo/evidence/ui-interested.png
|
||||
- After stopping the backend container and reloading the page, the UI showed
|
||||
"Could not load stories" with an HTTP 502 from the proxy.
|
||||
- Screenshot saved: /home/connor/docs/projects/news/.omo/evidence/ui-error.png
|
||||
|
||||
Dry-run verification
|
||||
--------------------
|
||||
Command:
|
||||
|
||||
$ : > /tmp/opencode/news-triage-scratch/mock-ntfy.log
|
||||
$ RUST_LOG=info \
|
||||
NEWS_DATABASE_URL=sqlite::memory: \
|
||||
NEWS_CONFIG_PATH=/tmp/opencode/news-triage-scratch/config/config-dryrun.toml \
|
||||
NTFY_URL=http://127.0.0.1:31337 \
|
||||
timeout 30 ./target/debug/news-server --dry-run --once
|
||||
|
||||
Result:
|
||||
- exit=0
|
||||
- /tmp/opencode/news-triage-scratch/mock-ntfy.log contained 0 entries.
|
||||
- Only the fixture stories were processed (default real sources were disabled
|
||||
in config):
|
||||
|
||||
Major trade deal signed at summit 2.400 1.187 83.333 Digest
|
||||
Coastal storm forces evacuations 2.400 0.730 50.000 Suppress:BelowThreshold
|
||||
Election officials certify final results 2.400 0.689 16.667 Suppress:BelowThreshold
|
||||
|
||||
This confirms `--dry-run` does not emit ntfy notifications.
|
||||
|
||||
Replay determinism verification
|
||||
-------------------------------
|
||||
Command:
|
||||
|
||||
$ ./target/debug/news-cli replay crates/news-cli/tests/fixtures/sample-week > replay1b.out
|
||||
$ ./target/debug/news-cli replay crates/news-cli/tests/fixtures/sample-week > replay2b.out
|
||||
$ cmp replay1b.out replay2b.out && echo identical
|
||||
identical
|
||||
|
||||
Result: byte-identical output across two replay runs.
|
||||
|
||||
Bug discovered: NaN relevance for blocklisted items
|
||||
---------------------------------------------------
|
||||
When source-c (celebrity gossip) was enabled, the backend produced a story
|
||||
whose title contained blocklist keywords. The relevance scorer returned
|
||||
`f64::NEG_INFINITY` or `NaN` for the relevance field. The `/api/stories`
|
||||
endpoint serialized this as JSON `null`.
|
||||
|
||||
Frontend error (from browser console):
|
||||
|
||||
could not decode response: invalid type: null, expected f64
|
||||
|
||||
This comes from:
|
||||
- Frontend: crates/news-web/src/api.rs defines `StoryRow { relevance: f64 }`.
|
||||
- Backend: crates/news-server/src/api/stories.rs returns the raw f64 value
|
||||
without replacing NaN/Infinity.
|
||||
|
||||
Impact: any configuration that includes a blocklisted source renders the
|
||||
entire /api/stories response undecodable by the frontend. The UI shows a
|
||||
blank or error state and the user cannot interact with stories at all.
|
||||
|
||||
Workaround used for remaining tests: removed source-c from the scratch
|
||||
config. This is NOT a fix; the code still has the defect.
|
||||
|
||||
Additional observations
|
||||
-----------------------
|
||||
- Default sources are always seeded as enabled by `SourceRepo::seed_default_sources()`
|
||||
(crates/news-store/src/sources.rs). To run an offline test, the operator must
|
||||
explicitly disable Al Jazeera, BBC, and AP in config. This is surprising but
|
||||
documented behavior.
|
||||
- Initial F3 dry-run attempts with an outdated host binary hung silently at
|
||||
startup when configured with localhost fixture URLs. Rebuilding with
|
||||
`cargo build -p news-server --locked` resolved the hang.
|
||||
|
||||
Conclusion
|
||||
----------
|
||||
The notification path, API surface, container health, and dry-run mode all
|
||||
function as intended. UI interaction works under the workaround config.
|
||||
However, the NaN/null relevance bug is a showstopper for the end-to-end user
|
||||
experience whenever a blocklisted source is present. F3 therefore rejects the
|
||||
release until the bug is fixed and re-verified.
|
||||
|
||||
F3 Verdict: REJECT
|
||||
Reference in New Issue
Block a user