diff --git a/deploy/README.md b/deploy/README.md index 58ed5f4..9e4f68e 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -127,12 +127,47 @@ sudo systemctl disable --now news-update.timer | `POST /api/feedback/.../bogus` | `{"error":"bad signature"}` | reaches the app, unauthenticated, and rejects a forgery | | `POST /api/feedback/.../` | `{"ok":true}` and a row in `feedback` | **the loop that never once worked in v0.2** | -## 3. Scheduling +## 3. Scheduling — APPLIED 2026-09-08 +Two systemd timers, units in this directory: + +| unit | cadence | does | +|---|---|---| +| `news-cycle.timer` | every 15 min | `poll` then `notify` | +| `news-digest.timer` | daily 08:45 | `digest` | + +```sh +sudo cp deploy/news-*.service deploy/news-*.timer /etc/systemd/system/ +sudo systemctl daemon-reload +sudo systemctl enable --now news-cycle.timer news-digest.timer ``` -*/15 * * * * podman exec news-backend python -m news.cli poll -*/20 * * * * podman exec news-backend python -m news.cli notify -45 8 * * * podman exec news-backend python -m news.cli digest + +**Copy the units, do not symlink them.** The previous service's units were +symlinks into `docs/projects/news/deploy/`, and when that repo was renamed to +`news_triage` they became dangling and the timer failed every ten minutes, +unnoticed, for days. The cost of copying is remembering to re-copy after editing +a unit here; the cost of symlinking is a silent failure the next time a +directory moves. + +Design notes: + +- `news-cycle.service` has two `ExecStart` lines and the first is prefixed `-`, + so a failed fetch does not skip `notify` -- there may still be ripe clusters + from an earlier cycle worth sending. +- `RandomizedDelaySec=90` on the cycle: feeds get polled by everyone on the + quarter hour, and this both spreads that load and avoids hitting all 56 + sources at the same instant. +- `Persistent=true` on the digest, so a machine that was off at 08:45 still + sends the briefing on the next boot rather than silently skipping a day. +- `TimeoutStartSec=20min` on the cycle: a cold start embeds the whole backlog at + ~11 items/sec. Steady state is far quicker -- a real incremental cycle + measured 74s for 95 new items. + +Reading the logs is noisier than it should be, because podman writes its own +container-exec events into the same unit journal: + +```sh +journalctl -u news-cycle -o cat | grep -vE '^[0-9]{4}-[0-9]{2}-[0-9]{2} .* container ' ``` ## 4. Verifying it actually works diff --git a/deploy/news-cycle.service b/deploy/news-cycle.service new file mode 100644 index 0000000..dbf2eff --- /dev/null +++ b/deploy/news-cycle.service @@ -0,0 +1,15 @@ +[Unit] +Description=news: fetch, embed, cluster, then send any alerts +Documentation=file:///home/connor/docs/projects/news/deploy/README.md +After=network-online.target +Wants=network-online.target + +[Service] +Type=oneshot +# `-` on the poll so a failed fetch does not skip notify: there may still be +# ripe clusters from an earlier cycle worth sending. +ExecStart=-/usr/bin/podman exec connor_news-backend_1 python -m news.cli poll +ExecStart=/usr/bin/podman exec connor_news-backend_1 python -m news.cli notify +# A cold cycle embeds the whole backlog at ~11 items/sec, so it can legitimately +# run for minutes. Steady state is far quicker; this is a ceiling, not a target. +TimeoutStartSec=20min diff --git a/deploy/news-cycle.timer b/deploy/news-cycle.timer new file mode 100644 index 0000000..4f80ad5 --- /dev/null +++ b/deploy/news-cycle.timer @@ -0,0 +1,13 @@ +[Unit] +Description=news: run a poll/notify cycle every 15 minutes + +[Timer] +OnBootSec=3min +OnUnitActiveSec=15min +# Feeds are polled by many people on the quarter hour; spread the load and stay +# a good citizen. Also avoids every source being hit at the same instant. +RandomizedDelaySec=90 +AccuracySec=30s + +[Install] +WantedBy=timers.target diff --git a/deploy/news-digest.service b/deploy/news-digest.service new file mode 100644 index 0000000..8d99ae0 --- /dev/null +++ b/deploy/news-digest.service @@ -0,0 +1,11 @@ +[Unit] +Description=news: write the daily briefing and push a pointer to it +Documentation=file:///home/connor/docs/projects/news/deploy/README.md +After=network-online.target +Wants=network-online.target + +[Service] +Type=oneshot +ExecStart=/usr/bin/podman exec connor_news-backend_1 python -m news.cli digest +# One LLM call, which can take ~20s; allow for a slow upstream. +TimeoutStartSec=5min diff --git a/deploy/news-digest.timer b/deploy/news-digest.timer new file mode 100644 index 0000000..dcb6e78 --- /dev/null +++ b/deploy/news-digest.timer @@ -0,0 +1,12 @@ +[Unit] +Description=news: daily briefing at 08:45 local + +[Timer] +OnCalendar=*-*-* 08:45:00 +# If the machine was asleep or down at 08:45, still send it on the next boot +# rather than silently skipping the day. +Persistent=true +AccuracySec=1min + +[Install] +WantedBy=timers.target