"Set Sail" — one-click full pipeline with progress tracking #37

New Issue

connor · 2026-03-18T11:40:45-04:00

connor commented

2026-03-18 11:40:45 -04:00

The core value proposition of Shanty is automation: the user adds music to their watchlist and the app handles everything else. Currently this requires manually triggering 4-5 separate actions. There should be a single "Set Sail" button that runs the entire pipeline.

The pipeline runs 5 steps sequentially: sync → download → index → tag → organize. Rather than a monolithic task with its own progress model, each step should be registered as a separate task in the existing task system. Steps that haven't started yet are marked with a
new Pending status, so the dashboard shows the full queue of work naturally — no special UI needed beyond what already exists.

Pipeline Steps

Sync — sync wanted items to the download queue
Download — process the download queue
Index — index newly downloaded files
Tag — auto-tag tracks that need metadata
Organize — organize tagged files into the library structure

Requirements

Backend: New task status Pending — add to the TaskStatus enum. A Pending task is one that will run as soon as tasks ahead of it complete. This is distinct from Running, Completed, and Failed. (A future Scheduled status can be added later for time-based scheduling,
but that's out of scope here.)
Backend: POST /api/pipeline endpoint that:
- Registers all 5 tasks at once, each with Pending status
- Returns the list of task IDs immediately
- Spawns an async runner that executes each task sequentially:
- Set task to Running, execute it, set to Completed or Failed
- Move to the next task
  - Skips phases with nothing to do (e.g., if sync returns 0 new items but there are already pending downloads, still proceed to download)
  - After completion, the individual task results contain their own stats (same as if triggered individually)
Frontend:
- Prominent "Set Sail" button on the dashboard (with a pirate-themed icon — ship wheel, anchor, or skull)
- No special progress UI needed — the existing "Background Tasks" section on the dashboard already shows all tasks with their status and progress. The 5 pipeline tasks will appear as a batch of Pending → Running → Completed rows.
- After completion, each task's result is visible in the normal task list
- Button should be disabled while a pipeline is already running
Smart behavior:
- Each step still runs even if the previous had "nothing to do" (e.g., sync returns 0 new items but downloads may already be queued from before)
- Failed steps don't block subsequent steps — if download fails on 2 of 14 tracks, indexing still runs on the 12 that succeeded
- After the final step, promote all Downloaded wanted items to Owned (already implemented in the organize step)

Acceptance Criteria

Pending status exists in TaskStatus and displays correctly on the dashboard
"Set Sail" button on the dashboard triggers the full pipeline
Backend registers 5 individual tasks and runs them sequentially
Each task appears in the dashboard's existing task list with normal progress/status
Pipeline tasks transition: Pending → Running → Completed/Failed
Pipeline handles "nothing to do" phases gracefully (marks complete, moves on)
Failed steps don't block subsequent steps
Button is disabled while a pipeline is in progress
Summary per-step is visible in the normal task result fields

Dependencies

All existing pipeline crates (shanty-dl, shanty-index, shanty-tag, shanty-org)
Existing task system (just needs Pending status added)

The core value proposition of Shanty is automation: the user adds music to their watchlist and the app handles everything else. Currently this requires manually triggering 4-5 separate actions. There should be a single "Set Sail" button that runs the entire pipeline. The pipeline runs 5 steps sequentially: sync → download → index → tag → organize. Rather than a monolithic task with its own progress model, each step should be registered as a separate task in the existing task system. Steps that haven't started yet are marked with a new Pending status, so the dashboard shows the full queue of work naturally — no special UI needed beyond what already exists. Pipeline Steps 1. Sync — sync wanted items to the download queue 2. Download — process the download queue 3. Index — index newly downloaded files 4. Tag — auto-tag tracks that need metadata 5. Organize — organize tagged files into the library structure Requirements 1. Backend: New task status Pending — add to the TaskStatus enum. A Pending task is one that will run as soon as tasks ahead of it complete. This is distinct from Running, Completed, and Failed. (A future Scheduled status can be added later for time-based scheduling, but that's out of scope here.) 2. Backend: POST /api/pipeline endpoint that: - Registers all 5 tasks at once, each with Pending status - Returns the list of task IDs immediately - Spawns an async runner that executes each task sequentially: - Set task to Running, execute it, set to Completed or Failed - Move to the next task - Skips phases with nothing to do (e.g., if sync returns 0 new items but there are already pending downloads, still proceed to download) - After completion, the individual task results contain their own stats (same as if triggered individually) 3. Frontend: - Prominent "Set Sail" button on the dashboard (with a pirate-themed icon — ship wheel, anchor, or skull) - No special progress UI needed — the existing "Background Tasks" section on the dashboard already shows all tasks with their status and progress. The 5 pipeline tasks will appear as a batch of Pending → Running → Completed rows. - After completion, each task's result is visible in the normal task list - Button should be disabled while a pipeline is already running 4. Smart behavior: - Each step still runs even if the previous had "nothing to do" (e.g., sync returns 0 new items but downloads may already be queued from before) - Failed steps don't block subsequent steps — if download fails on 2 of 14 tracks, indexing still runs on the 12 that succeeded - After the final step, promote all Downloaded wanted items to Owned (already implemented in the organize step) Acceptance Criteria - Pending status exists in TaskStatus and displays correctly on the dashboard - "Set Sail" button on the dashboard triggers the full pipeline - Backend registers 5 individual tasks and runs them sequentially - Each task appears in the dashboard's existing task list with normal progress/status - Pipeline tasks transition: Pending → Running → Completed/Failed - Pipeline handles "nothing to do" phases gracefully (marks complete, moves on) - Failed steps don't block subsequent steps - Button is disabled while a pipeline is in progress - Summary per-step is visible in the normal task result fields Dependencies - All existing pipeline crates (shanty-dl, shanty-index, shanty-tag, shanty-org) - Existing task system (just needs Pending status added)

connor added the MVP HighPriority labels 2026-03-18 11:40:45 -04:00

connor started working 2026-03-18 14:35:55 -04:00

connor closed this issue

2026-03-18 14:49:13 -04:00

connor worked for 13 minutes

2026-03-18 14:49:13 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Total Time Spent: 13 minutes

connor

13 minutes

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Shanty/Main#37