Shanty/Main
1
0
Fork 0
Code Issues 18 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
3f837647e0b460e6846e1877e943cc4ce46dbde9
Main/ISSUES.md
Connor Johnstone 3f837647e0 Added tagging
2026-03-17 15:01:35 -04:00

1176 lines
51 KiB
Markdown
Raw Blame History

# Shanty — Project Issues
This file contains all planned issues for the Shanty project. Each issue is self-contained and can be copy-pasted into Gitea as a separate issue. Issues are numbered for cross-referencing but your Gitea numbering may differ — adjust dependency references accordingly.
---
## Issue #7: Implement music downloading via yt-dlp in `shanty-dl`
**Labels:** `mvp`, `downloading`, `priority:high`
### Description
The `shanty-dl` crate is responsible for actually downloading music files. For the MVP, the only backend is `yt-dlp`, which is capable of downloading audio from YouTube, SoundCloud, Bandcamp, and hundreds of other sites. The user provides a specific song (or list of songs), and the crate handles downloading, converting to the desired format, and storing the result.
This issue covers:
1. **yt-dlp integration** — shell out to `yt-dlp` (assumed to be installed on the system) to download audio. The crate should:
- Accept a search query (e.g., "Pink Floyd Time") or a direct URL
- Use yt-dlp's search feature (`ytsearch:`) to find the best match when given a query
- Download as the best available audio quality
- Convert to a target format (default: MP3 320kbps or OPUS, configurable)
- Save to a configurable output directory
- Embed available metadata (title, artist) into the downloaded file
2. **Download queue** — integrate with the `download_queue` table in `shanty-db`:
- Items can be added to the queue (manually or from the watchlist's "wanted" items)
- The downloader processes the queue, updating status as it goes (`pending` → `downloading` → `completed` / `failed`)
- Failed downloads include error information for debugging
- Support retrying failed downloads
3. **Post-download integration** — after a successful download:
- Add the file to the database via `shanty-index` (or at minimum insert a record)
- If the download was for a watchlist item, update that item's status to `downloaded`
- Optionally run the organizer (`shanty-org`) to move the file to the right location
4. **CLI interface** — the `shanty-dl` binary should accept:
- `download <query_or_url>` — download a single item
- `download --from-queue` — process all pending items in the download queue
- `--format <mp3|opus|flac|best>` — output format
- `--output <dir>` — output directory
- `--dry-run` — show what would be downloaded without doing it
- `queue add <query>` — add an item to the download queue
- `queue list` — show the current queue
- `queue retry` — retry all failed downloads
### Design Considerations
- The download backend should be trait-based (`DownloadBackend` trait) so that future backends (torrents via `transmission-rpc`, Soulseek, NZBGet, etc.) can be added. The trait should define methods like `search`, `download`, `supports_url`, etc.
- yt-dlp output parsing can be tricky — use `--print-json` or `--dump-json` for structured output.
- Consider concurrent downloads (configurable limit, e.g., 3 simultaneous) for queue processing.
- yt-dlp must be installed separately — the crate should check for its presence and give a clear error if missing.
- Be mindful of YouTube rate limiting — don't hammer the API.
### Acceptance Criteria
- [ ] Given a search query, the downloader finds and downloads the correct song via yt-dlp
- [ ] Given a direct URL, the downloader downloads the audio from that URL
- [ ] Downloaded files are converted to the requested format
- [ ] Downloads are recorded in the database
- [ ] The download queue works — items can be added, processed, and retried
- [ ] Failed downloads are logged with error details
- [ ] `DownloadBackend` trait exists and yt-dlp is the first implementation
- [ ] CLI interface works as specified
- [ ] The crate checks for yt-dlp availability and errors clearly if missing
- [ ] Concurrent queue processing works with a configurable limit
- [ ] Integration test: download a short public-domain audio clip, verify file exists and is valid
### Dependencies
- Issue #1 (workspace scaffolding)
- Issue #2 (shared database schema)
- External: `yt-dlp` must be installed on the system
---
## Issue #8: Implement online music search in `shanty-search`
**Labels:** `mvp`, `searching`, `priority:medium`
### Description
The `shanty-search` crate provides the ability to search for music online — finding artists, albums, and tracks that can then be added to the user's library (watchlist). This is the "discovery" piece that feeds into `shanty-watch`. It wraps online music databases and returns structured results.
This issue covers:
1. **MusicBrainz search** — implement search functionality against the MusicBrainz API:
- Search for artists by name → returns list of matching artists with MusicBrainz IDs, disambiguation, and basic info
- Search for albums (releases) by name and/or artist → returns matching releases with track listings
- Search for tracks (recordings) by name and/or artist → returns matching recordings
- Get an artist's discography (all releases) given a MusicBrainz artist ID
2. **Result types** — define clean, serializable result types:
- `ArtistSearchResult` — name, MBID, disambiguation, country, type (person/group)
- `AlbumSearchResult` — title, artist, MBID, year, track count, cover art URL
- `TrackSearchResult` — title, artist, album, MBID, duration, track number
- `Discography` — list of releases grouped by type (album, single, EP, compilation)
3. **Search provider trait** — define a `SearchProvider` trait so that alternative backends (Last.fm, Spotify API for metadata, Discogs, etc.) can be added later. The trait should cover:
- `search_artist(query) -> Vec<ArtistSearchResult>`
- `search_album(query, artist_hint) -> Vec<AlbumSearchResult>`
- `search_track(query, artist_hint) -> Vec<TrackSearchResult>`
- `get_artist_discography(artist_id) -> Discography`
4. **Caching** — optionally cache search results in `shanty-db` to reduce API calls for repeated searches. Cache entries should have a TTL (e.g., 24 hours).
5. **CLI interface** — the `shanty-search` binary should accept:
- `artist <query>` — search for an artist
- `album <query> [--artist <artist>]` — search for an album
- `track <query> [--artist <artist>]` — search for a track
- `discography <artist_name_or_mbid>` — show an artist's discography
- `--json` — output results as JSON (useful for piping to other tools)
- `--limit <n>` — max number of results (default 10)
### Design Considerations
- This crate shares the MusicBrainz client with `shanty-tag`. Consider whether the MB client should live in a shared utility crate (e.g., `shanty-mb`) or in `shanty-db`, or whether each crate has its own thin client. A shared client avoids duplication of rate-limiting logic.
- The web interface will be the primary consumer of this crate — search results will be displayed in the UI and the user will click to add items to their watchlist. So the result types should be JSON-serializable and contain enough info for display.
- MusicBrainz rate limits apply here too — reuse the rate limiter.
### Acceptance Criteria
- [ ] Artist search returns relevant results from MusicBrainz
- [ ] Album search works, optionally filtered by artist
- [ ] Track search works, optionally filtered by artist
- [ ] Artist discography retrieval works and groups releases by type
- [ ] Result types are well-defined and JSON-serializable
- [ ] `SearchProvider` trait exists with MusicBrainz as the first implementation
- [ ] CLI interface works as specified
- [ ] `--json` output is valid JSON
- [ ] Rate limiting is respected
- [ ] Optional caching works and respects TTL
- [ ] Tests exist with mocked API responses
### Dependencies
- Issue #1 (workspace scaffolding)
- Issue #2 (shared database schema — for caching)
- Consider: shared MusicBrainz client with Issue #4 (tagging)
---
## Issue #9: Implement the Actix web backend for `shanty-web`
**Labels:** `mvp`, `web`, `backend`, `priority:high`
### Description
The `shanty-web` crate is the Actix-web backend that ties all the other crates together and exposes a REST API for the Elm frontend. This is the "brain" of Shanty when used as a web application — it orchestrates indexing, tagging, downloading, and everything else through HTTP endpoints.
This issue covers the backend API. The frontend (Elm) is a separate issue.
1. **Core API endpoints:**
**Library / Watchlist**
- `GET /api/artists` — list all artists in the library (with filter/pagination)
- `GET /api/artists/:id` — get artist details including albums and watchlist status
- `POST /api/artists` — add an artist to the watchlist
- `DELETE /api/artists/:id` — remove an artist from the watchlist
- `GET /api/albums` — list all albums (with filter/pagination)
- `GET /api/albums/:id` — get album details including tracks
- `POST /api/albums` — add an album to the watchlist
- `GET /api/tracks` — list all tracks (with filter/pagination)
- `GET /api/tracks/:id` — get track details
**Search**
- `GET /api/search/artist?q=<query>` — search for artists online
- `GET /api/search/album?q=<query>&artist=<artist>` — search for albums online
- `GET /api/search/track?q=<query>&artist=<artist>` — search for tracks online
- `GET /api/search/discography/:artist_id` — get an artist's full discography
**Downloads**
- `POST /api/downloads` — trigger a download (by query or URL)
- `GET /api/downloads/queue` — get the current download queue
- `POST /api/downloads/retry/:id` — retry a failed download
- `DELETE /api/downloads/:id` — cancel/remove a download
**System**
- `GET /api/status` — system status (DB stats, queue length, currently running tasks)
- `POST /api/index` — trigger a library re-scan
- `POST /api/tag` — trigger auto-tagging for untagged tracks
- `GET /api/config` — get current configuration
- `PUT /api/config` — update configuration
2. **Background task system** — long-running operations (indexing, tagging, bulk downloads) should run as background tasks. The API should:
- Return immediately with a task ID
- Provide a `GET /api/tasks/:id` endpoint to check task progress
- Support WebSocket or SSE for real-time progress updates to the frontend
3. **Configuration** — the web app needs a configuration system:
- Music library root directory
- Database path
- Download output directory
- Organization format template
- Auto-tagging settings
- Port/bind address
- Use a TOML or YAML config file, with environment variable overrides
4. **Static file serving** — serve the compiled Elm frontend from a static directory
5. **Error handling** — consistent JSON error responses with appropriate HTTP status codes
### Design Considerations
- Use `actix-web` as specified in the design doc.
- Each crate's library API should be called directly — don't shell out to the CLI binaries.
- Consider using `actix-rt` for spawning background tasks.
- API responses should use consistent pagination (offset/limit or cursor-based).
- Consider CORS configuration for development (Elm dev server on a different port).
### Acceptance Criteria
- [ ] Actix-web server starts and serves the API on a configurable port
- [ ] All listed API endpoints are implemented and return appropriate JSON responses
- [ ] Background tasks work for long-running operations (indexing, tagging)
- [ ] Task progress can be queried
- [ ] Configuration is loaded from a TOML file with env var overrides
- [ ] Static file serving works for the frontend
- [ ] Error responses are consistent JSON with proper status codes
- [ ] Pagination works on list endpoints
- [ ] CORS is configured for development
- [ ] Integration tests exist for the key API endpoints
- [ ] The server gracefully shuts down (completes in-progress tasks)
### Dependencies
- Issue #1 (workspace scaffolding)
- Issue #2 (shared database schema)
- Issue #3 (music indexing)
- Issue #4 (music tagging)
- Issue #6 (watchlist management)
- Issue #7 (music downloading)
- Issue #8 (music searching)
---
## Issue #10: Implement the Elm frontend for `shanty-web`
**Labels:** `mvp`, `web`, `frontend`, `priority:high`
### Description
The Elm frontend is the "face" of Shanty — the primary way users interact with the application. It communicates with the Actix backend via the REST API defined in Issue #9. The philosophy is that the user should be able to add artists/albums/songs to their library and have the app handle everything else automatically.
This issue covers the MVP frontend:
1. **Project setup:**
- Initialize an Elm project in a subdirectory (e.g., `shanty-web/frontend/` or a top-level `frontend/` directory)
- Set up a build pipeline (e.g., `elm make` with an npm script or a Makefile target) that compiles to a JS bundle served by the Actix backend
- Basic HTML shell with the Elm app mounted
2. **Core pages/views:**
**Dashboard**
- Overview of library stats (total artists, albums, tracks)
- Recent additions
- Current download queue status
- Any running background tasks
**Library Browser**
- Browse artists → albums → tracks hierarchy
- Show watchlist status per item (wanted/downloaded/owned)
- Filter and sort options
- Click into an artist to see their albums, click into an album to see its tracks
**Search & Add**
- Search bar that queries the backend's search API
- Display search results (artists, albums, tracks) in a clear layout
- "Add to Library" button on each result that adds it to the watchlist
- When adding an artist, show their discography and let the user select which albums to monitor
**Downloads**
- View the download queue (pending, in-progress, completed, failed)
- Trigger manual downloads by entering a search query or URL
- Retry failed downloads
- Cancel pending downloads
**Settings**
- View and edit configuration (library path, download directory, format template, etc.)
- Trigger manual actions (re-index, re-tag)
3. **Styling:**
- Use a clean, dark-theme design (fitting the "high seas" aesthetic)
- Responsive layout that works on desktop and tablet
- No CSS framework required — keep it simple with custom CSS or a minimal utility approach
4. **Real-time updates:**
- Use WebSocket or SSE to receive progress updates from background tasks
- Update the UI live when downloads complete or indexing finishes
### Design Considerations
- Elm enforces a clean architecture (Model-View-Update) — lean into this. Define clear message types for each action.
- Use `elm/http` for API calls and `elm/json` for decoding responses.
- Consider using `elm/url` for client-side routing (SPA with browser history).
- Keep the UI functional over pretty for MVP — polish can come later.
- All API types should have corresponding Elm decoders/encoders.
### Acceptance Criteria
- [ ] Elm project compiles and the frontend loads in a browser when served by the Actix backend
- [ ] Dashboard shows library stats, recent additions, and queue status
- [ ] Library browser allows navigating artists → albums → tracks
- [ ] Search works and displays results from MusicBrainz
- [ ] Users can add artists/albums/tracks to their library from search results
- [ ] Download queue is viewable and manageable (retry, cancel)
- [ ] Manual download by query/URL works
- [ ] Settings page allows viewing and editing configuration
- [ ] UI updates in real-time for background tasks
- [ ] Client-side routing works (SPA navigation)
- [ ] Layout is responsive and uses a dark theme
- [ ] No runtime errors in the Elm app (Elm's type system should help here)
### Dependencies
- Issue #9 (Actix web backend — the API must exist for the frontend to consume)
---
## Issue #11: Implement configuration management and app startup orchestration
**Labels:** `mvp`, `configuration`, `priority:medium`
### Description
Shanty needs a unified configuration system that all crates can use, plus a top-level orchestrator that starts the web server and any background services. Currently each crate accepts its own CLI args, but when running as a unified web app, there needs to be one config file and one startup command.
This issue covers:
1. **Configuration crate or module** — create a shared configuration type (either in `shanty-db` or a new `shanty-config` crate) that includes:
- `library_path` — root directory for organized music files
- `import_paths` — list of directories to scan for new music
- `database_path` — path to the SQLite database file
- `download_path` — temporary directory for downloads before organization
- `organization_format` — template string for file organization
- `web.port` — HTTP port (default 8085)
- `web.bind_address` — bind address (default 0.0.0.0)
- `tagging.auto_tag` — whether to auto-tag new files after indexing
- `tagging.write_tags` — whether to write tags back to files
- `tagging.confidence_threshold` — match confidence threshold
- `download.concurrent_limit` — max simultaneous downloads
- `download.default_format` — default audio format
- Load from a TOML file (default: `~/.config/shanty/config.toml` or `./shanty.toml`)
- Override any value with environment variables (e.g., `SHANTY_WEB_PORT=9090`)
2. **Top-level binary** — create a `shanty` binary (the main entry point) that:
- Loads configuration
- Initializes the database (runs migrations)
- Starts the web server
- Optionally runs an initial library scan on startup
- Handles graceful shutdown (SIGTERM/SIGINT)
3. **Logging** — set up structured logging (using `tracing` crate) across all crates:
- Configurable log level via config file and env var (`SHANTY_LOG` or `RUST_LOG`)
- Log to stdout and optionally to a file
- Include context (crate name, operation) in log entries
### Acceptance Criteria
- [ ] A single config file controls all crate behavior
- [ ] Config loads from TOML with environment variable overrides
- [ ] Default config is generated if no config file exists
- [ ] `shanty` binary starts the web server and initializes the database
- [ ] Graceful shutdown works on SIGTERM/SIGINT
- [ ] Structured logging is configured and works across all crates
- [ ] Log level is configurable
- [ ] Documentation/comments explain each config option
### Dependencies
- Issue #1 (workspace scaffolding)
- Issue #2 (shared database schema)
- Issue #9 (web backend)
---
## Issue #12: Add error handling strategy and shared error types
**Labels:** `mvp`, `quality`, `priority:medium`
### Description
As a multi-crate workspace, Shanty needs a consistent error handling strategy. Each crate will have its own error conditions, but they need to compose cleanly — especially when the web backend calls into library crates and needs to translate errors into HTTP responses.
This issue covers:
1. **Shared error types** — create an error module (in `shanty-db` or a new `shanty-core` crate) that defines:
- A base error enum that other crates can extend or wrap
- Common error variants: `DatabaseError`, `IoError`, `NetworkError`, `ConfigError`, `NotFoundError`, `ExternalToolError` (for yt-dlp, etc.)
- Implement `std::error::Error`, `Display`, and `From` conversions for common types
- Consider using `thiserror` for ergonomic error type derivation
2. **Per-crate errors** — each crate should define its own error type that wraps the shared errors plus crate-specific variants. For example, `shanty-tag` might have `TagError::NoMatchFound`, `TagError::ApiRateLimited`, etc.
3. **Web error mapping** — in `shanty-web`, implement `actix_web::ResponseError` for the error types so they automatically convert to appropriate HTTP responses (404 for NotFound, 500 for internal errors, 429 for rate limits, etc.)
4. **Result type alias** — define a `Result<T> = std::result::Result<T, ShantyError>` alias for convenience
### Acceptance Criteria
- [ ] Shared error types exist and are usable by all crates
- [ ] `thiserror` (or equivalent) is used for clean error type definitions
- [ ] Each crate has its own error type wrapping the shared base
- [ ] Web errors map to correct HTTP status codes
- [ ] Error messages are user-friendly (not raw debug output)
- [ ] `From` implementations allow easy error propagation with `?`
- [ ] No unwrap/expect calls in library code (only in tests and CLI `main`)
### Dependencies
- Issue #1 (workspace scaffolding)
---
## Issue #13: Add CI pipeline and basic test infrastructure
**Labels:** `mvp`, `ci`, `quality`, `priority:medium`
### Description
Set up a CI pipeline (for Gitea Actions or whatever CI system is available) and establish testing patterns that all crates should follow.
This issue covers:
1. **CI pipeline** — create a CI configuration that runs on every push/PR:
- `cargo fmt --check` — enforce consistent formatting
- `cargo clippy --workspace -- -D warnings` — catch common mistakes
- `cargo build --workspace` — verify everything compiles
- `cargo test --workspace` — run all tests
- Optional: `cargo audit` — check for known vulnerabilities in dependencies
2. **Test infrastructure:**
- Set up a test helper module/crate with utilities for creating temporary databases, test audio files, etc.
- Define integration test patterns: each crate's `tests/` directory should have at least one integration test
- Consider using `assert_cmd` for testing CLI binaries
- Provide a small set of test audio files (public domain / Creative Commons) or a script to generate them (e.g., using `sox` to create short silent audio files with metadata)
3. **Development tooling:**
- `Makefile` or `justfile` with common commands: `build`, `test`, `lint`, `fmt`, `run`
- `rust-toolchain.toml` to pin the Rust version
### Acceptance Criteria
- [ ] CI pipeline runs on push and verifies fmt, clippy, build, and test
- [ ] CI configuration file exists in the repo
- [ ] `rust-toolchain.toml` pins the Rust edition/version
- [ ] Test helper utilities exist for common test setup (temp DB, test files)
- [ ] A `Makefile` or `justfile` exists with common development commands
- [ ] At least a placeholder integration test exists in each MVP crate
### Dependencies
- Issue #1 (workspace scaffolding)
---
## Issue #14: Implement shared MusicBrainz client crate
**Labels:** `mvp`, `api`, `priority:medium`
### Description
Both `shanty-tag` (Issue #4) and `shanty-search` (Issue #8) need to communicate with the MusicBrainz API. Rather than duplicating the client, rate-limiting logic, and type definitions, extract a shared MusicBrainz client.
This could be a standalone crate (e.g., `shanty-mb`) or a module within `shanty-db`. It should provide:
1. **HTTP client** — configured with:
- Proper User-Agent header (`Shanty/<version> (<contact>)`) as required by MusicBrainz
- Rate limiting (max 1 request/second for unauthenticated use)
- Retry logic with exponential backoff for transient failures
- Timeout configuration
2. **API methods:**
- `search_artists(query, limit)` → artist results
- `search_releases(query, artist_hint, limit)` → release results
- `search_recordings(query, artist_hint, limit)` → recording results
- `get_artist(mbid)` → full artist details
- `get_release(mbid)` → full release details with track listing
- `get_recording(mbid)` → full recording details
- `get_artist_releases(mbid)` → artist's discography
- `get_cover_art(release_mbid)` → cover art URL from Cover Art Archive
3. **Response types** — well-typed Rust structs for all API responses, with `serde` deserialization
4. **Testing** — mock server tests (using `mockito` or `wiremock`) to test client behavior without hitting the real API
### Acceptance Criteria
- [ ] Shared MusicBrainz client crate exists and is usable by `shanty-tag` and `shanty-search`
- [ ] Rate limiting enforces max 1 req/sec
- [ ] All listed API methods are implemented
- [ ] Response types are well-defined and deserializable
- [ ] User-Agent header is set correctly
- [ ] Retry logic handles transient failures
- [ ] Tests exist using a mock HTTP server
- [ ] The client is configurable (timeout, rate limit, base URL for testing)
### Dependencies
- Issue #1 (workspace scaffolding)
---
## Issue #15: Implement Docker packaging and deployment
**Labels:** `mvp`, `deployment`, `priority:low`
### Description
Shanty should be easy to self-host. A Docker image is the most common way people deploy self-hosted media apps. This issue covers creating a Docker setup for Shanty.
1. **Dockerfile** — multi-stage build:
- Builder stage: compile the Rust workspace and Elm frontend
- Runtime stage: minimal image (e.g., `debian-slim` or `alpine`) with just the compiled binary, frontend assets, and runtime dependencies (`yt-dlp`, `ffmpeg`)
- Include `yt-dlp` and `ffmpeg` in the runtime image (needed for downloading/conversion)
2. **docker-compose.yml** — example compose file with:
- The Shanty service
- Volume mounts for music library, config, and database
- Environment variable configuration
- Example port mapping
3. **Documentation** — a section in the README or a dedicated `DOCKER.md` explaining:
- How to build and run with Docker
- Volume mount requirements
- Environment variable configuration
- Example docker-compose setup
### Acceptance Criteria
- [ ] `docker build` produces a working image
- [ ] The container starts and the web UI is accessible
- [ ] Music library directory can be mounted as a volume
- [ ] Database persists across container restarts (via volume)
- [ ] Config can be provided via volume mount or environment variables
- [ ] `yt-dlp` and `ffmpeg` are available inside the container
- [ ] `docker-compose.yml` works out of the box for basic usage
- [ ] Image size is reasonable (under 500MB)
### Dependencies
- Issue #9 (web backend)
- Issue #10 (Elm frontend)
- Issue #11 (configuration and startup)
---
## Issue #16: [Post-MVP] Implement acoustic fingerprinting fallback in `shanty-tag`
**Labels:** `post-mvp`, `tagging`, `enhancement`
### Description
When online metadata lookup fails (Issue #4), fall back to acoustic fingerprinting to identify tracks. This uses the Chromaprint/AcoustID system to generate a fingerprint from the audio data and look it up in the AcoustID database, which links back to MusicBrainz.
1. **Chromaprint integration** — use the `chromaprint` library (via FFI bindings or a Rust crate if available) to generate acoustic fingerprints from audio files
2. **AcoustID lookup** — submit the fingerprint + duration to the AcoustID API to get matching MusicBrainz recording IDs
3. **Integration with existing tag flow** — modify the tagging pipeline: try online text search first → if no match, generate fingerprint → look up via AcoustID → retrieve metadata from MusicBrainz using the matched recording ID
4. **Store fingerprints** — save generated fingerprints in the database so they don't need to be recomputed
### Acceptance Criteria
- [ ] Chromaprint fingerprints can be generated from audio files
- [ ] AcoustID API lookup returns matching MusicBrainz IDs
- [ ] Fingerprinting is used as a fallback when text-based search fails
- [ ] Generated fingerprints are cached in the database
- [ ] CLI flag `--fingerprint` forces fingerprint-based lookup
- [ ] Works for common audio formats (MP3, FLAC, OGG, OPUS)
### Dependencies
- Issue #4 (online metadata lookup — this extends it)
- External: `chromaprint` library must be available on the system (or bundled)
---
## Issue #17: [Post-MVP] Add torrent download backend to `shanty-dl`
**Labels:** `post-mvp`, `downloading`, `enhancement`
### Description
Add a torrent-based download backend to `shanty-dl` as an alternative to yt-dlp. This would allow users to download music from torrent sources, which often have higher quality (lossless FLAC) than YouTube.
1. **Transmission RPC client** — integrate with Transmission (a popular torrent client) via its RPC API:
- Add a torrent by URL or magnet link
- Monitor download progress
- Get the downloaded file path when complete
- Configure download directory
2. **Torrent search** — this is trickier and more legally sensitive. Options:
- Accept magnet links / .torrent URLs directly (user provides the source)
- Integrate with Jackett/Prowlarr (torrent indexer aggregators that many self-hosters already run) to search multiple indexers
- Start with direct magnet/URL input only, add Jackett integration later
3. **Backend registration** — implement the `DownloadBackend` trait from Issue #7 for the torrent backend. The download manager should be able to route requests to the appropriate backend.
4. **Post-download processing** — torrents may download entire albums. After download:
- Identify which files are audio files
- Index them into the database
- Organize them according to the user's format template
### Acceptance Criteria
- [ ] Torrent backend implements the `DownloadBackend` trait
- [ ] Can add torrents via magnet link or .torrent URL through Transmission RPC
- [ ] Download progress is tracked and queryable
- [ ] Downloaded audio files are identified and indexed
- [ ] Configuration for Transmission connection (host, port, auth) exists
- [ ] Falls back gracefully if Transmission is not available
### Dependencies
- Issue #7 (download system with trait-based backends)
- External: Transmission daemon must be running
---
## Issue #18: [Post-MVP] Add Soulseek download backend to `shanty-dl`
**Labels:** `post-mvp`, `downloading`, `enhancement`
### Description
Add a Soulseek-based download backend. Soulseek is a peer-to-peer network popular among music collectors, often having rare and high-quality files. Integration would likely use `nicotine+` (a Soulseek client) or a direct protocol implementation.
1. **Soulseek client integration** — options:
- Use `slskd` (a modern Soulseek client with a REST API) as the backend — similar to how we use Transmission for torrents
- Direct protocol implementation (complex, probably not worth it for MVP+1)
2. **Search and download** — search Soulseek for a track, present results (with quality/format info), and download the best match
3. **Implement `DownloadBackend` trait** for the Soulseek backend
### Acceptance Criteria
- [ ] Soulseek backend implements the `DownloadBackend` trait
- [ ] Can search for tracks on Soulseek via slskd API
- [ ] Can download files from Soulseek
- [ ] Download progress is tracked
- [ ] Quality/format info is available in search results for choosing the best source
- [ ] Configuration for slskd connection exists
### Dependencies
- Issue #7 (download system with trait-based backends)
- External: `slskd` instance must be running
---
## Issue #19: [Post-MVP] Implement new release monitoring in `shanty-watch`
**Labels:** `post-mvp`, `watching`, `enhancement`
### Description
Extend `shanty-watch` to periodically check for new releases from monitored artists. When a new release is detected, it should be added to the watchlist as "available" and optionally trigger a notification or automatic download.
1. **Release checking** — periodically query MusicBrainz (or other configured provider) for each monitored artist's releases. Compare against what's already known in the database. New releases get added to the watchlist with status `available`.
2. **Scheduling** — the check should run on a configurable schedule (e.g., daily, every 12 hours). Use a simple scheduler or cron-like system within the app.
3. **Auto-download option** — if enabled, automatically add new releases to the download queue when detected.
4. **Notification hook** — when a new release is detected, emit an event that `shanty-notify` can consume (or directly trigger a notification if the notify crate is available).
5. **Handling rate limits** — if the user monitors hundreds of artists, checking all of them requires many API calls. Spread checks over time to stay within rate limits. Track when each artist was last checked to avoid redundant queries.
### Acceptance Criteria
- [ ] New releases are detected for monitored artists
- [ ] Detected releases are added to the watchlist as `available`
- [ ] Checking runs on a configurable schedule
- [ ] Rate limits are respected even with many monitored artists
- [ ] Auto-download option works when enabled
- [ ] A notification event is emitted for new releases
- [ ] Last-checked timestamp is stored per artist
- [ ] Can trigger a manual check for a specific artist or all artists
### Dependencies
- Issue #6 (watchlist management)
- Issue #14 (shared MusicBrainz client)
---
## Issue #20: [Post-MVP] Implement notification system in `shanty-notify`
**Labels:** `post-mvp`, `notifications`, `enhancement`
### Description
The `shanty-notify` crate sends notifications to the user when events occur (new releases detected, downloads completed, errors requiring attention, etc.).
1. **Notification backends** — implement a trait-based notification system supporting:
- **Apprise** — a notification aggregator that supports 80+ services (Discord, Slack, Telegram, email, Pushover, Gotify, etc.). By integrating with Apprise, Shanty gets all those services for free.
- **Webhook** — simple HTTP POST to a user-configured URL with a JSON payload
- **Log only** — just log the notification (useful as a default/fallback)
2. **Event types:**
- `NewReleaseDetected` — a monitored artist has a new release
- `DownloadCompleted` — a download finished successfully
- `DownloadFailed` — a download failed (after retries)
- `IndexingCompleted` — a library scan finished
- `TaggingCompleted` — auto-tagging finished
3. **Configuration** — per-event-type notification settings (e.g., notify on new releases via Discord but not on download completion)
### Acceptance Criteria
- [ ] Notification trait exists with multiple backend implementations
- [ ] Apprise integration works (via shelling out to `apprise` CLI or its API)
- [ ] Webhook backend sends proper HTTP POST with JSON payload
- [ ] Notifications are triggered by the appropriate events
- [ ] Per-event-type configuration works
- [ ] CLI: `shanty-notify test` sends a test notification
### Dependencies
- Issue #1 (workspace scaffolding)
- Issue #19 (new release monitoring — primary trigger for notifications)
---
## Issue #21: [Post-MVP] Implement playlist generation in `shanty-playlist`
**Labels:** `post-mvp`, `playlisting`, `enhancement`
### Description
The `shanty-playlist` crate generates playlists based on the user's indexed music library. Some of this work has been done in a prior project called "drift" and the approach should be adapted for Shanty's database and architecture.
1. **Playlist generation strategies:**
- **Similar artists** — given a seed artist, find similar artists (via Last.fm or MusicBrainz) that exist in the library and build a playlist from their top tracks
- **Genre-based** — build a playlist from tracks matching a given genre or set of genres
- **Random/shuffle** — weighted random selection with configurable rules (e.g., no two tracks by the same artist in a row, prefer recently added, etc.)
- **Smart playlists** — rule-based playlists (e.g., "all tracks added in the last 30 days with genre 'rock'")
2. **Output formats:**
- M3U playlist file
- JSON (for API consumption)
- Direct integration with the music server (if `shanty-serve` is available)
3. **CLI interface:**
- `shanty-playlist similar <artist>` — generate a similar-artists playlist
- `shanty-playlist genre <genre>` — generate a genre playlist
- `shanty-playlist random [--count <n>]` — random playlist
- `shanty-playlist smart <rules>` — rule-based playlist
- `--format <m3u|json>` — output format
- `--count <n>` — number of tracks (default 50)
4. **Web API integration** — expose playlist generation through the web API so the frontend can generate and display playlists
### Acceptance Criteria
- [ ] Similar-artist playlist generation works
- [ ] Genre-based playlist generation works
- [ ] Random playlist with rules works
- [ ] M3U output format is valid and plays in standard media players
- [ ] JSON output is well-structured
- [ ] CLI interface works as specified
- [ ] Web API endpoint for generating playlists exists
- [ ] Playlists only include tracks that exist in the local library
### Dependencies
- Issue #2 (database — needs indexed tracks to build playlists from)
- Issue #3 (music indexing)
- Issue #14 (MusicBrainz client — for similar artist lookups)
---
## Issue #22: [Post-MVP] Implement music serving/streaming in `shanty-serve`
**Labels:** `post-mvp`, `serving`, `enhancement`
### Description
The `shanty-serve` crate serves music files for streaming playback. The initial approach is to implement a Subsonic-compatible API, which would allow users to use existing Subsonic-compatible clients (DSub, Ultrasonic, Sonixd, Sublime Music, etc.) to stream their library.
1. **Subsonic API compatibility** — implement the core Subsonic REST API endpoints:
- `ping` — server status
- `getLicense` — return a valid license (always "valid" for self-hosted)
- `getMusicFolders` — return configured library paths
- `getArtists` / `getArtist` — artist listing and details
- `getAlbum` — album details with tracks
- `getSong` — track details
- `stream` — stream the actual audio file (with optional transcoding)
- `download` — download the original file
- `getCoverArt` — serve album cover art
- `search3` — search the library
- `getPlaylists` / `getPlaylist` — playlist support
- `scrobble` — track play history
2. **Transcoding** — optionally transcode to a lower bitrate on-the-fly (using ffmpeg) for bandwidth-limited clients. Support common output formats (MP3, OPUS, AAC).
3. **Authentication** — the Subsonic API uses a simple username/password scheme. Implement basic auth that works with existing clients.
### Design Considerations
- The Subsonic API is XML-based by default but most clients support JSON. Support both.
- This could be a separate Actix server on a different port, or additional routes on the main web server.
- Consider starting with the minimum viable set of Subsonic endpoints that popular clients need to function.
### Acceptance Criteria
- [ ] Subsonic-compatible API serves music from the library
- [ ] At least one popular Subsonic client (e.g., Sonixd, DSub) can connect and browse/play
- [ ] Audio streaming works for all supported formats
- [ ] Cover art serving works
- [ ] Basic search works
- [ ] Authentication works
- [ ] Optional transcoding works via ffmpeg
### Dependencies
- Issue #2 (database)
- Issue #3 (music indexing — library must be populated)
- External: `ffmpeg` for transcoding
---
## Issue #23: [Post-MVP] Implement built-in web player in the Elm frontend
**Labels:** `post-mvp`, `frontend`, `playback`, `enhancement`
### Description
Add a built-in music player to the Elm web UI so users can listen to their library directly in the browser without needing an external Subsonic client.
1. **Audio player component:**
- HTML5 `<audio>` element integration in Elm
- Play/pause, next/previous, seek bar, volume control
- Track info display (title, artist, album, cover art)
- Persistent player bar at the bottom of the UI (doesn't disappear when navigating)
2. **Queue management:**
- "Play now" adds a track and starts playing
- "Add to queue" appends to the play queue
- "Play album" / "Play artist" loads all tracks into the queue
- Queue is viewable, reorderable, and clearable
- Shuffle and repeat modes
3. **Integration with library browser:**
- Play buttons on tracks, albums, and artists in the library view
- "Now playing" indicator in the library browser
- Clicking a track in album view plays that track and queues the rest of the album
### Acceptance Criteria
- [ ] Audio playback works in the browser for all supported formats
- [ ] Player controls (play, pause, seek, volume, next, previous) work
- [ ] Track info and cover art display while playing
- [ ] Play queue is functional (add, remove, reorder, clear)
- [ ] Shuffle and repeat modes work
- [ ] Player persists across page navigation
- [ ] Playing a track from the library browser works seamlessly
### Dependencies
- Issue #10 (Elm frontend)
- Issue #22 (music serving — or alternatively, serve files directly from the Actix backend)
---
## Issue #24: [Post-MVP] Configurable organization templates and advanced renaming in `shanty-org`
**Labels:** `post-mvp`, `organization`, `enhancement`
### Description
Extend the organization system (Issue #5) with more powerful and configurable templates.
1. **Advanced template variables:**
- `{original_filename}` — the original filename without extension
- `{quality}` — e.g., "FLAC", "MP3 320", "MP3 V0"
- `{album_type}` — album, single, EP, compilation
- `{first_letter}` — first letter of artist (for A-Z directory grouping)
- Conditional blocks: `{if disc_count > 1}Disc {disc_number}/{endif}`
2. **Multiple organization profiles** — allow different templates for different use cases (e.g., one for lossless, one for lossy, one for singles vs. albums)
3. **Conflict resolution strategies** — configurable behavior when target path already exists:
- Skip
- Overwrite
- Rename with incrementing number
- Keep higher quality version
4. **Reverse organization** — given the current organized structure and the format template, allow "flattening" or reorganizing to a different template
### Acceptance Criteria
- [ ] All new template variables work
- [ ] Conditional blocks in templates work
- [ ] Multiple organization profiles can be configured
- [ ] Each conflict resolution strategy works correctly
- [ ] Re-organization from one template to another works
- [ ] Existing tests still pass
### Dependencies
- Issue #5 (basic organization)
---
## Issue #25: [Post-MVP] Add Last.fm scrobbling and integration
**Labels:** `post-mvp`, `integration`, `enhancement`
### Description
Integrate with Last.fm for two purposes: (1) scrobbling played tracks (if using the built-in player or Subsonic server), and (2) using Last.fm's data for better recommendations and similar-artist lookups.
1. **Last.fm authentication** — implement the Last.fm auth flow (API key + user authentication)
2. **Scrobbling** — when a track is played (via web player or Subsonic), scrobble it to Last.fm
3. **Similar artists API** — use Last.fm's `artist.getSimilar` for playlist generation and discovery
4. **Top tracks** — use Last.fm's `artist.getTopTracks` for suggesting which tracks to download for a monitored artist
5. **`SearchProvider` implementation** — implement the search trait from Issue #8 using Last.fm as a backend
### Acceptance Criteria
- [ ] Last.fm authentication works
- [ ] Track scrobbling works during playback
- [ ] Similar artist lookups return relevant results
- [ ] Top tracks for an artist can be retrieved
- [ ] Last.fm works as an alternative search provider
- [ ] API key configuration is documented
### Dependencies
- Issue #8 (search provider trait)
- Issue #22 or #23 (playback — for scrobbling)
---
## Issue #26: [Post-MVP] Add import from existing Lidarr database
**Labels:** `post-mvp`, `migration`, `enhancement`
### Description
Many potential Shanty users will be migrating from Lidarr. Provide a migration tool that imports their existing Lidarr library (artists, albums, wanted items, quality profiles) into Shanty.
1. **Lidarr database reading** — Lidarr uses a SQLite database. Read the relevant tables:
- Artists (monitored status, metadata, quality profile)
- Albums (monitored status, whether it's been downloaded)
- Root folders
2. **Mapping** — convert Lidarr's data model to Shanty's:
- Lidarr artists → Shanty artists (with MusicBrainz IDs)
- Lidarr monitored albums → Shanty watchlist items
- Lidarr quality profiles → approximate Shanty download format preferences
3. **CLI tool:**
- `shanty import lidarr --db /path/to/lidarr.db`
- `--dry-run` to preview what would be imported
- Report: X artists, Y albums, Z items to be monitored
### Acceptance Criteria
- [ ] Can read a Lidarr SQLite database
- [ ] Artists and their monitored status are imported correctly
- [ ] Albums and their wanted/downloaded status are imported
- [ ] MusicBrainz IDs are preserved
- [ ] `--dry-run` works
- [ ] Import summary is displayed
- [ ] Duplicate detection — don't import artists/albums that already exist in Shanty
### Dependencies
- Issue #2 (database schema)
- Issue #6 (watchlist management)
---
## Issue #27: [Post-MVP] Add cover art management
**Labels:** `post-mvp`, `metadata`, `enhancement`
### Description
Manage album cover art — download, store, and serve cover art images for albums in the library.
1. **Cover art downloading:**
- Download from Cover Art Archive (MusicBrainz) using release MBID
- Fall back to embedded cover art in audio files
- Store in a configurable directory (e.g., `covers/{mbid}.jpg` or alongside the album)
2. **Embedded art extraction** — extract cover art embedded in audio file tags and save as a standalone file
3. **Art serving** — serve cover art via the web API for display in the frontend and Subsonic clients
4. **Art embedding** — optionally embed cover art back into audio files that lack it
### Acceptance Criteria
- [ ] Cover art is downloaded from Cover Art Archive when available
- [ ] Embedded cover art is extracted as a fallback
- [ ] Cover art is served via the API
- [ ] Cover art displays in the web frontend
- [ ] Cover art is available via the Subsonic API
- [ ] Optionally embed art into audio files
### Dependencies
- Issue #4 (tagging — provides MusicBrainz IDs for Cover Art Archive)
- Issue #9 (web backend — for serving)
---
## Issue #28: [Post-MVP] Add multi-user support
**Labels:** `post-mvp`, `enhancement`, `web`
### Description
Add basic multi-user support so multiple people in a household can share a Shanty instance with their own watchlists and preferences.
1. **User accounts** — basic username/password authentication
2. **Per-user watchlists** — each user has their own monitored artists/albums
3. **Shared library** — the actual music library is shared, but watchlists are personal
4. **Admin user** — one user is admin and can manage settings, trigger system operations, manage other users
5. **Session management** — JWT or cookie-based sessions for the web UI
### Acceptance Criteria
- [ ] Users can register and log in
- [ ] Each user has their own watchlist
- [ ] The music library is shared across users
- [ ] Admin can access settings and manage users
- [ ] Non-admin users can only manage their own watchlist
- [ ] Session management works in the web UI
- [ ] Subsonic auth maps to Shanty users
### Dependencies
- Issue #9 (web backend)
- Issue #10 (Elm frontend)
- Issue #6 (watchlist — needs to become per-user)
---
## Issue #29: Add comprehensive documentation
**Labels:** `mvp`, `documentation`, `priority:low`
### Description
Write user-facing and developer-facing documentation for Shanty.
1. **User documentation:**
- Installation guide (from source, Docker, binary releases)
- Configuration reference (all config options explained)
- Quick start guide (get from zero to playing music)
- FAQ
2. **Developer documentation:**
- Architecture overview (crate structure, data flow)
- Contributing guide (how to set up the dev environment, coding standards, PR process)
- API documentation (REST API reference for the web backend)
3. **Inline documentation:**
- Rustdoc comments on all public types and functions
- `cargo doc --workspace` should produce useful documentation
### Acceptance Criteria
- [ ] README covers installation, quick start, and basic usage
- [ ] Configuration reference documents all options
- [ ] Architecture doc explains the crate structure and how they interact
- [ ] Contributing guide exists
- [ ] REST API is documented (consider using OpenAPI/Swagger)
- [ ] `cargo doc` produces clean documentation with no warnings
### Dependencies
- Most other MVP issues (documentation should reflect implemented features)
---
## Issue #30: [Post-MVP] Add NZBGet/SABnzbd download backend to `shanty-dl`
**Labels:** `post-mvp`, `downloading`, `enhancement`
### Description
Add a Usenet-based download backend using NZBGet or SABnzbd as the download client. This is similar in approach to the torrent backend (Issue #17) — integrate with the existing download client's API.
1. **NZBGet/SABnzbd API client** — communicate with the download client to:
- Submit NZB files for download
- Monitor download progress
- Retrieve completed downloads
2. **NZB source integration** — integrate with NZB indexers (via Prowlarr/NZBHydra2, or direct indexer API) to search for music
3. **Implement `DownloadBackend` trait**
### Acceptance Criteria
- [ ] Can submit NZBs to NZBGet or SABnzbd
- [ ] Download progress is tracked
- [ ] Completed downloads are detected and processed
- [ ] Implements the `DownloadBackend` trait
- [ ] Configuration for the Usenet client connection exists
### Dependencies
- Issue #7 (download system with trait-based backends)
---
## Issue #31: [Post-MVP] Implement quality profile system
**Labels:** `post-mvp`, `enhancement`, `downloading`
### Description
Add a quality profile system that lets users define their preferred audio quality and format, similar to Lidarr's quality profiles. This guides the download system in choosing the best available source.
1. **Quality profiles** — define named profiles like:
- "Lossless" — prefer FLAC > ALAC > WAV, minimum bitrate 800kbps
- "High Quality" — prefer FLAC > MP3 320 > OPUS 256
- "Standard" — MP3 320 > MP3 V0 > OPUS 128
- "Any" — accept anything
2. **Per-item quality assignment** — watchlist items can be assigned a quality profile (default to a global setting)
3. **Download source scoring** — when multiple download sources are available, score them against the quality profile and pick the best match
4. **Upgrade detection** — if a user has an MP3 version but their profile prefers FLAC, flag it as "upgradeable"
### Acceptance Criteria
- [ ] Quality profiles can be defined with format and bitrate preferences
- [ ] Profiles can be assigned to watchlist items
- [ ] Download source selection respects the quality profile
- [ ] Upgrade detection works (lower quality → can be upgraded)
- [ ] Default profile is configurable
- [ ] At least 3 built-in profiles exist
### Dependencies
- Issue #6 (watchlist)
- Issue #7 (downloading)
---
## Issue #32: [Post-MVP] Add TUI interface
**Labels:** `post-mvp`, `enhancement`, `tui`
### Description
Several crates were designed to have TUI (terminal user interface) capability. Implement a unified TUI using `ratatui` that provides an interactive terminal experience for managing Shanty without a web browser.
1. **Library browser** — navigate artists/albums/tracks in the terminal
2. **Search** — search for music and add to watchlist
3. **Download queue** — view and manage downloads
4. **System status** — overview of running tasks, library stats
5. **Key bindings** — vim-style navigation
This is particularly useful for headless servers where users SSH in, or for users who prefer terminal workflows.
### Acceptance Criteria
- [ ] TUI launches with `shanty tui` or as the default for individual crate binaries
- [ ] Library browsing works with keyboard navigation
- [ ] Search and add-to-library flow works
- [ ] Download queue is viewable and manageable
- [ ] System status is displayed
- [ ] Responsive to terminal size changes
### Dependencies
- Issue #2 (database)
- Issue #9 (may share the backend API, or call library crates directly)
Reference in New Issue View Git Blame Copy Permalink