Compare commits
55 Commits
51f2c2ae8f
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| e1beb5f12b | |||
| 219404263c | |||
| ada336d945 | |||
| 494c8ddecb | |||
| cab680ad5d | |||
| 31c9785ed2 | |||
| e198643c57 | |||
| beb80b8770 | |||
| 3aecde5d0b | |||
| 01365dbb80 | |||
| b2f030b52d | |||
| f77cea47b1 | |||
| cb4105564c | |||
| 68c0f477dd | |||
| 3153518c57 | |||
| b3fd844c15 | |||
| f14ddea805 | |||
| 0d22da6aaa | |||
| a63d72ba48 | |||
| 8dcf40fe7c | |||
| 5e44275bff | |||
| f5fd450aaf | |||
| e3fc3789ce | |||
| 42ab414f83 | |||
| 79440617ba | |||
| 295380d5ad | |||
| 1f68208547 | |||
| 9df13a1c3a | |||
| 144393756e | |||
| 7058247884 | |||
| 12ff2ca2ed | |||
| 2e8f3f6b11 | |||
| 911d27713b | |||
| a39b76452f | |||
| 4e6ccd8359 | |||
| 3ae6c3e0e3 | |||
| 2c49f82384 | |||
| 31616a76e1 | |||
| 49d1d2252e | |||
| 929772512e | |||
| 941c1ffe50 | |||
| 2611f283f6 | |||
| 4745060e69 | |||
| 026d1f446b | |||
| 314400bde5 | |||
| 57a8b7eee4 | |||
| 1861f086fa | |||
| 3708829c3b | |||
| 6821427471 | |||
| 53b3a644a1 | |||
| c7110fbbe7 | |||
| 24f5c4e02f | |||
| 5213049e19 | |||
| 763774ac1e | |||
| 88b280c2b2 |
@@ -0,0 +1,214 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## What Is Shanty?
|
||||
|
||||
Shanty is a self-hosted music management application ("better Lidarr"). It searches MusicBrainz for music metadata, downloads from YouTube via yt-dlp, tags and organizes files, and serves the library over the Subsonic protocol. It is a Cargo workspace where each component is both a standalone CLI tool and a library consumed by the web app.
|
||||
|
||||
## Development Commands
|
||||
|
||||
A `justfile` provides common workflows. Run `just` to see all targets.
|
||||
|
||||
```sh
|
||||
just check # fmt + lint + test (full pre-commit check)
|
||||
just dev # build frontend + run server
|
||||
just build # cargo build --workspace
|
||||
just test # cargo test --workspace
|
||||
just lint # cargo clippy --workspace -- -D warnings
|
||||
just fmt # cargo fmt
|
||||
just frontend # cd shanty-web/frontend && trunk build
|
||||
just run # cargo run --bin shanty
|
||||
```
|
||||
|
||||
**Running single crate tests:**
|
||||
```sh
|
||||
cargo test --package shanty-db
|
||||
cargo test --package shanty-tag
|
||||
```
|
||||
|
||||
**Running a single test by name:**
|
||||
```sh
|
||||
cargo test --package shanty-tag test_tag_with_match
|
||||
```
|
||||
|
||||
**Frontend build (Yew/Trunk → WASM):**
|
||||
```sh
|
||||
cd shanty-web/frontend && trunk build # dev
|
||||
cd shanty-web/frontend && trunk build --release # optimized
|
||||
```
|
||||
|
||||
**Running the server with verbose logging:**
|
||||
```sh
|
||||
cargo run --bin shanty -- -v # info
|
||||
cargo run --bin shanty -- -vv # debug
|
||||
cargo run --bin shanty -- -vvv # trace
|
||||
```
|
||||
|
||||
**MusicBrainz dump import subcommand:**
|
||||
```sh
|
||||
cargo run --bin shanty -- mb-import --download --data-dir /path/to/dumps
|
||||
```
|
||||
|
||||
**Prerequisites:** Rust (stable, edition 2024), yt-dlp, ffmpeg, Python 3, ytmusicapi, Trunk. The `rust-toolchain.toml` pins stable and adds the `wasm32-unknown-unknown` target.
|
||||
|
||||
## Design Philosophy
|
||||
|
||||
1. **Modular crates.** Each crate is a library and a CLI binary. The web app imports the library side; the CLI binary is for standalone use. Crates are git submodules hosted at `ssh://connor@git.rcjohnstone.com:2222/Shanty/{name}.git`. The exceptions are `shanty-config` and `shanty-data`, which are local workspace crates (not submodules).
|
||||
|
||||
2. **MBID-first matching.** All matching and deduplication in the web app uses MusicBrainz recording MBIDs, never string-based name matching. Name matching is only used in standalone CLI tools as a fallback.
|
||||
|
||||
3. **Provider-swappable data layer.** All external API calls go through trait-based providers in `shanty-data`. Metadata, artist images, bios, lyrics, and cover art each have a trait with multiple implementations. The active provider is selected via config.
|
||||
|
||||
4. **Track-level watchlist.** When a user watches an artist or album, it is expanded into individual track `wanted_item` records via MusicBrainz, each with a recording MBID. This enables per-track status tracking through the pipeline.
|
||||
|
||||
5. **Release groups, not releases.** The UI shows deduplicated release groups (album concepts), not individual releases (which have tons of reissues/regional editions). Filtered by secondary type -- default is studio only.
|
||||
|
||||
## Workspace Structure
|
||||
|
||||
All crates live in the workspace root.
|
||||
|
||||
| Crate | Purpose |
|
||||
|-------|---------|
|
||||
| `shanty` (root) | Top-level binary entry point, Actix server setup, graceful shutdown, background task spawning |
|
||||
| `shanty-config` | Shared config types (AppConfig), YAML loading/saving, environment variable overrides |
|
||||
| `shanty-data` | Unified external data providers: MusicBrainz (remote + local hybrid), Wikipedia, fanart.tv, Last.fm, LRCLIB, Cover Art Archive, MB dump import |
|
||||
| `shanty-db` | Sea-ORM + SQLite schema, migrations, query modules for all tables |
|
||||
| `shanty-index` | Scan directories, extract metadata from audio files via lofty |
|
||||
| `shanty-tag` | MusicBrainz lookup, fuzzy matching, file tag writing |
|
||||
| `shanty-org` | File organization with configurable format templates |
|
||||
| `shanty-watch` | Watchlist management, MusicBrainz discography expansion (artist/album to tracks) |
|
||||
| `shanty-dl` | yt-dlp download backend, rate limiting, download queue processing, ytmusicapi search |
|
||||
| `shanty-search` | SearchProvider trait, MusicBrainz search + release group listing |
|
||||
| `shanty-playlist` | Playlist generation strategies (similar-artist, genre, random, smart rules) |
|
||||
| `shanty-web` | Actix backend routes, Yew (WASM) frontend, background task modules |
|
||||
| `shanty-notify` | Notifications via Apprise/webhooks (stub -- not yet implemented) |
|
||||
| `shanty-serve` | Subsonic streaming (stub -- functionality is in shanty-web) |
|
||||
| `shanty-play` | Built-in web player (stub -- not yet implemented) |
|
||||
|
||||
The frontend is at `shanty-web/frontend/` and is excluded from the Cargo workspace. It builds separately with Trunk to `shanty-web/static/`.
|
||||
|
||||
## Key Architectural Patterns
|
||||
|
||||
### Data Providers (shanty-data)
|
||||
|
||||
`shanty-data` owns all external API calls. Key traits:
|
||||
|
||||
- `MetadataFetcher` -- artist info, release tracks, release resolution (MusicBrainz)
|
||||
- `ArtistImageFetcher` -- artist photos (Wikipedia, fanart.tv)
|
||||
- `ArtistBioFetcher` -- artist biographies (Wikipedia, Last.fm)
|
||||
- `LyricsFetcher` -- song lyrics (LRCLIB)
|
||||
- `CoverArtFetcher` -- album art (Cover Art Archive)
|
||||
- `SimilarArtistFetcher` -- similar artist data (Last.fm)
|
||||
|
||||
**HybridMusicBrainzFetcher** wraps `LocalMusicBrainzFetcher` (optional) + `MusicBrainzFetcher` (remote API). It tries the local SQLite database first and falls back to the rate-limited remote API. The local DB is populated by importing MusicBrainz JSON dumps.
|
||||
|
||||
### Web Server (shanty-web + root binary)
|
||||
|
||||
The root `shanty` binary sets up the Actix server, creates shared `AppState`, and spawns background tasks. The `shanty-web` crate provides the route handlers and frontend.
|
||||
|
||||
**AppState** holds: database connection, MusicBrainz client (hybrid), search provider, Wikipedia fetcher, shared config (behind `Arc<RwLock>`), task manager, scheduler info, Firefox login session state.
|
||||
|
||||
### Background Tasks
|
||||
|
||||
Four background loops run via `tokio::spawn` + sleep:
|
||||
|
||||
1. **cookie_refresh** -- refreshes YouTube cookies via headless Firefox (every 6 hours, configurable)
|
||||
2. **pipeline_scheduler** -- runs the full download pipeline automatically (every 3 hours, configurable)
|
||||
3. **monitor** -- checks monitored artists for new releases (every 12 hours, configurable)
|
||||
4. **mb_update** -- re-imports MusicBrainz dumps if auto_update is enabled (weekly)
|
||||
|
||||
One-off tasks (index, tag, organize, download process, monitor check, MB import) are spawned on demand and tracked via `TaskManager`.
|
||||
|
||||
### Database
|
||||
|
||||
Sea-ORM with SQLite. Migrations run automatically on startup. Key tables:
|
||||
|
||||
- `artists` -- name (unique), musicbrainz_id (unique), monitored flag, top_songs/similar_artists (JSON)
|
||||
- `albums` -- name, album_artist, year, genre, musicbrainz_id, artist_id FK
|
||||
- `tracks` -- file_path (unique), all metadata fields, musicbrainz_id, artist_id/album_id FKs
|
||||
- `wanted_items` -- item_type, name, musicbrainz_id, artist_id, status (Wanted/Available/Downloaded/Owned), user_id
|
||||
- `download_queue` -- query, wanted_item_id FK, status, retry_count
|
||||
- `search_cache` -- query_key (unique), provider, result_json, expires_at (used for MB data, lyrics, artist enrichment)
|
||||
- `users` -- username, password_hash (bcrypt), role (Admin/User), subsonic_password (plaintext per Subsonic protocol)
|
||||
- `playlists` / `playlist_tracks` -- saved playlists with ordered track references
|
||||
|
||||
### Subsonic API
|
||||
|
||||
Mounted at `/rest/*` with separate authentication (username + MD5 token, per the Subsonic protocol spec). Supports browsing, streaming, playlists, search, cover art, and scrobbling. Opus files are auto-transcoded to MP3 via ffmpeg for client compatibility.
|
||||
|
||||
### Frontend
|
||||
|
||||
Yew 0.21 with client-side rendering (CSR, no SSR). Built with Trunk to WASM. The compiled output goes to `shanty-web/static/` and is served by Actix with SPA fallback (all non-API routes serve `index.html`).
|
||||
|
||||
## Data Flow (Pipeline)
|
||||
|
||||
The full automated pipeline, triggered by "Set Sail" in the UI or by the pipeline scheduler:
|
||||
|
||||
1. **Search** -- find artist/album on MusicBrainz
|
||||
2. **Watch** -- add to watchlist (expands to individual track wanted_items with recording MBIDs)
|
||||
3. **Sync** -- `shanty_dl::sync_wanted_to_queue` creates download_queue entries for wanted items
|
||||
4. **Download** -- yt-dlp downloads via YouTube Music search (ytmusicapi Python script), creates track records in DB with MBIDs from wanted_items
|
||||
5. **Index** -- scan library, extract metadata from new files
|
||||
6. **Tag** -- MusicBrainz lookup by MBID (skips search since MBID is known), write tags to files
|
||||
7. **Organize** -- move files to `{artist}/{album}/{track_number} - {title}.{ext}` in the library
|
||||
8. **Promote** -- all Downloaded wanted_items are marked as Owned
|
||||
|
||||
## Configuration
|
||||
|
||||
YAML config file at `~/.config/shanty/config.yaml` (or `SHANTY_CONFIG` env var). Environment variables override YAML values. In Docker, the config file is at `/config/config.yaml`.
|
||||
|
||||
Key environment variables:
|
||||
- `SHANTY_DATA_DIR` -- base directory for all application data (Docker: `/data`)
|
||||
- `SHANTY_DATABASE_URL` -- SQLite connection string
|
||||
- `SHANTY_LIBRARY_PATH` -- music library root
|
||||
- `SHANTY_CONFIG` -- path to config YAML
|
||||
- `SHANTY_WEB_PORT` / `SHANTY_WEB_BIND` -- server binding
|
||||
- `SHANTY_LASTFM_API_KEY` -- Last.fm API key (for bios and similar-artist playlists)
|
||||
- `SHANTY_FANART_API_KEY` -- fanart.tv API key (for artist images/banners)
|
||||
|
||||
The config is loaded once at startup and held in `Arc<RwLock<AppConfig>>`. It can be updated at runtime via the `/api/config` PUT endpoint, which writes back to the YAML file and updates the in-memory config.
|
||||
|
||||
## Coding Standards
|
||||
|
||||
- **Rust edition 2024** with resolver v3
|
||||
- `cargo clippy -- -D warnings` must pass
|
||||
- `cargo fmt` for formatting
|
||||
- All crates must compile independently
|
||||
- Never use `#[allow(dead_code)]` -- remove dead code instead
|
||||
- Never create local DB records for artists the user is just browsing (only persist when they watch)
|
||||
- Use MBIDs for all matching in the web app, never name-based matching
|
||||
- Artist credits: use the primary (first) artist only, never concatenate collaborators
|
||||
|
||||
## Testing
|
||||
|
||||
```sh
|
||||
cargo test --workspace # all tests
|
||||
cargo test --package shanty-db # single crate
|
||||
```
|
||||
|
||||
The frontend is excluded from workspace tests (it has its own build process via Trunk).
|
||||
|
||||
**Test patterns used across crates:**
|
||||
- **In-memory SQLite:** Integration tests create `Database::new("sqlite::memory:")` for fast, isolated DB testing. No fixtures directory -- data is inserted programmatically.
|
||||
- **Mock providers:** Each crate that depends on external APIs defines its own mock trait implementations (e.g., `MockProvider` for `MetadataProvider`, `MockBackend` for `DownloadBackend`). Mocks are self-contained per crate.
|
||||
- **Temp files:** `tempfile::TempDir` + `lofty` to create real MP3 files with valid ID3 tags for index/org/tag tests.
|
||||
- **Integration tests:** `{crate}/tests/integration.rs` (async via `#[tokio::test]`)
|
||||
- **Unit tests:** `#[cfg(test)]` modules inline in source files for pure functions (sanitization, parsing, normalization, template rendering).
|
||||
|
||||
## Important Constraints
|
||||
|
||||
- **MusicBrainz rate limit:** 1 request per 1.1 seconds for the remote API. Mitigated by the local SQLite database (imported from MB dumps) and aggressive caching in `search_cache`.
|
||||
- **YouTube cookies expire** roughly every 2 weeks. Auto-refreshed by headless Firefox every 6 hours when cookie_refresh is enabled.
|
||||
- **Session key is random on startup** -- user sessions do not survive server restarts.
|
||||
- **Subsonic password is stored in plaintext** per the Subsonic protocol specification. Users are warned about this in the UI.
|
||||
- **Opus transcoding** for Subsonic clients transcodes the entire file to memory before streaming. Not ideal for very large files.
|
||||
|
||||
## Making Changes
|
||||
|
||||
- Backend route changes: edit files in `shanty-web/src/routes/`
|
||||
- Frontend changes: edit files in `shanty-web/frontend/src/`, then `cd shanty-web/frontend && trunk build`
|
||||
- Config changes: update `shanty-config/src/lib.rs` (add field + default), update `apply_env_overrides` if adding env var support
|
||||
- Database schema changes: add a migration in `shanty-db`, update entities and queries
|
||||
- Adding a new external data source: add a provider implementation in `shanty-data` behind the appropriate trait
|
||||
- Each crate submodule must be committed and pushed independently before updating the parent workspace
|
||||
Generated
+1
@@ -3465,6 +3465,7 @@ dependencies = [
|
||||
"chrono",
|
||||
"clap",
|
||||
"dirs",
|
||||
"futures-util",
|
||||
"hex",
|
||||
"md-5",
|
||||
"quick-xml",
|
||||
|
||||
+1
-1
@@ -37,9 +37,9 @@ COPY --from=backend /build/shanty-dl/scripts/cookie_manager.py /usr/share/shanty
|
||||
RUN mkdir -p /config /data /music
|
||||
|
||||
ENV SHANTY_CONFIG=/config/config.yaml
|
||||
ENV SHANTY_DATA_DIR=/data
|
||||
ENV SHANTY_DATABASE_URL=sqlite:///data/shanty.db?mode=rwc
|
||||
ENV SHANTY_LIBRARY_PATH=/music
|
||||
ENV SHANTY_DOWNLOAD_PATH=/data/downloads
|
||||
|
||||
EXPOSE 8085 6080
|
||||
|
||||
|
||||
+2
-5
@@ -3,14 +3,11 @@ services:
|
||||
build: .
|
||||
ports:
|
||||
- "8085:8085"
|
||||
- "6080:6080" # noVNC for YouTube login
|
||||
# - "6080:6080" # Optional: expose if YouTube login iframe doesn't load
|
||||
volumes:
|
||||
- ./config:/config
|
||||
- shanty-data:/data
|
||||
- ./data:/data # Database, downloads, MB data. Put on SSD if possible.
|
||||
- /path/to/music:/music # Change this to your music library path
|
||||
environment:
|
||||
- SHANTY_WEB_BIND=0.0.0.0
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
shanty-data:
|
||||
|
||||
+258
@@ -0,0 +1,258 @@
|
||||
# API Reference
|
||||
|
||||
Shanty exposes a REST API at `/api/*` and a Subsonic-compatible API at `/rest/*`. All REST endpoints require authentication via session cookie unless noted otherwise.
|
||||
|
||||
## Authentication
|
||||
|
||||
All `/api/*` endpoints (except the auth endpoints listed below) require an active session. To authenticate, call the login endpoint and include the returned session cookie in subsequent requests.
|
||||
|
||||
### Endpoints
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/api/auth/setup-required` | No | Check if initial setup is needed (returns `{"required": true}` if no users exist) |
|
||||
| POST | `/api/auth/setup` | No | Create the first admin user. Only works when no users exist. Body: `{"username": "...", "password": "..."}` |
|
||||
| POST | `/api/auth/login` | No | Log in. Body: `{"username": "...", "password": "..."}`. Returns user info and sets session cookie. |
|
||||
| POST | `/api/auth/logout` | Yes | Log out and clear session. |
|
||||
| GET | `/api/auth/me` | Yes | Get current user info (id, username, role). |
|
||||
| GET | `/api/auth/users` | Admin | List all users. |
|
||||
| POST | `/api/auth/users` | Admin | Create a new user. Body: `{"username": "...", "password": "..."}` |
|
||||
| DELETE | `/api/auth/users/{id}` | Admin | Delete a user. Cannot delete yourself. |
|
||||
| PUT | `/api/auth/subsonic-password` | Yes | Set Subsonic password for current user. Body: `{"password": "..."}` |
|
||||
| GET | `/api/auth/subsonic-password-status` | Yes | Check if current user has a Subsonic password set. |
|
||||
|
||||
---
|
||||
|
||||
## Artists
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/api/artists` | Yes | List all artists in the library. Supports `?limit=50&offset=0` pagination. Returns artist names, MBIDs, monitored status, and track counts (watched/owned/total). |
|
||||
| POST | `/api/artists` | Yes | Add (watch) an artist. Body: `{"name": "...", "mbid": "..."}`. At least one of name or mbid is required. Expands the artist's full discography into individual track wanted_items. |
|
||||
| GET | `/api/artists/{id}` | Yes | Get artist by local database ID. Returns artist info and local albums. |
|
||||
| GET | `/api/artists/{id}/full` | Yes | Get full artist detail with MusicBrainz enrichment. Accepts local ID or MBID. Returns discography with per-album status (owned/partial/wanted/unwatched), artist photo, bio, and banner. Supports `?quick=true` to skip per-album track fetches. |
|
||||
| DELETE | `/api/artists/{id}` | Admin | Delete an artist from the library. |
|
||||
| POST | `/api/artists/{id}/monitor` | Yes | Enable monitoring for an artist. |
|
||||
| DELETE | `/api/artists/{id}/monitor` | Yes | Disable monitoring for an artist. |
|
||||
|
||||
---
|
||||
|
||||
## Albums
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/api/albums` | Yes | List all local albums. Supports `?limit=50&offset=0`. |
|
||||
| POST | `/api/albums` | Yes | Add (watch) an album. Body: `{"artist": "...", "album": "...", "mbid": "..."}`. Accepts release or release-group MBID. Expands to individual track wanted_items. |
|
||||
| GET | `/api/albums/{mbid}` | Yes | Get album track listing by MBID (release or release-group). Returns tracks with per-track status. |
|
||||
|
||||
---
|
||||
|
||||
## Tracks
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/api/tracks` | Yes | List tracks. Supports `?limit=50&offset=0` and `?q=search` for text search. |
|
||||
| GET | `/api/tracks/{id}` | Yes | Get a single track by local database ID. |
|
||||
|
||||
---
|
||||
|
||||
## Search
|
||||
|
||||
Search queries MusicBrainz (either the local database or the remote API).
|
||||
|
||||
| Method | Path | Auth | Parameters | Description |
|
||||
|--------|------|------|------------|-------------|
|
||||
| GET | `/api/search/artist` | Yes | `q` (required), `limit` (default 25) | Search for artists by name. Returns MBID, name, disambiguation. |
|
||||
| GET | `/api/search/album` | Yes | `q` (required), `artist` (optional), `limit` (default 25) | Search for albums. |
|
||||
| GET | `/api/search/track` | Yes | `q` (required), `artist` (optional), `limit` (default 25) | Search for tracks. |
|
||||
| GET | `/api/search/discography/{mbid}` | Yes | (none) | Get an artist's full discography (release groups) by MBID. |
|
||||
|
||||
---
|
||||
|
||||
## Downloads
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/api/downloads/queue` | Yes | List download queue. Optional `?status=pending\|downloading\|completed\|failed\|cancelled`. |
|
||||
| POST | `/api/downloads` | Yes | Enqueue a manual download. Body: `{"query": "..."}`. |
|
||||
| POST | `/api/downloads/sync` | Yes | Sync wanted items to download queue. Returns count of found/enqueued/skipped. |
|
||||
| POST | `/api/downloads/process` | Yes | Start processing the download queue (background task). Returns a task_id. |
|
||||
| POST | `/api/downloads/retry/{id}` | Yes | Requeue a failed download. |
|
||||
| DELETE | `/api/downloads/{id}` | Yes | Cancel a download. |
|
||||
|
||||
---
|
||||
|
||||
## Playlists
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| POST | `/api/playlists/generate` | Yes | Generate a playlist (preview, not saved). Body includes strategy, seed_artists, count, etc. |
|
||||
| GET | `/api/playlists` | Yes | List saved playlists for the current user. |
|
||||
| POST | `/api/playlists` | Yes | Save a playlist. Body: `{"name": "...", "description": "...", "track_ids": [1, 2, 3]}` |
|
||||
| GET | `/api/playlists/{id}` | Yes | Get a playlist with its tracks. |
|
||||
| PUT | `/api/playlists/{id}` | Yes | Update playlist name/description. Body: `{"name": "...", "description": "..."}` |
|
||||
| DELETE | `/api/playlists/{id}` | Yes | Delete a playlist. |
|
||||
| GET | `/api/playlists/{id}/m3u` | Yes | Export playlist as M3U file download. |
|
||||
| POST | `/api/playlists/{id}/tracks` | Yes | Add a track. Body: `{"track_id": 123}` |
|
||||
| PUT | `/api/playlists/{id}/tracks` | Yes | Reorder tracks. Body: `{"track_ids": [3, 1, 2]}` |
|
||||
| DELETE | `/api/playlists/{id}/tracks/{track_id}` | Yes | Remove a track from the playlist. |
|
||||
|
||||
### Playlist Generation Body
|
||||
|
||||
```json
|
||||
{
|
||||
"strategy": "similar",
|
||||
"seed_artists": ["Pink Floyd", "Radiohead"],
|
||||
"count": 50,
|
||||
"popularity_bias": 0.5,
|
||||
"ordering": "interleave",
|
||||
"genres": [],
|
||||
"rules": []
|
||||
}
|
||||
```
|
||||
|
||||
Strategies: `similar`, `genre`, `random`, `smart`.
|
||||
Ordering options: `interleave`, `score`, `random`.
|
||||
|
||||
---
|
||||
|
||||
## Lyrics
|
||||
|
||||
| Method | Path | Auth | Parameters | Description |
|
||||
|--------|------|------|------------|-------------|
|
||||
| GET | `/api/lyrics` | Yes | `artist` (required), `title` (required) | Fetch lyrics from LRCLIB. Returns plain lyrics and time-synced lyrics if available. Cached for 30 days. |
|
||||
|
||||
---
|
||||
|
||||
## System and Pipeline
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/api/status` | Yes | Dashboard status: library summary, download queue, tagging queue, active tasks, scheduler info. |
|
||||
| POST | `/api/pipeline` | Yes | Trigger the full pipeline (sync + download + tag + organize). Returns task_ids. |
|
||||
| POST | `/api/index` | Yes | Trigger a library scan (background task). |
|
||||
| POST | `/api/tag` | Yes | Trigger tagging of untagged tracks (background task). |
|
||||
| POST | `/api/organize` | Yes | Trigger file organization (background task). |
|
||||
| GET | `/api/tasks/{id}` | Yes | Get task status (running/completed/failed, progress, message). |
|
||||
|
||||
---
|
||||
|
||||
## Watchlist
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/api/watchlist` | Yes | List wanted items for the current user. |
|
||||
| DELETE | `/api/watchlist/{id}` | Yes | Remove a wanted item. |
|
||||
|
||||
---
|
||||
|
||||
## YouTube Authentication
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/api/ytauth/status` | Yes | Get YouTube auth status: authenticated, cookie age, refresh status, yt-dlp version, API key status. |
|
||||
| POST | `/api/ytauth/login-start` | Admin | Launch Firefox + noVNC for interactive YouTube login. Returns VNC URL. |
|
||||
| POST | `/api/ytauth/login-stop` | Admin | Stop Firefox, export cookies, enable auto-refresh. |
|
||||
| POST | `/api/ytauth/refresh` | Admin | Trigger immediate headless cookie refresh. |
|
||||
| DELETE | `/api/ytauth/cookies` | Admin | Clear all cookies and Firefox profile, disable refresh. |
|
||||
|
||||
---
|
||||
|
||||
## Monitor
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| POST | `/api/monitor/check` | Admin | Trigger an immediate check of all monitored artists for new releases. |
|
||||
| GET | `/api/monitor/status` | Yes | List all monitored artists with last check timestamps. |
|
||||
|
||||
---
|
||||
|
||||
## Scheduler
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| POST | `/api/scheduler/skip-pipeline` | Admin | Skip the next scheduled pipeline run. |
|
||||
| POST | `/api/scheduler/skip-monitor` | Admin | Skip the next scheduled monitor check. |
|
||||
|
||||
---
|
||||
|
||||
## MusicBrainz
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/api/mb-status` | Yes | Check if local MusicBrainz database is available and get stats (artist/release/recording counts). |
|
||||
| POST | `/api/mb-import` | Admin | Trigger MusicBrainz database import (background task). Downloads dumps and imports. |
|
||||
|
||||
---
|
||||
|
||||
## Config
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/api/config` | Yes | Get current configuration. |
|
||||
| PUT | `/api/config` | Admin | Update configuration. Body is the full AppConfig JSON object. Writes to YAML file and updates in-memory config. |
|
||||
|
||||
---
|
||||
|
||||
## Subsonic API
|
||||
|
||||
The Subsonic API is served at `/rest/*` and uses the Subsonic authentication protocol (username + MD5 token or plaintext password, passed as query parameters).
|
||||
|
||||
All Subsonic endpoints accept the standard Subsonic query parameters: `u` (username), `p` (password or `enc:` hex-encoded), `t` (MD5 token), `s` (salt), `v` (API version), `c` (client name), `f` (response format, defaults to XML).
|
||||
|
||||
Each endpoint is available at both `/rest/endpoint` and `/rest/endpoint.view`.
|
||||
|
||||
### System
|
||||
|
||||
| Path | Description |
|
||||
|------|-------------|
|
||||
| `/rest/ping` | Server health check. |
|
||||
| `/rest/getLicense` | Returns license status (always valid). |
|
||||
|
||||
### Browsing
|
||||
|
||||
| Path | Description |
|
||||
|------|-------------|
|
||||
| `/rest/getMusicFolders` | List music library folders. |
|
||||
| `/rest/getIndexes` | Get alphabetical artist index. |
|
||||
| `/rest/getMusicDirectory` | Get contents of a directory (artist's albums or album's tracks). Parameter: `id`. |
|
||||
| `/rest/getArtists` | Get all artists (ID3 tag based). |
|
||||
| `/rest/getArtist` | Get an artist with albums. Parameter: `id`. |
|
||||
| `/rest/getAlbum` | Get an album with tracks. Parameter: `id`. |
|
||||
| `/rest/getSong` | Get a single track. Parameter: `id`. |
|
||||
| `/rest/getGenres` | List all genres. |
|
||||
|
||||
### Search
|
||||
|
||||
| Path | Description |
|
||||
|------|-------------|
|
||||
| `/rest/search3` | Search artists, albums, and tracks. Parameters: `query`, `artistCount`, `albumCount`, `songCount`, `artistOffset`, `albumOffset`, `songOffset`. |
|
||||
|
||||
### Media
|
||||
|
||||
| Path | Description |
|
||||
|------|-------------|
|
||||
| `/rest/stream` | Stream an audio file. Parameter: `id`. Transcodes Opus to MP3 if transcoding is enabled. |
|
||||
| `/rest/download` | Download an audio file (original format). Parameter: `id`. |
|
||||
| `/rest/getCoverArt` | Get cover art image. Parameter: `id`. |
|
||||
|
||||
### Playlists
|
||||
|
||||
| Path | Description |
|
||||
|------|-------------|
|
||||
| `/rest/getPlaylists` | List all playlists. |
|
||||
| `/rest/getPlaylist` | Get a playlist with tracks. Parameter: `id`. |
|
||||
| `/rest/createPlaylist` | Create or update a playlist. Parameters: `name`, `songId` (repeatable). |
|
||||
| `/rest/deletePlaylist` | Delete a playlist. Parameter: `id`. |
|
||||
|
||||
### Annotation
|
||||
|
||||
| Path | Description |
|
||||
|------|-------------|
|
||||
| `/rest/scrobble` | Record a play. Parameters: `id`, `submission` (true/false). |
|
||||
|
||||
### User
|
||||
|
||||
| Path | Description |
|
||||
|------|-------------|
|
||||
| `/rest/getUser` | Get user details. Parameter: `username`. |
|
||||
@@ -0,0 +1,461 @@
|
||||
# Configuration Reference
|
||||
|
||||
Shanty is configured via a YAML file and environment variables. Environment variables always override values from the YAML file.
|
||||
|
||||
## Config File Location
|
||||
|
||||
By default, Shanty looks for its config file at:
|
||||
|
||||
- **Linux:** `~/.config/shanty/config.yaml`
|
||||
- **Docker:** `/config/config.yaml`
|
||||
- **Custom:** Set the `SHANTY_CONFIG` environment variable to any path.
|
||||
|
||||
If no config file exists, Shanty uses sensible defaults for everything. You can also configure most settings from the web UI under **Settings**, which writes changes back to the YAML file.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
These environment variables override their corresponding YAML settings. In Docker, set them in the `environment` section of your `docker-compose.yml`.
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `SHANTY_CONFIG` | `~/.config/shanty/config.yaml` | Path to the YAML config file |
|
||||
| `SHANTY_DATA_DIR` | `~/.local/share/shanty` | Base directory for application data (database, downloads, Firefox profile). In Docker, this is `/data`. |
|
||||
| `SHANTY_DATABASE_URL` | `sqlite://~/.local/share/shanty/shanty.db?mode=rwc` | Full SQLite connection string |
|
||||
| `SHANTY_LIBRARY_PATH` | System music directory (`~/Music`) | Path to your music library |
|
||||
| `SHANTY_WEB_PORT` | `8085` | HTTP server port |
|
||||
| `SHANTY_WEB_BIND` | `0.0.0.0` | HTTP server bind address |
|
||||
| `SHANTY_LASTFM_API_KEY` | (none) | Last.fm API key for artist bios and similar-artist playlists. Get one at https://www.last.fm/api/account/create |
|
||||
| `SHANTY_FANART_API_KEY` | (none) | fanart.tv API key for artist images and banners. Get one at https://fanart.tv/get-an-api-key/ |
|
||||
|
||||
## Full Config Reference
|
||||
|
||||
Below is every configuration option grouped by section.
|
||||
|
||||
---
|
||||
|
||||
### Root Options
|
||||
|
||||
These are top-level keys in the YAML file.
|
||||
|
||||
#### `library_path`
|
||||
- **Type:** string (file path)
|
||||
- **Default:** System music directory (e.g., `~/Music`)
|
||||
- **Env override:** `SHANTY_LIBRARY_PATH`
|
||||
- **Description:** The root directory of your music library. Organized files are placed here according to the `organization_format` template.
|
||||
- **Example:** `library_path: /home/user/Music`
|
||||
|
||||
#### `database_url`
|
||||
- **Type:** string
|
||||
- **Default:** `sqlite://~/.local/share/shanty/shanty.db?mode=rwc`
|
||||
- **Env override:** `SHANTY_DATABASE_URL`
|
||||
- **Description:** SQLite database connection string. The `?mode=rwc` suffix means read-write-create (the database file is created automatically if it does not exist).
|
||||
- **Example:** `database_url: sqlite:///data/shanty.db?mode=rwc`
|
||||
|
||||
#### `download_path`
|
||||
- **Type:** string (file path)
|
||||
- **Default:** `~/.local/share/shanty/downloads`
|
||||
- **Description:** Temporary directory where yt-dlp stores downloaded files before they are tagged and organized into the library.
|
||||
- **Example:** `download_path: /tmp/shanty-downloads`
|
||||
|
||||
#### `organization_format`
|
||||
- **Type:** string (template)
|
||||
- **Default:** `{artist}/{album}/{track_number} - {title}.{ext}`
|
||||
- **Description:** Template for organizing files in the library. Available placeholders: `{artist}`, `{album}`, `{title}`, `{track_number}`, `{ext}`, `{year}`, `{genre}`.
|
||||
- **Example:** `organization_format: "{artist}/{album} ({year})/{track_number} - {title}.{ext}"`
|
||||
|
||||
#### `allowed_secondary_types`
|
||||
- **Type:** list of strings
|
||||
- **Default:** `[]` (empty, meaning studio albums only)
|
||||
- **Description:** Which secondary release group types to include when displaying an artist's discography. By default, only pure studio albums (with no secondary type) are shown. Add types to this list to also see compilations, live albums, etc.
|
||||
- **Options:** `Compilation`, `Live`, `Soundtrack`, `Remix`, `DJ-mix`, `Demo`
|
||||
- **Example:**
|
||||
```yaml
|
||||
allowed_secondary_types:
|
||||
- Compilation
|
||||
- Live
|
||||
```
|
||||
|
||||
#### `log_level`
|
||||
- **Type:** string
|
||||
- **Default:** `info`
|
||||
- **Description:** Log verbosity level. The `-v` CLI flag overrides this at runtime.
|
||||
- **Options:** `error`, `warn`, `info`, `debug`, `trace`
|
||||
- **Example:** `log_level: debug`
|
||||
|
||||
---
|
||||
|
||||
### `web` Section
|
||||
|
||||
Controls the HTTP server.
|
||||
|
||||
#### `web.port`
|
||||
- **Type:** integer
|
||||
- **Default:** `8085`
|
||||
- **Env override:** `SHANTY_WEB_PORT`
|
||||
- **Description:** Port the web server listens on.
|
||||
- **Example:**
|
||||
```yaml
|
||||
web:
|
||||
port: 9090
|
||||
```
|
||||
|
||||
#### `web.bind`
|
||||
- **Type:** string
|
||||
- **Default:** `0.0.0.0`
|
||||
- **Env override:** `SHANTY_WEB_BIND`
|
||||
- **Description:** Address the web server binds to. Use `0.0.0.0` to accept connections from any interface, or `127.0.0.1` to restrict to localhost.
|
||||
- **Example:**
|
||||
```yaml
|
||||
web:
|
||||
bind: 127.0.0.1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `tagging` Section
|
||||
|
||||
Controls automatic file tagging behavior.
|
||||
|
||||
#### `tagging.auto_tag`
|
||||
- **Type:** boolean
|
||||
- **Default:** `false`
|
||||
- **Description:** Whether to automatically tag files during indexing. When false, tagging must be triggered manually or as part of the pipeline.
|
||||
- **Example:**
|
||||
```yaml
|
||||
tagging:
|
||||
auto_tag: true
|
||||
```
|
||||
|
||||
#### `tagging.write_tags`
|
||||
- **Type:** boolean
|
||||
- **Default:** `true`
|
||||
- **Description:** Whether to actually write tags to audio files. When false, the tagger runs in dry-run mode (useful for testing).
|
||||
- **Example:**
|
||||
```yaml
|
||||
tagging:
|
||||
write_tags: false
|
||||
```
|
||||
|
||||
#### `tagging.confidence`
|
||||
- **Type:** float (0.0 to 1.0)
|
||||
- **Default:** `0.8`
|
||||
- **Description:** Minimum confidence threshold for fuzzy matching when the tagger tries to identify a track. Higher values mean stricter matching. Tracks below this threshold are skipped.
|
||||
- **Example:**
|
||||
```yaml
|
||||
tagging:
|
||||
confidence: 0.9
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `download` Section
|
||||
|
||||
Controls the download backend (yt-dlp).
|
||||
|
||||
#### `download.format`
|
||||
- **Type:** string
|
||||
- **Default:** `opus`
|
||||
- **Description:** Audio format for downloaded files.
|
||||
- **Options:** `opus`, `mp3`, `m4a`, `flac`
|
||||
- **Example:**
|
||||
```yaml
|
||||
download:
|
||||
format: mp3
|
||||
```
|
||||
|
||||
#### `download.search_source`
|
||||
- **Type:** string
|
||||
- **Default:** `ytmusic`
|
||||
- **Description:** Search backend for finding tracks to download.
|
||||
- **Options:** `ytmusic` (YouTube Music via ytmusicapi)
|
||||
- **Example:**
|
||||
```yaml
|
||||
download:
|
||||
search_source: ytmusic
|
||||
```
|
||||
|
||||
#### `download.cookies_path`
|
||||
- **Type:** string (file path) or null
|
||||
- **Default:** null (no cookies)
|
||||
- **Description:** Path to a Netscape-format cookies file for YouTube authentication. Usually managed automatically by the YouTube auth feature. When set, the authenticated rate limit is used.
|
||||
- **Example:**
|
||||
```yaml
|
||||
download:
|
||||
cookies_path: /data/cookies.txt
|
||||
```
|
||||
|
||||
#### `download.rate_limit`
|
||||
- **Type:** integer
|
||||
- **Default:** `250`
|
||||
- **Description:** Maximum download requests per hour when not authenticated. YouTube's actual limit is roughly 300; the default includes a safety margin.
|
||||
- **Example:**
|
||||
```yaml
|
||||
download:
|
||||
rate_limit: 200
|
||||
```
|
||||
|
||||
#### `download.rate_limit_auth`
|
||||
- **Type:** integer
|
||||
- **Default:** `1800`
|
||||
- **Description:** Maximum download requests per hour when authenticated with cookies. YouTube's actual limit is roughly 2000; the default includes a safety margin.
|
||||
- **Example:**
|
||||
```yaml
|
||||
download:
|
||||
rate_limit_auth: 1500
|
||||
```
|
||||
|
||||
#### `download.cookie_refresh_enabled`
|
||||
- **Type:** boolean
|
||||
- **Default:** `false`
|
||||
- **Description:** Whether to automatically refresh YouTube cookies using headless Firefox. Enabled automatically when you log in via the YouTube Authentication feature in Settings.
|
||||
- **Example:**
|
||||
```yaml
|
||||
download:
|
||||
cookie_refresh_enabled: true
|
||||
```
|
||||
|
||||
#### `download.cookie_refresh_hours`
|
||||
- **Type:** integer
|
||||
- **Default:** `6`
|
||||
- **Description:** How often (in hours) to refresh YouTube cookies. YouTube cookies typically expire after about 2 weeks, so refreshing every 6 hours keeps them fresh.
|
||||
- **Example:**
|
||||
```yaml
|
||||
download:
|
||||
cookie_refresh_hours: 12
|
||||
```
|
||||
|
||||
#### `download.vnc_port`
|
||||
- **Type:** integer
|
||||
- **Default:** `6080`
|
||||
- **Description:** Port for the noVNC web interface used during interactive YouTube login. This port must be accessible from your browser.
|
||||
- **Example:**
|
||||
```yaml
|
||||
download:
|
||||
vnc_port: 6081
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `indexing` Section
|
||||
|
||||
Controls the library scanner.
|
||||
|
||||
#### `indexing.concurrency`
|
||||
- **Type:** integer
|
||||
- **Default:** `4`
|
||||
- **Description:** Number of files to process concurrently during a library scan. Higher values use more CPU and memory but scan faster.
|
||||
- **Example:**
|
||||
```yaml
|
||||
indexing:
|
||||
concurrency: 8
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `metadata` Section
|
||||
|
||||
Controls where artist metadata, images, lyrics, and cover art come from.
|
||||
|
||||
#### `metadata.metadata_source`
|
||||
- **Type:** string
|
||||
- **Default:** `musicbrainz`
|
||||
- **Description:** Source for structured metadata (artist info, release data, track listings).
|
||||
- **Options:** `musicbrainz`
|
||||
- **Example:**
|
||||
```yaml
|
||||
metadata:
|
||||
metadata_source: musicbrainz
|
||||
```
|
||||
|
||||
#### `metadata.artist_image_source`
|
||||
- **Type:** string
|
||||
- **Default:** `wikipedia`
|
||||
- **Description:** Source for artist photos. Wikipedia works without an API key. fanart.tv provides higher quality images and background banners but requires `SHANTY_FANART_API_KEY`.
|
||||
- **Options:** `wikipedia`, `fanarttv`
|
||||
- **Example:**
|
||||
```yaml
|
||||
metadata:
|
||||
artist_image_source: fanarttv
|
||||
```
|
||||
|
||||
#### `metadata.artist_bio_source`
|
||||
- **Type:** string
|
||||
- **Default:** `wikipedia`
|
||||
- **Description:** Source for artist biographies. Wikipedia works without an API key. Last.fm provides more music-focused bios but requires `SHANTY_LASTFM_API_KEY`.
|
||||
- **Options:** `wikipedia`, `lastfm`
|
||||
- **Example:**
|
||||
```yaml
|
||||
metadata:
|
||||
artist_bio_source: lastfm
|
||||
```
|
||||
|
||||
#### `metadata.lyrics_source`
|
||||
- **Type:** string
|
||||
- **Default:** `lrclib`
|
||||
- **Description:** Source for song lyrics. LRCLIB provides both plain and time-synced lyrics.
|
||||
- **Options:** `lrclib`
|
||||
- **Example:**
|
||||
```yaml
|
||||
metadata:
|
||||
lyrics_source: lrclib
|
||||
```
|
||||
|
||||
#### `metadata.cover_art_source`
|
||||
- **Type:** string
|
||||
- **Default:** `coverartarchive`
|
||||
- **Description:** Source for album cover art.
|
||||
- **Options:** `coverartarchive`
|
||||
- **Example:**
|
||||
```yaml
|
||||
metadata:
|
||||
cover_art_source: coverartarchive
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `scheduling` Section
|
||||
|
||||
Controls automatic background tasks.
|
||||
|
||||
#### `scheduling.pipeline_enabled`
|
||||
- **Type:** boolean
|
||||
- **Default:** `true`
|
||||
- **Description:** Whether the download pipeline (sync, download, tag, organize) runs automatically on a schedule.
|
||||
- **Example:**
|
||||
```yaml
|
||||
scheduling:
|
||||
pipeline_enabled: false
|
||||
```
|
||||
|
||||
#### `scheduling.pipeline_interval_hours`
|
||||
- **Type:** integer
|
||||
- **Default:** `3`
|
||||
- **Description:** Hours between automatic pipeline runs. The timer starts after the previous run completes.
|
||||
- **Example:**
|
||||
```yaml
|
||||
scheduling:
|
||||
pipeline_interval_hours: 6
|
||||
```
|
||||
|
||||
#### `scheduling.monitor_enabled`
|
||||
- **Type:** boolean
|
||||
- **Default:** `true`
|
||||
- **Description:** Whether the monitor automatically checks for new releases from monitored artists.
|
||||
- **Example:**
|
||||
```yaml
|
||||
scheduling:
|
||||
monitor_enabled: false
|
||||
```
|
||||
|
||||
#### `scheduling.monitor_interval_hours`
|
||||
- **Type:** integer
|
||||
- **Default:** `12`
|
||||
- **Description:** Hours between automatic monitor checks.
|
||||
- **Example:**
|
||||
```yaml
|
||||
scheduling:
|
||||
monitor_interval_hours: 24
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `subsonic` Section
|
||||
|
||||
Controls the Subsonic-compatible streaming API.
|
||||
|
||||
#### `subsonic.enabled`
|
||||
- **Type:** boolean
|
||||
- **Default:** `true`
|
||||
- **Description:** Whether the Subsonic API is active. The API is served at `/rest/*` on the same port as the web UI.
|
||||
- **Example:**
|
||||
```yaml
|
||||
subsonic:
|
||||
enabled: false
|
||||
```
|
||||
|
||||
#### `subsonic.transcoding_enabled`
|
||||
- **Type:** boolean
|
||||
- **Default:** `true`
|
||||
- **Description:** Whether to transcode audio files (e.g., Opus to MP3) when streaming via the Subsonic API. Requires ffmpeg. Disable if your clients natively support your download format.
|
||||
- **Example:**
|
||||
```yaml
|
||||
subsonic:
|
||||
transcoding_enabled: false
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `musicbrainz` Section
|
||||
|
||||
Controls the local MusicBrainz database for faster lookups.
|
||||
|
||||
#### `musicbrainz.local_db_path`
|
||||
- **Type:** string (file path) or null
|
||||
- **Default:** null (auto-detected at `{data_dir}/shanty-mb.db`)
|
||||
- **Description:** Path to the local MusicBrainz SQLite database. If not set and the file exists at the default location, it is used automatically. If not set and the file does not exist, only the remote MusicBrainz API is used.
|
||||
- **Example:**
|
||||
```yaml
|
||||
musicbrainz:
|
||||
local_db_path: /data/shanty-mb.db
|
||||
```
|
||||
|
||||
#### `musicbrainz.auto_update`
|
||||
- **Type:** boolean
|
||||
- **Default:** `false`
|
||||
- **Description:** Whether to automatically re-download and re-import MusicBrainz dumps on a weekly schedule.
|
||||
- **Example:**
|
||||
```yaml
|
||||
musicbrainz:
|
||||
auto_update: true
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Full Example Config
|
||||
|
||||
```yaml
|
||||
library_path: /music
|
||||
download_path: /data/downloads
|
||||
organization_format: "{artist}/{album}/{track_number} - {title}.{ext}"
|
||||
log_level: info
|
||||
|
||||
allowed_secondary_types:
|
||||
- Compilation
|
||||
|
||||
web:
|
||||
port: 8085
|
||||
bind: 0.0.0.0
|
||||
|
||||
tagging:
|
||||
write_tags: true
|
||||
confidence: 0.8
|
||||
|
||||
download:
|
||||
format: opus
|
||||
search_source: ytmusic
|
||||
rate_limit: 250
|
||||
rate_limit_auth: 1800
|
||||
|
||||
indexing:
|
||||
concurrency: 4
|
||||
|
||||
metadata:
|
||||
metadata_source: musicbrainz
|
||||
artist_image_source: wikipedia
|
||||
artist_bio_source: wikipedia
|
||||
lyrics_source: lrclib
|
||||
cover_art_source: coverartarchive
|
||||
|
||||
scheduling:
|
||||
pipeline_enabled: true
|
||||
pipeline_interval_hours: 3
|
||||
monitor_enabled: true
|
||||
monitor_interval_hours: 12
|
||||
|
||||
subsonic:
|
||||
enabled: true
|
||||
transcoding_enabled: true
|
||||
|
||||
musicbrainz:
|
||||
auto_update: false
|
||||
```
|
||||
@@ -0,0 +1,84 @@
|
||||
# Artist Monitoring
|
||||
|
||||
Monitoring lets Shanty automatically detect when your favorite artists release new music. When a new album or single is found, its tracks are added to your watchlist and queued for download.
|
||||
|
||||
## Watching tracks vs. monitoring artists
|
||||
|
||||
In Shanty, **tracks** are what you watch. When you click "Watch All" on an artist page, every track in their current discography becomes a watched track. Those watched tracks get synced, downloaded, tagged, and organized by the pipeline.
|
||||
|
||||
**Monitoring** is different. Monitoring an artist means Shanty periodically checks MusicBrainz for new release groups (albums, singles, EPs) that did not exist when you last looked. When new releases are found, their tracks are automatically added to your watchlist.
|
||||
|
||||
In short:
|
||||
|
||||
- **Watching** is about tracks: "I want these specific tracks in my library."
|
||||
- **Monitoring** is about artists: "Tell me when this artist puts out something new, and watch those tracks for me automatically."
|
||||
|
||||
You can watch tracks from an artist without monitoring them (just grab their existing catalog and move on). You can also monitor an artist to stay current with future releases.
|
||||
|
||||
## How to set up
|
||||
|
||||
1. Open the Shanty web UI and navigate to an artist's page (search for them or open them from your library).
|
||||
2. Click the **Monitor** button on the artist page.
|
||||
3. The artist is now monitored. A "Monitored" badge appears next to their name, and a checkmark shows in the Library view.
|
||||
|
||||
To stop monitoring an artist, click the **Unmonitor** button on their page.
|
||||
|
||||
## How it works
|
||||
|
||||
When monitoring is enabled, Shanty runs a background check on a schedule:
|
||||
|
||||
1. For each monitored artist, it fetches the current list of release groups from MusicBrainz.
|
||||
2. It compares this against the tracks already in your watchlist.
|
||||
3. Any new release groups are expanded into individual tracks and added to the watchlist.
|
||||
4. The artist's `last_checked_at` timestamp is updated.
|
||||
|
||||
If you have the pipeline scheduler enabled (the default), new tracks are automatically downloaded on the next pipeline run.
|
||||
|
||||
## Automatic scheduling
|
||||
|
||||
Both the monitor check and the download pipeline run on configurable schedules:
|
||||
|
||||
| Task | Default interval | Config key |
|
||||
|------|-----------------|------------|
|
||||
| Monitor check | Every 12 hours | `scheduling.monitor_interval_hours` |
|
||||
| Pipeline (Set Sail) | Every 3 hours | `scheduling.pipeline_interval_hours` |
|
||||
|
||||
This means that in the default configuration:
|
||||
|
||||
1. The monitor checks for new releases every 12 hours.
|
||||
2. If new releases are found, their tracks are added to the watchlist.
|
||||
3. Within 3 hours, the pipeline runs automatically, downloading and processing the new tracks.
|
||||
|
||||
## Configuring schedules
|
||||
|
||||
In your config file:
|
||||
|
||||
```yaml
|
||||
scheduling:
|
||||
pipeline_enabled: true
|
||||
pipeline_interval_hours: 3
|
||||
monitor_enabled: true
|
||||
monitor_interval_hours: 12
|
||||
```
|
||||
|
||||
Or adjust these settings in the web UI under **Settings**.
|
||||
|
||||
To disable automatic scheduling entirely:
|
||||
|
||||
```yaml
|
||||
scheduling:
|
||||
pipeline_enabled: false
|
||||
monitor_enabled: false
|
||||
```
|
||||
|
||||
When disabled, you can still trigger checks manually from the Dashboard.
|
||||
|
||||
## Manual checks
|
||||
|
||||
You can trigger a monitor check at any time from the Dashboard by clicking **Check Monitored Artists** (under the individual actions). The results show how many artists were checked, how many new releases were found, and how many tracks were added.
|
||||
|
||||
Similarly, you can trigger the pipeline manually by clicking **Set Sail** at any time.
|
||||
|
||||
## Skipping scheduled runs
|
||||
|
||||
If you want to skip the next scheduled pipeline or monitor run without disabling them permanently, you can do so from the Dashboard. The skip is a one-time action -- the schedule resumes normally after the skipped run.
|
||||
@@ -0,0 +1,80 @@
|
||||
# MusicBrainz Local Database
|
||||
|
||||
MusicBrainz is the source of all music metadata in Shanty: artist information, album listings, track listings, and release data. By default, Shanty queries the MusicBrainz API over the internet, which is rate-limited to 1 request every 1.1 seconds.
|
||||
|
||||
Importing the MusicBrainz database locally eliminates this bottleneck for most lookups.
|
||||
|
||||
## What this does
|
||||
|
||||
The import downloads a full copy of the MusicBrainz database (as JSON dumps) and loads it into a local SQLite database on your machine. When Shanty needs to look up an artist's discography, release track listings, or recording details, it checks the local database first. If the data is found locally, the response is instant. If not (for example, very new releases), it falls back to the remote API.
|
||||
|
||||
## Why you might want this
|
||||
|
||||
Without the local database, browsing a new artist's full discography requires many API calls (one per release group, one per release to get track listings). With the rate limit, loading an artist with 15 albums can take 30-60 seconds.
|
||||
|
||||
With the local database, the same operation completes in under a second.
|
||||
|
||||
If you are casually using Shanty for a few artists, the caching system works well enough without the local database. But if you frequently browse and add new artists, the local database makes the experience dramatically faster.
|
||||
|
||||
## Requirements
|
||||
|
||||
- **Download size:** approximately 24 GB (compressed JSON dumps from MusicBrainz)
|
||||
- **Database size:** approximately 16 GB (the resulting SQLite database)
|
||||
- **Disk type:** SSD strongly recommended for best performance. On an SSD, artist pages load in under a second. On a spinning hard drive, the database still works and is a big improvement over the API (a few seconds per page load instead of 30-60 seconds), but you will notice the difference compared to SSD.
|
||||
- **Time:** 12-24 hours for the initial import, depending on your hardware and disk speed.
|
||||
- **Disk space total:** You need roughly 40 GB free during import (dumps + database). After import, you can delete the dump files to reclaim the 24 GB.
|
||||
|
||||
## How to set up via the web UI
|
||||
|
||||
1. Open the Shanty web UI and go to **Settings**.
|
||||
2. Scroll to the **MusicBrainz Database** section.
|
||||
3. Click **Import**.
|
||||
4. Shanty downloads the dump files and imports them. Progress is shown on the Dashboard.
|
||||
|
||||
You can continue using Shanty while the import runs. It will use the remote API until the import completes.
|
||||
|
||||
## How to set up via the CLI
|
||||
|
||||
If you prefer the command line (or want to run the import on a more powerful machine):
|
||||
|
||||
```sh
|
||||
# Inside Docker
|
||||
docker compose exec shanty ./shanty mb-import --download
|
||||
|
||||
# From source
|
||||
cargo run --release --bin shanty -- mb-import --download
|
||||
```
|
||||
|
||||
Options:
|
||||
- `--download` -- Download fresh dump files from metabrainz.org before importing. Without this flag, the import looks for existing dump files in the data directory.
|
||||
- `--data-dir /path/to/dumps` -- Custom directory for dump files. Defaults to `{data_dir}/mb-dumps`.
|
||||
|
||||
The database is created at `{data_dir}/shanty-mb.db` by default. You can override this with the `musicbrainz.local_db_path` config option.
|
||||
|
||||
## Automatic weekly updates
|
||||
|
||||
MusicBrainz publishes new database dumps regularly. You can configure Shanty to automatically re-download and re-import them:
|
||||
|
||||
```yaml
|
||||
musicbrainz:
|
||||
auto_update: true
|
||||
```
|
||||
|
||||
When enabled, Shanty checks weekly for new dumps and runs the import in the background if a newer dump is available. The existing local database continues to serve queries during the update.
|
||||
|
||||
## The hybrid approach
|
||||
|
||||
Shanty uses what it calls a "hybrid fetcher":
|
||||
|
||||
1. **Local database** is checked first for any MusicBrainz lookup (artist info, release groups, track listings, etc.).
|
||||
2. **Remote API** is used as a fallback when data is not in the local database (new releases added after the last import, for example).
|
||||
3. **Cache** is used for data that was fetched from the remote API, so repeated lookups do not hit the rate limit.
|
||||
|
||||
This means you get the best of both worlds: instant lookups for the vast majority of music that exists in the database dumps, and up-to-date data for newly released music via the API.
|
||||
|
||||
## Configuration options
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `musicbrainz.local_db_path` | `{data_dir}/shanty-mb.db` | Path to the local SQLite database |
|
||||
| `musicbrainz.auto_update` | `false` | Automatically re-import dumps weekly |
|
||||
@@ -0,0 +1,71 @@
|
||||
# Playlists
|
||||
|
||||
Shanty can generate playlists from your library using similar-artist data from Last.fm. You can also create and edit playlists manually, export them as M3U files, and access them from Subsonic clients.
|
||||
|
||||
## Requirements
|
||||
|
||||
Playlist generation requires a Last.fm API key. Set it via the `SHANTY_LASTFM_API_KEY` environment variable:
|
||||
|
||||
```yaml
|
||||
# In docker-compose.yml
|
||||
environment:
|
||||
- SHANTY_LASTFM_API_KEY=your_api_key_here
|
||||
```
|
||||
|
||||
Get a free API key at: https://www.last.fm/api/account/create
|
||||
|
||||
When creating your API account, use any application name and leave the callback URL blank or enter `http://localhost`.
|
||||
|
||||
## Generating a playlist
|
||||
|
||||
1. Open the Shanty web UI and go to **Playlists**.
|
||||
2. Click the **Generate** tab.
|
||||
3. Search for and select one or more **seed artists** from your library. The dropdown shows all artists you have tracks for.
|
||||
4. Adjust **popularity bias** (higher values favor well-known tracks, lower values dig deeper).
|
||||
5. Set the **track count** (how many tracks in the playlist).
|
||||
6. Choose **ordering**:
|
||||
- **Interleave** -- spreads artists evenly throughout the playlist, avoiding back-to-back tracks from the same artist.
|
||||
- **By Score** -- highest-scored tracks first (most similar to your seeds, weighted by popularity).
|
||||
- **Random** -- fully shuffled.
|
||||
7. Click **Generate**.
|
||||
|
||||
The generated playlist is shown as a preview. You can then save it or adjust the parameters and regenerate.
|
||||
|
||||
### How generation works
|
||||
|
||||
Shanty uses Last.fm's similar-artist data to find tracks from artists related to your seeds. For example, if you seed with "Radiohead," the playlist might include tracks by Radiohead, Thom Yorke, Portishead, Massive Attack, and other similar artists -- but only tracks that actually exist in your library.
|
||||
|
||||
The **popularity bias** slider (0-10) controls the balance:
|
||||
- Higher values favor popular, well-known tracks from each artist.
|
||||
- Lower values give more equal weight to deep cuts and lesser-known tracks.
|
||||
|
||||
Multiple seed artists are blended together -- artists similar to more of your seeds get higher scores.
|
||||
|
||||
## Creating a blank playlist
|
||||
|
||||
From the **Saved** tab, click **New** to create an empty playlist and open it in the editor. You can then add tracks manually.
|
||||
|
||||
## Saving a generated playlist
|
||||
|
||||
After generating a playlist, enter a name and click **Save Playlist**. Saved playlists appear in the **Saved** tab.
|
||||
|
||||
## Editing a playlist
|
||||
|
||||
1. Go to the **Saved** tab in Playlists.
|
||||
2. Click **Edit** on a playlist.
|
||||
3. In the Edit tab, you can:
|
||||
- **Drag and drop** tracks to reorder them.
|
||||
- **Remove** tracks by clicking the remove button next to each track.
|
||||
- **Add** tracks by searching in the track picker at the bottom.
|
||||
- **Rename** the playlist.
|
||||
4. Click **Save** when done.
|
||||
|
||||
## Exporting as M3U
|
||||
|
||||
From the **Saved** tab, click **M3U** on any playlist to download it as an M3U file. M3U files are compatible with most music players and contain file paths relative to your library.
|
||||
|
||||
## Playlists via Subsonic
|
||||
|
||||
Saved playlists are available through the Subsonic API. Any Subsonic client that supports playlists (most do) can see and play your Shanty playlists.
|
||||
|
||||
To use this, make sure you have [Subsonic set up](subsonic.md) with a Subsonic password configured.
|
||||
@@ -0,0 +1,96 @@
|
||||
# Subsonic Streaming
|
||||
|
||||
Shanty includes a Subsonic-compatible API that lets you stream your music library to mobile apps, desktop players, and other Subsonic clients.
|
||||
|
||||
## What is Subsonic?
|
||||
|
||||
Subsonic is a widely-supported protocol for streaming music from a personal server. Many music apps implement the Subsonic API, which means you can use any of them to stream your Shanty library to your phone, tablet, or desktop.
|
||||
|
||||
## Important note
|
||||
|
||||
Shanty implements a subset of the Subsonic API -- enough for browsing, streaming, playlists, and search. If you need a full-featured Subsonic server with advanced features like sharing, podcasts, or Internet radio, consider running [Navidrome](https://www.navidrome.org/) pointed at the same music library directory. Both can coexist.
|
||||
|
||||
## How to set up
|
||||
|
||||
### 1. Set a Subsonic password
|
||||
|
||||
The Subsonic protocol uses its own authentication system, separate from your Shanty web login.
|
||||
|
||||
1. Open the Shanty web UI and go to **Settings**.
|
||||
2. Find the **Subsonic API** section.
|
||||
3. Enter a password and click **Save**.
|
||||
|
||||
**Important -- please read:** The Subsonic password is stored as **plain text** in the database. This is not a bug or an oversight. The Subsonic protocol requires the server to verify authentication by computing `md5(password + client_salt)`, which means the server must have access to the original password. There is no way to store it securely (like a one-way hash) and still be compatible with the protocol. This is a well-known limitation of the Subsonic standard and is how all Subsonic-compatible servers handle it, including Navidrome.
|
||||
|
||||
Because of this, **do not reuse a password from any other account**. Choose a simple, unique password that you use only for Subsonic access. Your Shanty web login password is stored securely (Argon2id hash) and is completely separate.
|
||||
|
||||
### 2. Configure your client
|
||||
|
||||
In your Subsonic client app, add a new server with these settings:
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Server URL** | `http://your-server-ip:8085` |
|
||||
| **Username** | Your Shanty username |
|
||||
| **Password** | The Subsonic password you set (not your web login password) |
|
||||
|
||||
Most clients automatically append `/rest` to the server URL. If your client asks for just the base URL, enter `http://your-server-ip:8085`.
|
||||
|
||||
If your client has separate fields for server address and port, enter the IP/hostname and `8085` separately.
|
||||
|
||||
### 3. Test the connection
|
||||
|
||||
Most clients have a "Test Connection" button. Use it to verify that the server is reachable and your credentials are correct.
|
||||
|
||||
## Recommended clients
|
||||
|
||||
These clients have been tested with Shanty's Subsonic implementation:
|
||||
|
||||
### Android
|
||||
- **[Ultrasonic](https://f-droid.org/packages/org.moire.ultrasonic/)** -- Free and open source. Available on F-Droid and Google Play. Reliable and well-maintained.
|
||||
- **[DSub](https://play.google.com/store/apps/details?id=github.daneren2005.dsub)** -- Feature-rich, supports offline caching.
|
||||
- **[Symfonium](https://play.google.com/store/apps/details?id=app.symfonik.music.player)** -- Modern UI, excellent playback features. Paid app.
|
||||
|
||||
### Desktop
|
||||
- **[Feishin](https://github.com/jeffvli/feishin)** -- Cross-platform desktop client with a modern interface. Free and open source.
|
||||
|
||||
## Supported Subsonic endpoints
|
||||
|
||||
Shanty implements these Subsonic API endpoints:
|
||||
|
||||
- **System:** `ping`, `getLicense`
|
||||
- **Browsing:** `getMusicFolders`, `getIndexes`, `getMusicDirectory`, `getArtists`, `getArtist`, `getAlbum`, `getSong`, `getGenres`
|
||||
- **Search:** `search3`
|
||||
- **Media:** `stream`, `download`, `getCoverArt`
|
||||
- **Playlists:** `getPlaylists`, `getPlaylist`, `createPlaylist`, `deletePlaylist`
|
||||
- **Annotation:** `scrobble`
|
||||
- **User:** `getUser`
|
||||
|
||||
## Transcoding
|
||||
|
||||
If your music is in Opus format (the default download format), many mobile clients cannot play it directly. Shanty automatically transcodes Opus files to MP3 when streaming via the Subsonic API. This requires ffmpeg, which is included in the Docker image.
|
||||
|
||||
If you download in MP3 format or your clients support Opus natively, you can disable transcoding:
|
||||
|
||||
```yaml
|
||||
subsonic:
|
||||
transcoding_enabled: false
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**"Authentication failed" in client:**
|
||||
- Make sure you are using the Subsonic password, not your web login password.
|
||||
- Make sure the username matches exactly (case-sensitive).
|
||||
|
||||
**Client cannot connect:**
|
||||
- Verify the server URL includes the port (`http://ip:8085`, not just `http://ip`).
|
||||
- Check that port 8085 is accessible from the device (firewall rules, same network, etc.).
|
||||
|
||||
**No music showing up:**
|
||||
- Music must be organized in the library (run the pipeline at least once) to appear in the Subsonic API.
|
||||
- Only tracks with file paths in the database are served.
|
||||
|
||||
**Audio does not play:**
|
||||
- If using Opus format, make sure transcoding is enabled (it is by default).
|
||||
- Check that ffmpeg is installed (included in Docker, must be installed manually for source builds).
|
||||
@@ -0,0 +1,58 @@
|
||||
# YouTube Authentication
|
||||
|
||||
Shanty downloads music from YouTube via yt-dlp. By default, it makes requests as a guest, which limits you to roughly 250 downloads per hour. Authenticating with a Google account increases this to roughly 1800 downloads per hour and grants access to age-restricted content.
|
||||
|
||||
## Should you set this up?
|
||||
|
||||
If you are downloading just a few albums at a time, guest mode is fine. If you are adding several artists at once or building a large library, authentication will save you a lot of waiting.
|
||||
|
||||
## Warning: Use a throwaway account
|
||||
|
||||
There is a small but real risk that Google may flag your account for unusual activity. Automated downloading is against YouTube's terms of service. To protect yourself:
|
||||
|
||||
- Create a new Google account specifically for this purpose.
|
||||
- Do not use your main Google account.
|
||||
- Do not use an account tied to important services (Gmail, Google Drive, etc.).
|
||||
|
||||
## How to set up
|
||||
|
||||
1. Open the Shanty web UI and go to **Settings**.
|
||||
2. Scroll to the **YouTube Authentication** section.
|
||||
3. Click **Authenticate**.
|
||||
4. A noVNC browser window opens, showing Firefox inside the Shanty container.
|
||||
5. Navigate to YouTube and log in with your throwaway Google account.
|
||||
6. Once you are logged in, go back to the Shanty settings page and click **Done**.
|
||||
|
||||
Shanty extracts the YouTube cookies from Firefox and saves them. From this point forward, all downloads use your authenticated session.
|
||||
|
||||
## How auto-refresh works
|
||||
|
||||
YouTube cookies expire roughly every 2 weeks. Shanty automatically refreshes them using headless Firefox (no visible browser window needed):
|
||||
|
||||
- After you complete the login process, auto-refresh is enabled automatically.
|
||||
- Every 6 hours (configurable via `download.cookie_refresh_hours`), Shanty launches Firefox in headless mode, loads YouTube using the saved profile, and exports fresh cookies.
|
||||
- If a refresh fails, Shanty logs a warning and tries again at the next interval.
|
||||
|
||||
You can check the current cookie status in **Settings** under **YouTube Authentication**. It shows whether cookies are present, how old they are, and whether auto-refresh is enabled.
|
||||
|
||||
## Manual refresh
|
||||
|
||||
If cookies have expired and auto-refresh is not working, you can:
|
||||
|
||||
1. Click **Refresh** in the YouTube Authentication settings to trigger an immediate headless refresh.
|
||||
2. If that fails, click **Authenticate** to log in again through noVNC.
|
||||
|
||||
## Clearing cookies
|
||||
|
||||
To remove all YouTube authentication data, click **Clear** in the YouTube Authentication settings. This deletes the cookies file and the Firefox profile, and disables auto-refresh.
|
||||
|
||||
## Configuration options
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `download.cookies_path` | (auto) | Path to the cookies file. Managed automatically. |
|
||||
| `download.cookie_refresh_enabled` | `false` | Auto-enabled after login. |
|
||||
| `download.cookie_refresh_hours` | `6` | Hours between refresh attempts. |
|
||||
| `download.rate_limit` | `250` | Requests/hour without cookies. |
|
||||
| `download.rate_limit_auth` | `1800` | Requests/hour with cookies. |
|
||||
| `download.vnc_port` | `6080` | Port for the noVNC login interface. |
|
||||
@@ -0,0 +1,148 @@
|
||||
# Getting Started
|
||||
|
||||
This guide walks you through setting up Shanty from scratch. By the end, you will have a running instance that can search for music, download it, and play it on your phone.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
You need Docker and Docker Compose installed on your computer or server. If you do not have them:
|
||||
|
||||
- **Linux:** Follow the [Docker install guide](https://docs.docker.com/engine/install/) for your distribution.
|
||||
- **Mac/Windows:** Install [Docker Desktop](https://www.docker.com/products/docker-desktop/).
|
||||
|
||||
You also need a folder where you want your music library stored.
|
||||
|
||||
## Step 1: Create a docker-compose.yml
|
||||
|
||||
Create a new folder for your Shanty configuration, then create a file called `docker-compose.yml` inside it:
|
||||
|
||||
```sh
|
||||
mkdir shanty && cd shanty
|
||||
```
|
||||
|
||||
Create `docker-compose.yml` with these contents:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
shanty:
|
||||
image: git.rcjohnstone.com/connor/shanty:latest
|
||||
ports:
|
||||
- "8085:8085"
|
||||
# - "6080:6080" # Optional: expose if YouTube login iframe doesn't load
|
||||
volumes:
|
||||
- ./config:/config
|
||||
- ./data:/data
|
||||
- /path/to/your/music:/music
|
||||
environment:
|
||||
- SHANTY_WEB_BIND=0.0.0.0
|
||||
restart: unless-stopped
|
||||
```
|
||||
|
||||
Replace `/path/to/your/music` with the actual path to where you want your music stored. For example, `/home/yourname/Music` on Linux or `/Users/yourname/Music` on Mac.
|
||||
|
||||
### What the volumes do
|
||||
|
||||
| Mount point | Purpose |
|
||||
|-------------|---------|
|
||||
| `./config` | Stores your configuration file (`config.yaml`). Lives next to your docker-compose.yml. |
|
||||
| `./data` | Stores the database, downloaded files, and MusicBrainz data. Put this on an SSD if possible. |
|
||||
| `/music` | Your music library. This is where organized, tagged music ends up. |
|
||||
|
||||
## Step 2: Start the container
|
||||
|
||||
From the folder containing your `docker-compose.yml`:
|
||||
|
||||
```sh
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Wait a few seconds for the container to start. You can check the logs with:
|
||||
|
||||
```sh
|
||||
docker compose logs -f
|
||||
```
|
||||
|
||||
You should see a line like `starting server bind=0.0.0.0:8085`. Press Ctrl+C to stop watching logs (the container keeps running).
|
||||
|
||||
## Step 3: Create your account
|
||||
|
||||
Open your browser and go to:
|
||||
|
||||
```
|
||||
http://localhost:8085
|
||||
```
|
||||
|
||||
If Shanty is running on another machine, replace `localhost` with that machine's IP address.
|
||||
|
||||
On first launch, you will see a setup screen. Choose a username and password (at least 4 characters). This creates an admin account.
|
||||
|
||||
## Step 4: Search and watch an artist
|
||||
|
||||
1. Click the **Search** tab in the navigation.
|
||||
2. Type an artist name (for example, "Pink Floyd") and press Enter.
|
||||
3. Click on the artist in the search results to view their discography.
|
||||
4. Click **Watch All** to add their entire discography to your watchlist. Or click individual albums to watch specific ones.
|
||||
|
||||
You will see the artist appear on the **Library** page with track counts showing how many tracks are wanted versus owned.
|
||||
|
||||
**Note:** Browsing artist discographies may be slow on the first visit (30-60 seconds per artist) because Shanty queries MusicBrainz one request at a time. This gets much faster after the first visit (results are cached). For instant lookups, you can [import the MusicBrainz database locally](features/musicbrainz-db.md) -- but this is optional and can be done later.
|
||||
|
||||
## Step 5: Download your music
|
||||
|
||||
Click the **Set Sail** button on the Dashboard. This runs the full pipeline:
|
||||
|
||||
1. **Sync** -- creates download jobs for all wanted tracks
|
||||
2. **Download** -- fetches audio from YouTube Music via yt-dlp
|
||||
3. **Tag** -- writes MusicBrainz metadata to the files
|
||||
4. **Organize** -- moves files into your music library in a clean folder structure
|
||||
|
||||
You can watch progress on the Dashboard. The download queue shows each track as it is processed.
|
||||
|
||||
Depending on how many tracks you are downloading, this can take a while. YouTube has rate limits (roughly 250 downloads per hour without authentication). See the [YouTube Authentication](features/youtube-auth.md) guide to increase this to roughly 1800 per hour.
|
||||
|
||||
## Step 6 (Optional): Set up Subsonic for mobile playback
|
||||
|
||||
Shanty includes a Subsonic-compatible API that lets you stream your library from mobile apps like Ultrasonic, DSub, or Symfonium.
|
||||
|
||||
1. In the Shanty web UI, go to **Settings**.
|
||||
2. Find the **Subsonic API** section.
|
||||
3. Set a Subsonic password. This is separate from your web login password.
|
||||
4. On your phone, install a Subsonic client (Ultrasonic for Android is free and works well).
|
||||
5. In the client, add a server with the URL `http://your-server-ip:8085`. The client adds `/rest` automatically.
|
||||
6. Enter your Shanty username and the Subsonic password you just set.
|
||||
|
||||
See the [Subsonic guide](features/subsonic.md) for more details and recommended clients.
|
||||
|
||||
## Step 7 (Optional): Set up YouTube authentication
|
||||
|
||||
Without authentication, YouTube limits you to roughly 250 downloads per hour. With authentication, this increases to roughly 1800 per hour. You also get access to age-restricted content.
|
||||
|
||||
1. Go to **Settings** in the Shanty web UI.
|
||||
2. Find the **YouTube Authentication** section and click **Authenticate**.
|
||||
3. A browser window opens in your browser via noVNC. Log in to a Google account.
|
||||
4. Click **Done** when finished.
|
||||
|
||||
**Important:** Use a throwaway Google account. There is a small risk of account restrictions. See the [YouTube Authentication guide](features/youtube-auth.md) for details.
|
||||
|
||||
## Step 8 (Optional): Import the MusicBrainz database
|
||||
|
||||
MusicBrainz has a rate limit of 1 request every 1.1 seconds. When you browse a new artist, loading their full discography can take 30-60 seconds. Importing the MusicBrainz database locally makes these lookups instant.
|
||||
|
||||
This requires about 24 GB of download space, produces a 16 GB database, and takes 12-24 hours for the initial import. An SSD is strongly recommended.
|
||||
|
||||
To start the import from the web UI:
|
||||
1. Go to **Settings**.
|
||||
2. Find the **MusicBrainz Database** section.
|
||||
3. Click **Import**.
|
||||
|
||||
Or from the command line:
|
||||
```sh
|
||||
docker compose exec shanty ./shanty mb-import --download
|
||||
```
|
||||
|
||||
See the [MusicBrainz Database guide](features/musicbrainz-db.md) for more details.
|
||||
|
||||
## What next?
|
||||
|
||||
- Set up [monitoring](features/monitoring.md) to automatically detect new releases from your favorite artists.
|
||||
- Generate [playlists](features/playlists.md) from your library using similar-artist data.
|
||||
- Check the [Configuration Reference](configuration.md) to customize download formats, organization templates, and scheduling.
|
||||
@@ -1,172 +1,86 @@
|
||||
# shanty
|
||||
# Shanty
|
||||
|
||||
A modular, self-hosted music management application for the high seas. Shanty aims to be a
|
||||
better alternative to Lidarr — managing, tagging, organizing, and downloading music, all
|
||||
built as a collection of standalone Rust tools that work together seamlessly through a web UI.
|
||||
[](docs/getting-started.md)
|
||||
[](docs/api.md)
|
||||
[](docs/configuration.md)
|
||||
[](LICENSE)
|
||||
|
||||
Shanty is a self-hosted music management application. It searches MusicBrainz for artists and albums, downloads music from YouTube, tags and organizes files, and serves your library over the Subsonic protocol for streaming on any device.
|
||||
|
||||
## Features
|
||||
|
||||
- **Search** MusicBrainz for artists, albums, and tracks
|
||||
- **Watch** artists/albums — automatically expands to individual track-level monitoring
|
||||
- **Download** music via yt-dlp with YouTube Music search (ytmusicapi)
|
||||
- **Tag** files with MusicBrainz metadata (fuzzy matching + MBID-based lookup)
|
||||
- **Watch** artists or albums to track their full discography at the individual track level
|
||||
- **Download** music via yt-dlp with YouTube Music search
|
||||
- **Tag** files automatically using MusicBrainz metadata
|
||||
- **Organize** files into clean directory structures with configurable templates
|
||||
- **Web UI** — Yew (Rust/WASM) dashboard with real-time status, search, library browser
|
||||
- **Stream** your library to any Subsonic-compatible music app
|
||||
- **Playlists** generated from similar-artist data (Last.fm), with drag-and-drop editing and M3U export
|
||||
- **Monitor** artists for new releases, automatically added to your watchlist
|
||||
- **Web UI** built in Rust/WASM for managing everything from your browser
|
||||
|
||||
## Architecture
|
||||
## Quick Start with Docker
|
||||
|
||||
Shanty is a Cargo workspace where each component is its own crate and git submodule.
|
||||
Each crate is both a library (for the web app) and a standalone CLI binary.
|
||||
This is the recommended way to run Shanty.
|
||||
|
||||
| Crate | Description | Status |
|
||||
|-------|-------------|--------|
|
||||
| `shanty-db` | Sea-ORM + SQLite schema, migrations, queries | Done |
|
||||
| `shanty-index` | Music file scanning and metadata extraction (lofty) | Done |
|
||||
| `shanty-tag` | MusicBrainz client, fuzzy matching, file tag writing | Done |
|
||||
| `shanty-org` | File organization with configurable format templates | Done |
|
||||
| `shanty-watch` | Watchlist management, MB discography expansion | Done |
|
||||
| `shanty-dl` | yt-dlp backend, rate limiting, download queue | Done |
|
||||
| `shanty-search` | SearchProvider trait, MB search + release groups | Done |
|
||||
| `shanty-web` | Actix backend + Yew frontend | Done (MVP) |
|
||||
| `shanty-notify` | Notifications (Apprise, webhooks) | Stub |
|
||||
| `shanty-playlist` | Playlist generation | Stub |
|
||||
| `shanty-serve` | Subsonic-compatible music streaming | Stub |
|
||||
| `shanty-play` | Built-in web player | Stub |
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Rust (edition 2024)
|
||||
- [yt-dlp](https://github.com/yt-dlp/yt-dlp) + ffmpeg
|
||||
- Python 3 + `pip install ytmusicapi`
|
||||
- [Trunk](https://trunkrs.dev/) (for building the frontend)
|
||||
|
||||
### Build
|
||||
|
||||
```sh
|
||||
# Backend (all crates)
|
||||
cargo build --workspace
|
||||
|
||||
# Frontend (Yew/WASM)
|
||||
cd shanty-web/frontend && trunk build
|
||||
```
|
||||
|
||||
### Run the web app
|
||||
|
||||
```sh
|
||||
cargo run --bin shanty-web -- -v
|
||||
# Open http://localhost:8085
|
||||
```
|
||||
|
||||
### CLI tools
|
||||
|
||||
Each crate also works as a standalone CLI:
|
||||
|
||||
```sh
|
||||
# Index a music directory
|
||||
cargo run --bin shanty-index -- /path/to/music -v
|
||||
|
||||
# Tag untagged tracks
|
||||
cargo run --bin shanty-tag -- --all --write-tags -v
|
||||
|
||||
# Search MusicBrainz
|
||||
cargo run --bin shanty-search -- artist "Pink Floyd"
|
||||
|
||||
# Add to watchlist (expands album to individual tracks)
|
||||
cargo run --bin shanty-watch -- add album "Green Day" "Dookie"
|
||||
|
||||
# Sync watchlist to download queue and process
|
||||
cargo run --bin shanty-dl -- queue sync -v
|
||||
cargo run --bin shanty-dl -- queue process -v
|
||||
|
||||
# Organize files
|
||||
cargo run --bin shanty-org -- --from-db --target ~/Music -v
|
||||
```
|
||||
|
||||
### Full pipeline (CLI)
|
||||
|
||||
```sh
|
||||
shanty-watch add album "Green Day" "Dookie" --mbid <release-mbid>
|
||||
shanty-dl queue sync -v
|
||||
shanty-dl queue process -v
|
||||
shanty-tag --all --write-tags -v
|
||||
shanty-org --from-db --target ~/Music -v
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Create `~/.config/shanty/config.yaml`:
|
||||
**1.** Create a file called `docker-compose.yml`:
|
||||
|
||||
```yaml
|
||||
library_path: ~/Music
|
||||
download_path: ~/.local/share/shanty/downloads
|
||||
organization_format: "{artist}/{album}/{track_number} - {title}.{ext}"
|
||||
|
||||
# Filter which release types to show (empty = studio only)
|
||||
# Options: Compilation, Live, Soundtrack, Remix, DJ-mix, Demo
|
||||
allowed_secondary_types: []
|
||||
|
||||
web:
|
||||
port: 8085
|
||||
bind: 0.0.0.0
|
||||
|
||||
tagging:
|
||||
write_tags: true
|
||||
confidence: 0.85
|
||||
|
||||
download:
|
||||
format: opus
|
||||
search_source: ytmusic
|
||||
# cookies_path: ~/.config/shanty/cookies.txt # for higher YT rate limits
|
||||
services:
|
||||
shanty:
|
||||
image: git.rcjohnstone.com/connor/shanty:latest
|
||||
ports:
|
||||
- "8085:8085"
|
||||
# - "6080:6080" # Optional: expose if YouTube login iframe doesn't load
|
||||
volumes:
|
||||
- ./config:/config
|
||||
- ./data:/data
|
||||
- /path/to/your/music:/music
|
||||
environment:
|
||||
- SHANTY_WEB_BIND=0.0.0.0
|
||||
restart: unless-stopped
|
||||
```
|
||||
|
||||
## Docker
|
||||
**2.** Replace `/path/to/your/music` with the directory where you want your music library stored. The `./data` directory will be created automatically and stores the database, downloads, and any imported MusicBrainz data. If you plan to use the [local MusicBrainz database](docs/features/musicbrainz-db.md), put this on an SSD.
|
||||
|
||||
### Quick start
|
||||
**3.** Start the container:
|
||||
|
||||
```sh
|
||||
# Edit compose.yml to set your music library path, then:
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
**4.** Open `http://localhost:8085` in your browser. Create an account on first launch.
|
||||
|
||||
**5.** Search for an artist, click **Watch All**, then click **Set Sail** to start downloading.
|
||||
|
||||
## Quick Start from Source
|
||||
|
||||
For developers who want to build and run locally:
|
||||
|
||||
```sh
|
||||
# Prerequisites: Rust (edition 2024), yt-dlp, ffmpeg, Python 3, ytmusicapi, Trunk
|
||||
git clone https://git.rcjohnstone.com/connor/shanty.git
|
||||
cd shanty
|
||||
|
||||
# Build frontend
|
||||
cd shanty-web/frontend && trunk build --release && cd ../..
|
||||
|
||||
# Build and run
|
||||
cargo run --release --bin shanty -- -v
|
||||
# Open http://localhost:8085
|
||||
```
|
||||
|
||||
### Build from source
|
||||
## Documentation
|
||||
|
||||
```sh
|
||||
docker build -t shanty .
|
||||
docker run -d \
|
||||
-p 8085:8085 \
|
||||
-v ./config:/config \
|
||||
-v shanty-data:/data \
|
||||
-v /path/to/music:/music \
|
||||
shanty
|
||||
```
|
||||
|
||||
### Volumes
|
||||
|
||||
| Mount | Purpose |
|
||||
|-------|---------|
|
||||
| `/config` | Config file (`config.yaml`) |
|
||||
| `/data` | Database (`shanty.db`) and downloads |
|
||||
| `/music` | Your music library |
|
||||
|
||||
### Environment variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `SHANTY_CONFIG` | `/config/config.yaml` | Config file path |
|
||||
| `SHANTY_DATABASE_URL` | `sqlite:///data/shanty.db?mode=rwc` | Database URL |
|
||||
| `SHANTY_LIBRARY_PATH` | `/music` | Music library path |
|
||||
| `SHANTY_DOWNLOAD_PATH` | `/data/downloads` | Download directory |
|
||||
| `SHANTY_WEB_PORT` | `8085` | HTTP port |
|
||||
| `SHANTY_WEB_BIND` | `0.0.0.0` | Bind address |
|
||||
|
||||
## Testing
|
||||
|
||||
```sh
|
||||
cargo test --workspace
|
||||
```
|
||||
- [Getting Started](docs/getting-started.md) -- step-by-step setup guide
|
||||
- [Configuration Reference](docs/configuration.md) -- all config options and environment variables
|
||||
- [API Reference](docs/api.md) -- REST and Subsonic API documentation
|
||||
- **Feature Guides:**
|
||||
- [YouTube Authentication](docs/features/youtube-auth.md) -- higher rate limits with cookie auth
|
||||
- [MusicBrainz Local Database](docs/features/musicbrainz-db.md) -- instant metadata lookups
|
||||
- [Subsonic Streaming](docs/features/subsonic.md) -- mobile and desktop playback
|
||||
- [Playlists](docs/features/playlists.md) -- generation, editing, and export
|
||||
- [Monitoring](docs/features/monitoring.md) -- automatic new release detection
|
||||
|
||||
## License
|
||||
|
||||
|
||||
+29
-13
@@ -44,6 +44,10 @@ pub struct AppConfig {
|
||||
|
||||
#[serde(default)]
|
||||
pub musicbrainz: MusicBrainzConfig,
|
||||
|
||||
/// Log level: "error", "warn", "info", "debug", "trace". Default: "info".
|
||||
#[serde(default = "default_log_level")]
|
||||
pub log_level: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
@@ -65,6 +69,9 @@ pub struct TaggingConfig {
|
||||
|
||||
#[serde(default = "default_confidence")]
|
||||
pub confidence: f64,
|
||||
|
||||
#[serde(default = "default_tag_concurrency")]
|
||||
pub concurrency: usize,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
@@ -219,6 +226,7 @@ impl Default for AppConfig {
|
||||
scheduling: SchedulingConfig::default(),
|
||||
subsonic: SubsonicConfig::default(),
|
||||
musicbrainz: MusicBrainzConfig::default(),
|
||||
log_level: default_log_level(),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -238,10 +246,15 @@ impl Default for TaggingConfig {
|
||||
auto_tag: false,
|
||||
write_tags: true,
|
||||
confidence: default_confidence(),
|
||||
concurrency: default_tag_concurrency(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn default_tag_concurrency() -> usize {
|
||||
4
|
||||
}
|
||||
|
||||
impl Default for DownloadConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
@@ -279,23 +292,22 @@ impl Default for MetadataConfig {
|
||||
}
|
||||
}
|
||||
|
||||
fn default_log_level() -> String {
|
||||
"info".to_string()
|
||||
}
|
||||
|
||||
fn default_library_path() -> PathBuf {
|
||||
dirs::audio_dir().unwrap_or_else(|| PathBuf::from("~/Music"))
|
||||
}
|
||||
|
||||
fn default_database_url() -> String {
|
||||
let data_dir = dirs::data_dir()
|
||||
.unwrap_or_else(|| PathBuf::from("."))
|
||||
.join("shanty");
|
||||
std::fs::create_dir_all(&data_dir).ok();
|
||||
format!("sqlite://{}?mode=rwc", data_dir.join("shanty.db").display())
|
||||
let dd = data_dir();
|
||||
std::fs::create_dir_all(&dd).ok();
|
||||
format!("sqlite://{}?mode=rwc", dd.join("shanty.db").display())
|
||||
}
|
||||
|
||||
fn default_download_path() -> PathBuf {
|
||||
let dir = dirs::data_dir()
|
||||
.unwrap_or_else(|| PathBuf::from("."))
|
||||
.join("shanty")
|
||||
.join("downloads");
|
||||
let dir = data_dir().join("downloads");
|
||||
std::fs::create_dir_all(&dir).ok();
|
||||
dir
|
||||
}
|
||||
@@ -359,11 +371,18 @@ fn default_vnc_port() -> u16 {
|
||||
6080
|
||||
}
|
||||
|
||||
/// Return the application data directory (e.g. ~/.local/share/shanty).
|
||||
/// Return the application data directory.
|
||||
///
|
||||
/// Uses `SHANTY_DATA_DIR` env var if set, otherwise `~/.local/share/shanty`.
|
||||
/// In Docker, set `SHANTY_DATA_DIR=/data` so all data goes to the persistent volume.
|
||||
pub fn data_dir() -> PathBuf {
|
||||
if let Ok(v) = std::env::var("SHANTY_DATA_DIR") {
|
||||
PathBuf::from(v)
|
||||
} else {
|
||||
dirs::data_dir()
|
||||
.unwrap_or_else(|| PathBuf::from("."))
|
||||
.join("shanty")
|
||||
}
|
||||
}
|
||||
|
||||
// --- Loading and Saving ---
|
||||
@@ -430,9 +449,6 @@ impl AppConfig {
|
||||
if let Ok(v) = std::env::var("SHANTY_LIBRARY_PATH") {
|
||||
config.library_path = PathBuf::from(v);
|
||||
}
|
||||
if let Ok(v) = std::env::var("SHANTY_DOWNLOAD_PATH") {
|
||||
config.download_path = PathBuf::from(v);
|
||||
}
|
||||
if let Ok(v) = std::env::var("SHANTY_WEB_PORT")
|
||||
&& let Ok(port) = v.parse()
|
||||
{
|
||||
|
||||
@@ -59,6 +59,13 @@ impl HybridMusicBrainzFetcher {
|
||||
self.remote.get_artist_by_mbid(mbid).await
|
||||
}
|
||||
|
||||
/// Get artist info from local DB only (no remote API fallback).
|
||||
/// Returns `None` if the local DB is unavailable or doesn't have this artist.
|
||||
pub fn get_artist_info_local(&self, mbid: &str) -> Option<ArtistInfo> {
|
||||
self.local_if_available()
|
||||
.and_then(|l| l.get_artist_info_sync(mbid).ok())
|
||||
}
|
||||
|
||||
/// Get detailed artist info by MBID. Tries local first, then remote.
|
||||
pub async fn get_artist_info(&self, mbid: &str) -> DataResult<ArtistInfo> {
|
||||
if let Some(local) = self.local_if_available()
|
||||
|
||||
@@ -589,6 +589,19 @@ pub async fn download_dump(
|
||||
let url = format!("{DUMP_BASE_URL}{timestamp}/{filename}");
|
||||
let target_path = target_dir.join(filename);
|
||||
|
||||
// Skip if we already have this file from the same dump timestamp
|
||||
let stamp_path = target_dir.join(format!("{filename}.timestamp"));
|
||||
if target_path.exists()
|
||||
&& let Ok(existing_stamp) = std::fs::read_to_string(&stamp_path)
|
||||
&& existing_stamp.trim() == timestamp
|
||||
{
|
||||
progress(&format!(
|
||||
"Skipping {filename} (already downloaded from {timestamp})"
|
||||
));
|
||||
tracing::info!(file = %filename, timestamp = %timestamp, "dump file already up to date, skipping download");
|
||||
return Ok(target_path);
|
||||
}
|
||||
|
||||
progress(&format!("Downloading {filename}..."));
|
||||
tracing::info!(url = %url, target = %target_path.display(), "downloading MB dump");
|
||||
|
||||
@@ -634,6 +647,9 @@ pub async fn download_dump(
|
||||
"download complete"
|
||||
);
|
||||
|
||||
// Write timestamp marker so we can skip this file on re-runs
|
||||
let _ = std::fs::write(&stamp_path, timestamp);
|
||||
|
||||
Ok(target_path)
|
||||
}
|
||||
|
||||
@@ -699,6 +715,9 @@ pub fn run_import(
|
||||
PRAGMA foreign_keys = OFF;",
|
||||
)?;
|
||||
|
||||
// Wrap entire import in a transaction so interrupted imports don't leave empty tables
|
||||
conn.execute_batch("BEGIN;")?;
|
||||
|
||||
let mut stats = ImportStats::default();
|
||||
|
||||
// Import artists
|
||||
@@ -802,6 +821,9 @@ pub fn run_import(
|
||||
rusqlite::params![stats.recordings.to_string()],
|
||||
)?;
|
||||
|
||||
// Commit the transaction — if we got here, everything succeeded
|
||||
conn.execute_batch("COMMIT;")?;
|
||||
|
||||
progress(&format!("Import complete: {stats}"));
|
||||
Ok(stats)
|
||||
}
|
||||
|
||||
+66
-11
@@ -44,20 +44,21 @@ impl LocalMusicBrainzFetcher {
|
||||
/// Check whether the database has been populated with data.
|
||||
pub fn is_available(&self) -> bool {
|
||||
let conn = self.conn.lock().unwrap();
|
||||
// Check if the mb_artists table exists and has rows
|
||||
// Check if the mb_artists table exists and has at least one row.
|
||||
// Use EXISTS (SELECT 1 ...) instead of COUNT(*) to avoid scanning the entire table.
|
||||
conn.query_row(
|
||||
"SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name='mb_artists'",
|
||||
"SELECT EXISTS(SELECT 1 FROM sqlite_master WHERE type='table' AND name='mb_artists')",
|
||||
[],
|
||||
|row| row.get::<_, i32>(0),
|
||||
|row| row.get::<_, bool>(0),
|
||||
)
|
||||
.map(|c| c > 0)
|
||||
.unwrap_or(false)
|
||||
&& conn
|
||||
.query_row("SELECT COUNT(*) FROM mb_artists LIMIT 1", [], |row| {
|
||||
row.get::<_, i32>(0)
|
||||
})
|
||||
.unwrap_or(0)
|
||||
> 0
|
||||
.query_row(
|
||||
"SELECT EXISTS(SELECT 1 FROM mb_artists LIMIT 1)",
|
||||
[],
|
||||
|row| row.get::<_, bool>(0),
|
||||
)
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
/// Get statistics about the imported data.
|
||||
@@ -312,7 +313,6 @@ impl MetadataFetcher for LocalMusicBrainzFetcher {
|
||||
.unwrap_or_else(|| "Unknown Artist".into()),
|
||||
releases: vec![],
|
||||
genres: vec![],
|
||||
secondary_artists: vec![],
|
||||
})
|
||||
},
|
||||
);
|
||||
@@ -343,9 +343,63 @@ impl MetadataFetcher for LocalMusicBrainzFetcher {
|
||||
.collect();
|
||||
Ok(r)
|
||||
}
|
||||
Err(rusqlite::Error::QueryReturnedNoRows) => Err(DataError::Other(format!(
|
||||
Err(rusqlite::Error::QueryReturnedNoRows) => {
|
||||
// Not in standalone recordings — try finding via tracks-on-releases
|
||||
let track_recording = conn.query_row(
|
||||
"SELECT t.recording_mbid, t.title, rel.artist_mbid, a.name, t.duration_ms
|
||||
FROM mb_tracks t
|
||||
JOIN mb_releases rel ON t.release_mbid = rel.mbid
|
||||
LEFT JOIN mb_artists a ON rel.artist_mbid = a.mbid
|
||||
WHERE t.recording_mbid = ?1
|
||||
LIMIT 1",
|
||||
rusqlite::params![mbid],
|
||||
|row| {
|
||||
let duration: Option<i64> = row.get(4)?;
|
||||
Ok(RecordingDetails {
|
||||
mbid: row.get(0)?,
|
||||
title: row.get(1)?,
|
||||
artist_mbid: row.get(2)?,
|
||||
duration_ms: duration.map(|d| d as u64),
|
||||
artist: row
|
||||
.get::<_, Option<String>>(3)?
|
||||
.unwrap_or_else(|| "Unknown Artist".into()),
|
||||
releases: vec![],
|
||||
genres: vec![],
|
||||
})
|
||||
},
|
||||
);
|
||||
|
||||
match track_recording {
|
||||
Ok(mut r) => {
|
||||
// Fetch all releases containing this recording
|
||||
let mut stmt = conn
|
||||
.prepare(
|
||||
"SELECT DISTINCT rel.mbid, rel.title, rel.date
|
||||
FROM mb_tracks t
|
||||
JOIN mb_releases rel ON t.release_mbid = rel.mbid
|
||||
WHERE t.recording_mbid = ?1
|
||||
LIMIT 10",
|
||||
)
|
||||
.map_err(|e| DataError::Other(e.to_string()))?;
|
||||
r.releases = stmt
|
||||
.query_map(rusqlite::params![mbid], |row| {
|
||||
Ok(ReleaseRef {
|
||||
mbid: row.get(0)?,
|
||||
title: row.get(1)?,
|
||||
date: row.get(2)?,
|
||||
track_number: None,
|
||||
})
|
||||
})
|
||||
.map_err(|e| DataError::Other(e.to_string()))?
|
||||
.filter_map(|r| r.ok())
|
||||
.collect();
|
||||
Ok(r)
|
||||
}
|
||||
Err(_) => Err(DataError::Other(format!(
|
||||
"recording {mbid} not found locally"
|
||||
))),
|
||||
}
|
||||
}
|
||||
Err(e) => Err(DataError::Other(e.to_string())),
|
||||
}
|
||||
}
|
||||
@@ -475,6 +529,7 @@ impl MetadataFetcher for LocalMusicBrainzFetcher {
|
||||
secondary_types,
|
||||
first_release_date: row.get(4)?,
|
||||
first_release_mbid: row.get(5)?,
|
||||
featured: false, // Local DB only stores primary-credit releases
|
||||
})
|
||||
})
|
||||
.map_err(|e| DataError::Other(e.to_string()))?
|
||||
|
||||
@@ -251,7 +251,6 @@ impl MetadataFetcher for MusicBrainzFetcher {
|
||||
let r: MbRecordingDetail = self.get_json(&url).await?;
|
||||
|
||||
let (artist_name, artist_mbid) = extract_artist_credit(&r.artist_credit);
|
||||
let secondary_artists = extract_secondary_artists(&r.artist_credit);
|
||||
Ok(RecordingDetails {
|
||||
mbid: r.id,
|
||||
title: r.title,
|
||||
@@ -275,7 +274,6 @@ impl MetadataFetcher for MusicBrainzFetcher {
|
||||
.into_iter()
|
||||
.map(|g| g.name)
|
||||
.collect(),
|
||||
secondary_artists,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -347,7 +345,7 @@ impl MetadataFetcher for MusicBrainzFetcher {
|
||||
) -> DataResult<Vec<ReleaseGroupEntry>> {
|
||||
// Fetch album, single, and EP release groups
|
||||
let url = format!(
|
||||
"{BASE_URL}/release-group?artist={artist_mbid}&type=album|single|ep&fmt=json&limit=100"
|
||||
"{BASE_URL}/release-group?artist={artist_mbid}&type=album|single|ep&inc=artist-credits&fmt=json&limit=100"
|
||||
);
|
||||
let resp: MbReleaseGroupResponse = self.get_json(&url).await?;
|
||||
|
||||
@@ -355,7 +353,10 @@ impl MetadataFetcher for MusicBrainzFetcher {
|
||||
.release_groups
|
||||
.unwrap_or_default()
|
||||
.into_iter()
|
||||
.map(|rg| ReleaseGroupEntry {
|
||||
.map(|rg| {
|
||||
let primary = extract_artist_credit(&rg.artist_credit);
|
||||
let featured = primary.1.as_deref() != Some(artist_mbid);
|
||||
ReleaseGroupEntry {
|
||||
mbid: rg.id,
|
||||
title: rg.title,
|
||||
primary_type: rg.primary_type,
|
||||
@@ -364,6 +365,8 @@ impl MetadataFetcher for MusicBrainzFetcher {
|
||||
first_release_mbid: rg
|
||||
.releases
|
||||
.and_then(|r| r.into_iter().next().map(|rel| rel.id)),
|
||||
featured,
|
||||
}
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
@@ -388,32 +391,6 @@ fn extract_artist_credit(credits: &Option<Vec<MbArtistCredit>>) -> (String, Opti
|
||||
}
|
||||
}
|
||||
|
||||
/// Extract non-featuring secondary artists from MusicBrainz artist credits.
|
||||
/// Returns (name, mbid) pairs for collaborators that aren't "featuring" credits.
|
||||
fn extract_secondary_artists(credits: &Option<Vec<MbArtistCredit>>) -> Vec<(String, String)> {
|
||||
let Some(credits) = credits else {
|
||||
return vec![];
|
||||
};
|
||||
if credits.len() <= 1 {
|
||||
return vec![];
|
||||
}
|
||||
|
||||
// Walk credits after the first. Stop at any "feat"/"ft." joinphrase
|
||||
// from the PREVIOUS credit (since joinphrase is on the credit BEFORE the next artist).
|
||||
let mut result = Vec::new();
|
||||
for i in 0..credits.len() - 1 {
|
||||
let jp = credits[i].joinphrase.as_deref().unwrap_or("");
|
||||
let lower = jp.to_lowercase();
|
||||
if lower.contains("feat") || lower.contains("ft.") {
|
||||
break;
|
||||
}
|
||||
// The next credit is a non-featuring collaborator
|
||||
let next = &credits[i + 1];
|
||||
result.push((next.artist.name.clone(), next.artist.id.clone()));
|
||||
}
|
||||
result
|
||||
}
|
||||
|
||||
// --- MusicBrainz API response types ---
|
||||
|
||||
#[derive(Deserialize)]
|
||||
@@ -513,7 +490,6 @@ struct MbRecordingDetail {
|
||||
#[derive(Deserialize)]
|
||||
struct MbArtistCredit {
|
||||
artist: MbArtist,
|
||||
joinphrase: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
@@ -567,6 +543,8 @@ struct MbReleaseGroup {
|
||||
#[serde(rename = "first-release-date")]
|
||||
first_release_date: Option<String>,
|
||||
releases: Option<Vec<MbReleaseGroupRelease>>,
|
||||
#[serde(rename = "artist-credit")]
|
||||
artist_credit: Option<Vec<MbArtistCredit>>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
|
||||
@@ -43,9 +43,6 @@ pub struct RecordingDetails {
|
||||
pub releases: Vec<ReleaseRef>,
|
||||
pub duration_ms: Option<u64>,
|
||||
pub genres: Vec<String>,
|
||||
/// Non-featuring collaborators beyond the primary artist.
|
||||
#[serde(default)]
|
||||
pub secondary_artists: Vec<(String, String)>,
|
||||
}
|
||||
|
||||
/// Detailed artist info from a direct MBID lookup.
|
||||
@@ -99,6 +96,9 @@ pub struct ReleaseGroupEntry {
|
||||
pub first_release_date: Option<String>,
|
||||
/// MBID of the first release in this group (for fetching tracks).
|
||||
pub first_release_mbid: Option<String>,
|
||||
/// True if the queried artist is not the primary credit on this release group.
|
||||
#[serde(default)]
|
||||
pub featured: bool,
|
||||
}
|
||||
|
||||
/// A track within a release.
|
||||
|
||||
+1
-1
Submodule shanty-db updated: 1f983bbecf...181f736f25
+1
-1
Submodule shanty-index updated: 3494de1133...11a8d3a88e
+1
-1
Submodule shanty-org updated: a2152cbf8d...7ece462f58
@@ -9,5 +9,9 @@ pub mod selection;
|
||||
pub mod strategies;
|
||||
pub mod types;
|
||||
|
||||
pub use strategies::{PlaylistError, genre_based, random, similar_artists, smart, to_m3u};
|
||||
pub use types::{Candidate, PlaylistRequest, PlaylistResult, PlaylistTrack, SmartRules};
|
||||
pub use strategies::{
|
||||
CountryLookup, PlaylistError, genre_based, random, similar_artists, smart, to_m3u,
|
||||
};
|
||||
pub use types::{
|
||||
Candidate, PlaylistRequest, PlaylistResult, PlaylistTrack, SimilarConfig, SmartRules,
|
||||
};
|
||||
|
||||
+100
-13
@@ -21,6 +21,8 @@ pub fn score_tracks(
|
||||
tracks_by_artist: &HashMap<String, Vec<Track>>,
|
||||
top_tracks_by_artist: &HashMap<String, Vec<PopularTrack>>,
|
||||
popularity_bias: u8,
|
||||
_global_popularity: u8,
|
||||
max_tracks_per_artist: Option<u8>,
|
||||
) -> Vec<ScoredTrack> {
|
||||
let bias = popularity_bias.min(10) as usize;
|
||||
let mut scored = Vec::new();
|
||||
@@ -36,12 +38,17 @@ pub fn score_tracks(
|
||||
.cloned()
|
||||
.unwrap_or_default();
|
||||
|
||||
// Build playcount lookup by lowercase name
|
||||
// Build playcount lookups by lowercase name and by MBID
|
||||
let playcount_by_name: HashMap<String, u64> = top_tracks
|
||||
.iter()
|
||||
.map(|t| (t.name.to_lowercase(), t.playcount))
|
||||
.collect();
|
||||
|
||||
let playcount_by_mbid: HashMap<String, u64> = top_tracks
|
||||
.iter()
|
||||
.filter_map(|t| t.mbid.as_ref().map(|m| (m.clone(), t.playcount)))
|
||||
.collect();
|
||||
|
||||
let max_playcount = playcount_by_name
|
||||
.values()
|
||||
.copied()
|
||||
@@ -52,26 +59,54 @@ pub fn score_tracks(
|
||||
for track in local_tracks {
|
||||
let title_lower = track.title.as_ref().map(|t| t.to_lowercase());
|
||||
|
||||
let playcount = title_lower
|
||||
// Match by: exact title, MBID, and prefix — take the MAXIMUM playcount
|
||||
// across all methods so a popular base track isn't hidden by a less
|
||||
// popular variant that happens to match exactly.
|
||||
let mut best_playcount: Option<u64> = None;
|
||||
let mut consider = |pc: u64| {
|
||||
best_playcount = Some(best_playcount.map_or(pc, |cur: u64| cur.max(pc)));
|
||||
};
|
||||
|
||||
// Exact title match
|
||||
if let Some(pc) = title_lower
|
||||
.as_ref()
|
||||
.and_then(|t| playcount_by_name.get(t).copied())
|
||||
.or_else(|| {
|
||||
track
|
||||
{
|
||||
consider(pc);
|
||||
}
|
||||
|
||||
// MBID match
|
||||
if let Some(pc) = track
|
||||
.musicbrainz_id
|
||||
.as_ref()
|
||||
.and_then(|id| playcount_by_name.get(id).copied())
|
||||
});
|
||||
.and_then(|id| playcount_by_mbid.get(id).copied())
|
||||
{
|
||||
consider(pc);
|
||||
}
|
||||
|
||||
// If we have popularity data, require a match; otherwise assign uniform score
|
||||
// Prefix match: local title starts with a top track name, or vice versa
|
||||
if let Some(local) = title_lower.as_ref()
|
||||
&& let Some((_, &pc)) = playcount_by_name
|
||||
.iter()
|
||||
.filter(|(top_name, _)| {
|
||||
local.starts_with(top_name.as_str()) || top_name.starts_with(local.as_str())
|
||||
})
|
||||
.max_by_key(|&(_, &pc)| pc)
|
||||
{
|
||||
consider(pc);
|
||||
}
|
||||
|
||||
let playcount = best_playcount;
|
||||
|
||||
// If we have popularity data, use it; unmatched tracks get a low base score
|
||||
let (popularity, similarity, score) = if !playcount_by_name.is_empty() {
|
||||
let Some(playcount) = playcount else {
|
||||
continue;
|
||||
};
|
||||
let playcount = playcount.unwrap_or(0);
|
||||
|
||||
let popularity = if playcount > 0 {
|
||||
(playcount as f64 / max_playcount as f64).powf(POPULARITY_EXPONENTS[bias])
|
||||
} else {
|
||||
0.0
|
||||
// Unmatched track: small base score so it can still appear
|
||||
0.01
|
||||
};
|
||||
|
||||
let similarity = (match_score.exp()) / std::f64::consts::E;
|
||||
@@ -108,7 +143,9 @@ pub fn score_tracks(
|
||||
by_artist.entry(key).or_default().push(t);
|
||||
}
|
||||
|
||||
let cap = if popularity_bias == 0 {
|
||||
let cap = if let Some(explicit) = max_tracks_per_artist {
|
||||
Some((explicit as usize).max(1))
|
||||
} else if popularity_bias == 0 {
|
||||
None
|
||||
} else {
|
||||
let b = popularity_bias as f64;
|
||||
@@ -147,5 +184,55 @@ pub fn score_tracks(
|
||||
}
|
||||
}
|
||||
|
||||
by_artist.into_values().flatten().collect()
|
||||
let mut result: Vec<ScoredTrack> = by_artist.into_values().flatten().collect();
|
||||
|
||||
// Step 3: Apply global popularity weighting
|
||||
if _global_popularity > 0 {
|
||||
let gp = _global_popularity.min(10) as usize;
|
||||
let gp_exponent = POPULARITY_EXPONENTS[gp];
|
||||
let gp_strength = _global_popularity as f64 / 10.0;
|
||||
|
||||
// Find max playcount across ALL artists
|
||||
let global_max: u64 = top_tracks_by_artist
|
||||
.values()
|
||||
.flat_map(|tracks| tracks.iter().map(|t| t.playcount))
|
||||
.max()
|
||||
.unwrap_or(1)
|
||||
.max(1);
|
||||
|
||||
// Build a global playcount lookup (lowercase name -> max playcount)
|
||||
let mut global_playcounts: HashMap<String, u64> = HashMap::new();
|
||||
for tracks in top_tracks_by_artist.values() {
|
||||
for t in tracks {
|
||||
let key = t.name.to_lowercase();
|
||||
global_playcounts
|
||||
.entry(key)
|
||||
.and_modify(|c| *c = (*c).max(t.playcount))
|
||||
.or_insert(t.playcount);
|
||||
}
|
||||
}
|
||||
|
||||
// Apply to ALL tracks: popular ones get boosted, unknown ones get reduced.
|
||||
// Factor range: unknown tracks get `1 - gp_strength` (minimum 0.01),
|
||||
// top global track gets 1.0 + gp_strength (up to 2.0 at max setting).
|
||||
for t in &mut result {
|
||||
let playcount = t
|
||||
.title
|
||||
.as_ref()
|
||||
.and_then(|title| global_playcounts.get(&title.to_lowercase()).copied())
|
||||
.unwrap_or(0);
|
||||
|
||||
let global_pop = if playcount > 0 {
|
||||
(playcount as f64 / global_max as f64).powf(gp_exponent)
|
||||
} else {
|
||||
0.0
|
||||
};
|
||||
// Map global_pop [0, 1] to a factor centered around 1.0:
|
||||
// global_pop=0 → 1.0 - gp_strength, global_pop=1 → 1.0 + gp_strength
|
||||
let factor = (1.0 + gp_strength * (2.0 * global_pop - 1.0)).max(0.01);
|
||||
t.score *= factor;
|
||||
}
|
||||
}
|
||||
|
||||
result
|
||||
}
|
||||
|
||||
@@ -11,6 +11,8 @@ pub fn generate_playlist(
|
||||
candidates: &[Candidate],
|
||||
n: usize,
|
||||
seed_names: &HashSet<String>,
|
||||
max_artists: Option<u8>,
|
||||
skip_seed_enforcement: bool,
|
||||
) -> Vec<Candidate> {
|
||||
if candidates.is_empty() {
|
||||
return Vec::new();
|
||||
@@ -20,8 +22,14 @@ pub fn generate_playlist(
|
||||
let mut pool: Vec<&Candidate> = candidates.iter().collect();
|
||||
let mut result: Vec<Candidate> = Vec::new();
|
||||
let mut artist_counts: HashMap<String, usize> = HashMap::new();
|
||||
let mut distinct_artists_set: HashSet<String> = HashSet::new();
|
||||
let max_distinct = max_artists.map(|m| (m as usize).max(1));
|
||||
|
||||
let seed_min = (n / 10).max(1);
|
||||
let seed_min = if skip_seed_enforcement {
|
||||
0
|
||||
} else {
|
||||
(n / 10).max(1)
|
||||
};
|
||||
|
||||
let distinct_artists: usize = {
|
||||
let mut seen = HashSet::new();
|
||||
@@ -54,6 +62,13 @@ pub fn generate_playlist(
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter(|(_, c)| {
|
||||
// Max distinct artists: reject new artists once we hit the cap
|
||||
if let Some(max) = max_distinct
|
||||
&& distinct_artists_set.len() >= max
|
||||
&& !distinct_artists_set.contains(&c.artist)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
if force_seed {
|
||||
seed_names.contains(&c.artist)
|
||||
} else {
|
||||
@@ -79,6 +94,7 @@ pub fn generate_playlist(
|
||||
let picked = indices[dist.sample(&mut rng)];
|
||||
let track = pool.remove(picked);
|
||||
*artist_counts.entry(track.artist.clone()).or_insert(0) += 1;
|
||||
distinct_artists_set.insert(track.artist.clone());
|
||||
result.push(Candidate {
|
||||
score: track.score,
|
||||
artist: track.artist.clone(),
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::future::Future;
|
||||
use std::pin::Pin;
|
||||
|
||||
use sea_orm::DatabaseConnection;
|
||||
use shanty_data::{PopularTrack, SimilarArtist, SimilarArtistFetcher};
|
||||
@@ -12,6 +14,16 @@ use crate::types::*;
|
||||
/// Cache TTL: 7 days in seconds.
|
||||
const CACHE_TTL: i64 = 7 * 24 * 3600;
|
||||
|
||||
/// Trait for looking up an artist's country by MBID.
|
||||
/// Implementations should return quickly (local DB or cache), never blocking
|
||||
/// on rate-limited remote APIs during playlist generation.
|
||||
pub trait CountryLookup: Send + Sync {
|
||||
fn get_country<'a>(
|
||||
&'a self,
|
||||
mbid: &'a str,
|
||||
) -> Pin<Box<dyn Future<Output = Option<String>> + Send + 'a>>;
|
||||
}
|
||||
|
||||
/// Generate a playlist based on similar artists (the primary strategy).
|
||||
///
|
||||
/// Flow:
|
||||
@@ -26,9 +38,8 @@ pub async fn similar_artists(
|
||||
conn: &DatabaseConnection,
|
||||
fetcher: &impl SimilarArtistFetcher,
|
||||
seed_artists: Vec<String>,
|
||||
count: usize,
|
||||
popularity_bias: u8,
|
||||
ordering: &str,
|
||||
config: &SimilarConfig,
|
||||
_country_fetcher: Option<&dyn CountryLookup>,
|
||||
) -> Result<PlaylistResult, PlaylistError> {
|
||||
if seed_artists.is_empty() {
|
||||
return Err(PlaylistError::InvalidInput(
|
||||
@@ -37,25 +48,32 @@ pub async fn similar_artists(
|
||||
}
|
||||
|
||||
let num_seeds = seed_artists.len() as f64;
|
||||
let seed_similarity = config.seed_weight as f64 * 0.2;
|
||||
|
||||
// Merge similar artists from all seeds: key -> (name, total_score)
|
||||
let mut merged: HashMap<String, (String, f64)> = HashMap::new();
|
||||
// Track resolved seed names for enforcement (use DB names, not raw input)
|
||||
let mut resolved_seed_names: HashSet<String> = HashSet::new();
|
||||
// Track which keys are seeds (for country filter)
|
||||
let mut seed_keys: HashSet<String> = HashSet::new();
|
||||
|
||||
for seed in &seed_artists {
|
||||
// Resolve the seed artist: try name lookup in DB
|
||||
let (artist_name, artist_mbid) = resolve_artist(conn, seed).await?;
|
||||
resolved_seed_names.insert(artist_name.clone());
|
||||
|
||||
// Insert the seed itself with score 1.0
|
||||
let key = artist_mbid
|
||||
.clone()
|
||||
.unwrap_or_else(|| artist_name.to_lowercase());
|
||||
seed_keys.insert(key.clone());
|
||||
|
||||
// Insert the seed itself with configured weight
|
||||
if seed_similarity > 0.0 {
|
||||
let entry = merged
|
||||
.entry(key)
|
||||
.entry(key.clone())
|
||||
.or_insert_with(|| (artist_name.clone(), 0.0));
|
||||
entry.1 += 1.0;
|
||||
entry.1 += seed_similarity;
|
||||
}
|
||||
|
||||
// Fetch similar artists (cached or fresh)
|
||||
let similar = fetch_cached_similar(conn, fetcher, &artist_name, artist_mbid.as_deref())
|
||||
@@ -71,11 +89,41 @@ pub async fn similar_artists(
|
||||
}
|
||||
}
|
||||
|
||||
// Normalize scores by seed count
|
||||
let artists: Vec<(String, String, f64)> = merged
|
||||
// Normalize scores by seed count, sort by similarity descending
|
||||
let mut artists: Vec<(String, String, f64)> = merged
|
||||
.into_iter()
|
||||
.map(|(key, (name, total))| (key, name, total / num_seeds))
|
||||
.collect();
|
||||
artists.sort_by(|a, b| b.2.partial_cmp(&a.2).unwrap_or(std::cmp::Ordering::Equal));
|
||||
|
||||
// Apply discovery range: truncate to pool size
|
||||
let pool_size = discovery_pool_size(config.discovery_range);
|
||||
artists.truncate(pool_size);
|
||||
|
||||
// Country filter: only keep artists from the same countries as seeds
|
||||
if config.country_filter
|
||||
&& let Some(cf) = _country_fetcher
|
||||
{
|
||||
let mut seed_countries: HashSet<String> = HashSet::new();
|
||||
for key in &seed_keys {
|
||||
if let Some(country) = cf.get_country(key).await {
|
||||
seed_countries.insert(country);
|
||||
}
|
||||
}
|
||||
|
||||
if !seed_countries.is_empty() {
|
||||
let mut filtered = Vec::new();
|
||||
for entry in artists {
|
||||
let country = cf.get_country(&entry.0).await;
|
||||
match country {
|
||||
Some(c) if seed_countries.contains(&c) => filtered.push(entry),
|
||||
None => filtered.push(entry), // unknown = pass through
|
||||
_ => {} // known but different = exclude
|
||||
}
|
||||
}
|
||||
artists = filtered;
|
||||
}
|
||||
}
|
||||
|
||||
// Build track and top-track maps for scoring
|
||||
let mut tracks_by_artist: HashMap<String, Vec<shanty_db::entities::track::Model>> =
|
||||
@@ -104,7 +152,9 @@ pub async fn similar_artists(
|
||||
&artists,
|
||||
&tracks_by_artist,
|
||||
&top_tracks_by_artist,
|
||||
popularity_bias,
|
||||
config.popularity_bias,
|
||||
config.global_popularity,
|
||||
config.max_tracks_per_artist,
|
||||
);
|
||||
|
||||
// Convert to candidates
|
||||
@@ -123,10 +173,17 @@ pub async fn similar_artists(
|
||||
.collect();
|
||||
|
||||
// Select (use resolved DB names for seed enforcement, not raw input)
|
||||
let selected = selection::generate_playlist(&candidates, count, &resolved_seed_names);
|
||||
let skip_seed_enforcement = config.seed_weight == 0;
|
||||
let selected = selection::generate_playlist(
|
||||
&candidates,
|
||||
config.count,
|
||||
&resolved_seed_names,
|
||||
config.max_artists,
|
||||
skip_seed_enforcement,
|
||||
);
|
||||
|
||||
// Order
|
||||
let ordered = apply_ordering(selected, ordering);
|
||||
let ordered = apply_ordering(selected, &config.ordering);
|
||||
|
||||
Ok(PlaylistResult {
|
||||
tracks: candidates_to_tracks(ordered),
|
||||
@@ -135,6 +192,14 @@ pub async fn similar_artists(
|
||||
})
|
||||
}
|
||||
|
||||
/// Map discovery_range (0-10) to artist pool size.
|
||||
/// 0 -> 15, 5 -> ~100, 10 -> 500 (exponential curve).
|
||||
fn discovery_pool_size(range: u8) -> usize {
|
||||
let r = range.min(10) as f64;
|
||||
let size = 15.0 * (500.0_f64 / 15.0).powf(r / 10.0);
|
||||
size.round() as usize
|
||||
}
|
||||
|
||||
/// Generate a genre-based playlist.
|
||||
pub async fn genre_based(
|
||||
conn: &DatabaseConnection,
|
||||
@@ -176,7 +241,7 @@ pub async fn genre_based(
|
||||
.collect();
|
||||
|
||||
let seed_names = HashSet::new();
|
||||
let selected = selection::generate_playlist(&candidates, count, &seed_names);
|
||||
let selected = selection::generate_playlist(&candidates, count, &seed_names, None, true);
|
||||
let ordered = apply_ordering(selected, ordering);
|
||||
|
||||
Ok(PlaylistResult {
|
||||
@@ -306,7 +371,7 @@ pub async fn smart(
|
||||
.collect();
|
||||
|
||||
let seed_names = HashSet::new();
|
||||
let selected = selection::generate_playlist(&candidates, count, &seed_names);
|
||||
let selected = selection::generate_playlist(&candidates, count, &seed_names, None, true);
|
||||
let ordered = ordering::interleave_artists(selected);
|
||||
|
||||
Ok(PlaylistResult {
|
||||
|
||||
@@ -17,10 +17,62 @@ pub struct PlaylistRequest {
|
||||
pub ordering: String,
|
||||
#[serde(default)]
|
||||
pub rules: Option<SmartRules>,
|
||||
|
||||
/// Discovery range: how many similar artists to consider (0-10).
|
||||
/// 0 = focused (~15), 10 = wide open (~500). Default: 5.
|
||||
#[serde(default)]
|
||||
pub discovery_range: Option<u8>,
|
||||
/// Global popularity weighting (0-10). 0 = off, 10 = strong bias toward
|
||||
/// globally popular tracks across all artists. Default: 0.
|
||||
#[serde(default)]
|
||||
pub global_popularity: Option<u8>,
|
||||
/// Filter to same countries as seed artists. Default: false.
|
||||
#[serde(default)]
|
||||
pub country_filter: Option<bool>,
|
||||
/// Seed artist weight (0-10). 0 = exclude seeds, 5 = normal (similarity 1.0),
|
||||
/// 10 = double weight (similarity 2.0). Default: 5.
|
||||
#[serde(default)]
|
||||
pub seed_weight: Option<u8>,
|
||||
/// Explicit per-artist track cap. None or 0 = auto (derived from popularity_bias).
|
||||
#[serde(default)]
|
||||
pub max_tracks_per_artist: Option<u8>,
|
||||
/// Maximum distinct artists in the result. None or 0 = unlimited.
|
||||
#[serde(default)]
|
||||
pub max_artists: Option<u8>,
|
||||
}
|
||||
|
||||
/// Resolved configuration for the similar-artists strategy.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SimilarConfig {
|
||||
pub count: usize,
|
||||
pub popularity_bias: u8,
|
||||
pub ordering: String,
|
||||
pub discovery_range: u8,
|
||||
pub global_popularity: u8,
|
||||
pub country_filter: bool,
|
||||
pub seed_weight: u8,
|
||||
pub max_tracks_per_artist: Option<u8>,
|
||||
pub max_artists: Option<u8>,
|
||||
}
|
||||
|
||||
impl SimilarConfig {
|
||||
pub fn from_request(req: &PlaylistRequest) -> Self {
|
||||
Self {
|
||||
count: req.count,
|
||||
popularity_bias: req.popularity_bias,
|
||||
ordering: req.ordering.clone(),
|
||||
discovery_range: req.discovery_range.unwrap_or(5),
|
||||
global_popularity: req.global_popularity.unwrap_or(0),
|
||||
country_filter: req.country_filter.unwrap_or(false),
|
||||
seed_weight: req.seed_weight.unwrap_or(5),
|
||||
max_tracks_per_artist: req.max_tracks_per_artist.filter(|&v| v > 0),
|
||||
max_artists: req.max_artists.filter(|&v| v > 0),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn default_count() -> usize {
|
||||
50
|
||||
30
|
||||
}
|
||||
|
||||
fn default_popularity_bias() -> u8 {
|
||||
|
||||
@@ -32,6 +32,7 @@ fn make_track(id: i32, title: &str, artist_id: Option<i32>, mbid: Option<&str>)
|
||||
added_at: now,
|
||||
updated_at: now,
|
||||
file_mtime: None,
|
||||
tagged: false,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -82,15 +83,36 @@ fn test_score_tracks_basic() {
|
||||
let mut top_map = HashMap::new();
|
||||
top_map.insert("artist-1".to_string(), top_tracks);
|
||||
|
||||
let scored = score_tracks(&artists, &tracks_map, &top_map, 5);
|
||||
let scored = score_tracks(&artists, &tracks_map, &top_map, 5, 0, None);
|
||||
|
||||
// Should have 3 tracks (only ones matching top tracks)
|
||||
assert_eq!(scored.len(), 3);
|
||||
// All 5 tracks should be included (unmatched get a small base score)
|
||||
assert_eq!(scored.len(), 5);
|
||||
|
||||
// Higher playcount should yield higher score
|
||||
let scores: Vec<f64> = scored.iter().map(|t| t.score).collect();
|
||||
let max_score = scores.iter().cloned().fold(f64::NEG_INFINITY, f64::max);
|
||||
assert!(max_score > 0.0);
|
||||
// Matched tracks should score higher than unmatched ones
|
||||
let mut matched: Vec<f64> = scored
|
||||
.iter()
|
||||
.filter(|t| {
|
||||
t.title.as_deref().is_some_and(|n| {
|
||||
n.starts_with("Song 1") || n.starts_with("Song 2") || n.starts_with("Song 3")
|
||||
})
|
||||
})
|
||||
.map(|t| t.score)
|
||||
.collect();
|
||||
let mut unmatched: Vec<f64> = scored
|
||||
.iter()
|
||||
.filter(|t| {
|
||||
t.title
|
||||
.as_deref()
|
||||
.is_some_and(|n| n.starts_with("Song 4") || n.starts_with("Song 5"))
|
||||
})
|
||||
.map(|t| t.score)
|
||||
.collect();
|
||||
matched.sort_by(|a, b| b.partial_cmp(a).unwrap());
|
||||
unmatched.sort_by(|a, b| b.partial_cmp(a).unwrap());
|
||||
assert!(
|
||||
matched[0] > unmatched[0],
|
||||
"matched tracks should score higher than unmatched"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -106,7 +128,7 @@ fn test_score_tracks_no_top_tracks_uses_uniform() {
|
||||
|
||||
let top_map = HashMap::new(); // No top tracks
|
||||
|
||||
let scored = score_tracks(&artists, &tracks_map, &top_map, 5);
|
||||
let scored = score_tracks(&artists, &tracks_map, &top_map, 5, 0, None);
|
||||
|
||||
// All 3 tracks should be included with uniform scoring
|
||||
assert_eq!(scored.len(), 3);
|
||||
@@ -140,11 +162,11 @@ fn test_score_tracks_per_artist_cap() {
|
||||
top_map.insert("artist-1".to_string(), top_tracks);
|
||||
|
||||
// bias 10 → cap = 10
|
||||
let scored = score_tracks(&artists, &tracks_map, &top_map, 10);
|
||||
let scored = score_tracks(&artists, &tracks_map, &top_map, 10, 0, None);
|
||||
assert!(scored.len() <= 10);
|
||||
|
||||
// bias 0 → no cap
|
||||
let scored_no_cap = score_tracks(&artists, &tracks_map, &top_map, 0);
|
||||
let scored_no_cap = score_tracks(&artists, &tracks_map, &top_map, 0, 0, None);
|
||||
assert_eq!(scored_no_cap.len(), 50);
|
||||
}
|
||||
|
||||
@@ -163,7 +185,7 @@ fn test_similarity_transform() {
|
||||
tracks_map.insert("high".to_string(), vec![track_high]);
|
||||
tracks_map.insert("low".to_string(), vec![track_low]);
|
||||
|
||||
let scored = score_tracks(&artists, &tracks_map, &HashMap::new(), 5);
|
||||
let scored = score_tracks(&artists, &tracks_map, &HashMap::new(), 5, 0, None);
|
||||
assert_eq!(scored.len(), 2);
|
||||
|
||||
let high_score = scored
|
||||
@@ -188,7 +210,7 @@ fn test_generate_playlist_basic() {
|
||||
.collect();
|
||||
|
||||
let seeds = HashSet::new();
|
||||
let result = generate_playlist(&candidates, 10, &seeds);
|
||||
let result = generate_playlist(&candidates, 10, &seeds, None, false);
|
||||
|
||||
assert_eq!(result.len(), 10);
|
||||
}
|
||||
@@ -198,7 +220,7 @@ fn test_generate_playlist_respects_count() {
|
||||
let candidates: Vec<Candidate> = (1..=5).map(|i| make_candidate(i, "Artist", 1.0)).collect();
|
||||
|
||||
let seeds = HashSet::new();
|
||||
let result = generate_playlist(&candidates, 3, &seeds);
|
||||
let result = generate_playlist(&candidates, 3, &seeds, None, false);
|
||||
assert_eq!(result.len(), 3);
|
||||
}
|
||||
|
||||
@@ -207,7 +229,7 @@ fn test_generate_playlist_not_more_than_available() {
|
||||
let candidates: Vec<Candidate> = (1..=3).map(|i| make_candidate(i, "Artist", 1.0)).collect();
|
||||
|
||||
let seeds = HashSet::new();
|
||||
let result = generate_playlist(&candidates, 100, &seeds);
|
||||
let result = generate_playlist(&candidates, 100, &seeds, None, false);
|
||||
assert_eq!(result.len(), 3);
|
||||
}
|
||||
|
||||
@@ -215,7 +237,7 @@ fn test_generate_playlist_not_more_than_available() {
|
||||
fn test_generate_playlist_empty_candidates() {
|
||||
let candidates: Vec<Candidate> = vec![];
|
||||
let seeds = HashSet::new();
|
||||
let result = generate_playlist(&candidates, 10, &seeds);
|
||||
let result = generate_playlist(&candidates, 10, &seeds, None, false);
|
||||
assert!(result.is_empty());
|
||||
}
|
||||
|
||||
@@ -228,7 +250,7 @@ fn test_generate_playlist_per_artist_cap() {
|
||||
candidates.extend((21..=25).map(|i| make_candidate(i, "Minor", 1.0)));
|
||||
|
||||
let seeds = HashSet::new();
|
||||
let result = generate_playlist(&candidates, 15, &seeds);
|
||||
let result = generate_playlist(&candidates, 15, &seeds, None, false);
|
||||
|
||||
let prolific_count = result.iter().filter(|c| c.artist == "Prolific").count();
|
||||
let minor_count = result.iter().filter(|c| c.artist == "Minor").count();
|
||||
@@ -258,7 +280,7 @@ fn test_generate_playlist_seed_enforcement() {
|
||||
let mut seeds = HashSet::new();
|
||||
seeds.insert("Seed".to_string());
|
||||
|
||||
let result = generate_playlist(&candidates, 10, &seeds);
|
||||
let result = generate_playlist(&candidates, 10, &seeds, None, false);
|
||||
let seed_count = result.iter().filter(|c| c.artist == "Seed").count();
|
||||
|
||||
// seed_min = (10/10).max(1) = 1, so at least 1 seed track
|
||||
|
||||
+1
-1
Submodule shanty-search updated: b39dd6cc8e...cc0978db94
+1
-1
Submodule shanty-tag updated: 884b2e8d52...0f5d3f597a
+1
-1
Submodule shanty-watch updated: aef4708439...3593698854
+1
-1
Submodule shanty-web updated: 3dba620c9b...d17049d92a
+23
-18
@@ -14,6 +14,7 @@ use shanty_search::MusicBrainzSearch;
|
||||
use shanty_web::routes;
|
||||
use shanty_web::state::AppState;
|
||||
use shanty_web::tasks::TaskManager;
|
||||
use shanty_web::workers::WorkerManager;
|
||||
|
||||
#[derive(Parser)]
|
||||
#[command(name = "shanty", about = "Shanty — self-hosted music management")]
|
||||
@@ -56,18 +57,30 @@ async fn main() -> anyhow::Result<()> {
|
||||
|
||||
let cli = Cli::parse();
|
||||
|
||||
// Load config early so we can use log_level from it
|
||||
let mut config = AppConfig::load(cli.config.as_deref());
|
||||
|
||||
// CLI -v flags override config log_level
|
||||
let filter = match cli.verbose {
|
||||
0 => "info,shanty=info,shanty_web=info",
|
||||
1 => "info,shanty=debug,shanty_web=debug",
|
||||
_ => "debug,shanty=trace,shanty_web=trace",
|
||||
0 => {
|
||||
// Use config log_level
|
||||
let level = config.log_level.to_lowercase();
|
||||
match level.as_str() {
|
||||
"error" => "error".to_string(),
|
||||
"warn" => "warn".to_string(),
|
||||
"debug" => "debug,shanty=debug,shanty_web=debug".to_string(),
|
||||
"trace" => "trace,shanty=trace,shanty_web=trace".to_string(),
|
||||
_ => "info,shanty=info,shanty_web=info".to_string(),
|
||||
}
|
||||
}
|
||||
1 => "info,shanty=debug,shanty_web=debug".to_string(),
|
||||
_ => "debug,shanty=trace,shanty_web=trace".to_string(),
|
||||
};
|
||||
tracing_subscriber::fmt()
|
||||
.with_env_filter(
|
||||
EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new(filter)),
|
||||
EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new(&filter)),
|
||||
)
|
||||
.init();
|
||||
|
||||
let mut config = AppConfig::load(cli.config.as_deref());
|
||||
if let Some(port) = cli.port {
|
||||
config.web.port = port;
|
||||
}
|
||||
@@ -116,21 +129,13 @@ async fn main() -> anyhow::Result<()> {
|
||||
config: std::sync::Arc::new(tokio::sync::RwLock::new(config)),
|
||||
config_path,
|
||||
tasks: TaskManager::new(),
|
||||
workers: WorkerManager::new(),
|
||||
firefox_login: tokio::sync::Mutex::new(None),
|
||||
scheduler: tokio::sync::Mutex::new(shanty_web::state::SchedulerInfo {
|
||||
next_pipeline: None,
|
||||
next_monitor: None,
|
||||
skip_pipeline: false,
|
||||
skip_monitor: false,
|
||||
}),
|
||||
});
|
||||
|
||||
// Start background cookie refresh task
|
||||
shanty_web::cookie_refresh::spawn(state.config.clone());
|
||||
|
||||
// Start pipeline and monitor schedulers
|
||||
shanty_web::pipeline_scheduler::spawn(state.clone());
|
||||
shanty_web::monitor::spawn(state.clone());
|
||||
// Start work queue workers and unified scheduler
|
||||
WorkerManager::spawn_all(state.clone());
|
||||
shanty_web::scheduler::spawn(state.clone());
|
||||
shanty_web::mb_update::spawn(state.clone());
|
||||
|
||||
// Resolve static files directory
|
||||
|
||||
Reference in New Issue
Block a user