mirror of
https://github.com/Psychotoxical/psysonic.git
synced 2026-07-22 15:25:46 +00:00
feat(library): local library index and search (preview) (#846)
* feat(library): scaffold psysonic-library crate with v1 schema and store (#791) Adds a new workspace crate that will host the unified track store and the upcoming sync engine. PR-1a covers spec phases A1–A6: - migrations/001_initial.sql: full v1 schema — sync_state, track, album, artist, track_fts (+ ai/ad/au triggers), track_extension, track_offline, track_id_history, track_fact, track_artifact, canonical_track, canonical_identity, track_canonical_link, canonical_enrichment_link, and all §5.2 partial indexes. - store::LibraryStore: WAL + foreign_keys=ON SQLite connection rooted at app_data_dir/library.sqlite (distinct from the analysis cache, which uses app_config_dir). schema_migrations table + idempotent embedded migration runner; LIBRARY_DB_SCHEMA_VERSION = 1. - repos::TrackRepository::upsert_batch: 35-column transactional upsert with ON CONFLICT(server_id, id) all-fields rewrite; FTS rows follow via the triggers. - search::search_tracks: minimal bm25-ordered FTS5 helper scoped to a single server_id, filtering deleted rows. - filter::FilterFieldRegistry: static v1 registry (text, genre, year, starred = V1; bpm = SchemaV1UiLater; user_rating/suffix/bit_rate = Planned). Entity routing is a silent skip per §5.13.3. No Tauri commands, no frontend, no sync — those land in PR-2..PR-7. PR-1b will follow with the migration-runner edge-case tests, the initial_sync_cursor_json read/write API, and the breaking-migration hook stub (P22). * feat(library): A7 migration-runner safety net + initial-sync cursor API (PR-1b) (#792) * feat(library): wire migration-runner safety net and initial-sync cursor API PR-1b — Phase A7 infrastructure on top of PR-1a. Production behaviour is unchanged at v1 launch; everything here is plumbing that PR-3 will consume. - store::run_migrations_with: testable entry point that takes an explicit migration slice, a min-compatible-version threshold, and a breaking-bump hook. The prod `run_migrations` fixes those to MIGRATIONS, LIBRARY_DB_MIN_COMPATIBLE_VERSION, and the no-op stub. The slice is now sorted defensively before applying. - store::LIBRARY_DB_MIN_COMPATIBLE_VERSION: new public constant (currently equal to LIBRARY_DB_SCHEMA_VERSION). When a future release needs to invalidate v1 data, bumping this above the max applied version trips the hook on next open per spec §5.7 / P22. - store::MigrationOutcome (Applied | BreakingBump): crate-internal signal callers can branch on. PR-1b consumers ignore it; PR-3 / Settings will surface the "library rebuilt after update" toast when it surfaces. - store::handle_breaking_schema_bump: documented no-op stub. The drop + resync logic lands with the first real breaking bump. - repos::SyncStateRepository: ensure(server_id, scope) idempotently inserts a default row; get_initial_sync_cursor / set_initial_sync_cursor read and write sync_state.initial_sync_cursor_json via serde_json::Value. The set uses ON CONFLICT … DO UPDATE scoped to the cursor column only, so phase / poll-stats / tier survive cursor writes intact. - Tests cover: additive 002-style migration preserves prior data (spec §5.7 explicit integration test), runner sorts an unsorted source slice, breaking-bump hook fires when max applied < min_compatible, hook does not fire on a fresh DB, cursor round-trips a nested serde_json::Value, ON CONFLICT preserves sibling columns, library_scope separates rows per server. End-to-end "kill mid-500k-sync → resume same cursor" stays out of scope per the kickoff answer — it belongs to PR-3 / C2 where the InitialSyncRunner lives. * test(library): cover AC A3 — 500-row upsert_batch under perf budget * feat(library): Subsonic REST client for the sync engine (Phase B, PR-2) (#793) Phase B (B1-B9 per spec §10) — pure-Rust Subsonic client that the library-sync engine (PR-3) will drive. No Tauri commands, no events; the surface is added internally to psysonic-integration as a sibling of the existing navidrome native-REST module. - B1 — SubsonicClient + ping over /rest/{method}.view. Auth via the legacy salted-md5 token (spec v1.13+, advertised as 1.16.1). New SubsonicCredentials helper computes token = md5(password || salt) and ships a per-process unique salt nonce so back-to-back calls don't repeat. - B2 — get_scan_status → ScanStatus { scanning, count, folder_count, last_scan }. Lightweight poll for the Huge-tier path (§6.2.2). - B3 — get_album_list2(type, size, offset, musicFolderId?) + get_album(id). The two-call pattern the sync engine walks during initial ingest (§6.3). - B4 — search3(query, songCount, songOffset, musicFolderId?). Empty query → all songs paged (Navidrome quirk, spec §2.4). - B5 — get_indexes(musicFolderId?, ifModifiedSince?). Conditional fetch for file-tree fallback (S3 / §3.1). - B6 — get_song(id). Error code 70 maps to the dedicated SubsonicError::NotFound variant so the tombstone reconciler can match on the variant instead of parsing strings. - B8 — get_artists(musicFolderId?). ID3-path artist index; clients compare ArtistIndex.last_modified_ms against the local watermark to decide if a delta pass is needed (§2.2.1). - B9 — fingerprint_sample helper picks every-Nth track id for the server-fingerprint verify pass. Sampling is deterministic so reruns probe the same tracks. The verify-and-compare glue itself is library-side (PR-3 territory, deps on the store). Tests cover envelope parsing (status=ok/failed, code 70 → NotFound, missing body key), credentials (md5 vectors, salt uniqueness across 1k rapid calls, salt differs per from_password call), each endpoint end-to-end through wiremock with query-param matchers, OpenSubsonic forward-compat (unknown fields ignored on Song), and the trailing-slash base-URL normalisation. Cargo.toml — adds query + form + multipart to psysonic-integration's reqwest feature set. PR-2's client needs `query`; the other two were already used by existing navidrome::covers / remote::lastfm code and only worked via top-crate feature unification. Aligning the crate's own deps means `cargo test -p psysonic-integration` now compiles without depending on the workspace build. Out of scope: capability detection (C1 / PR-3), Navidrome native bulk path (uses existing psysonic-integration::navidrome::queries), fixtures harness expansion (G1). * feat(library): subsonic client follow-ups from PR-2 review (PR-2b) (#794) Picks up the three non-blocking items from cucadmuh's PR-793 review (handoffs/2026-05-19-pr-793-review.md) before PR-3 starts on top. - Fresh `(token, salt)` per request. `SubsonicClient` now caches the plaintext username + password and derives a new `SubsonicCredentials` inside `send()` for every endpoint call — matches the frontend's `subsonicClient.ts` `getAuthParams()` lifecycle and follows Subsonic replay-resistance guidance. Test path keeps a `with_static_credentials` constructor so wiremock matchers stay deterministic. New `build_credentials` (`pub(crate)`) routes the two modes. - `SUBSONIC_CLIENT_ID` now carries the crate version (`psysonic/<CARGO_PKG_VERSION>`) — aligns with the frontend's `psysonic/${version}` so Navidrome log lines correlate across the WebView and Rust sync paths. - `Song.mbid_recording` gains the `musicBrainzId` serde alias (plus the schema-column spelling) so the OpenSubsonic field lands on the same hot column the §5.1 schema names. P13 strong-key matching can now key off it on ingest. - `get_song_with_raw` / `get_album_with_raw` return both the typed projection and the raw `serde_json::Value` body sub-tree. PR-3 ingest will write that raw value verbatim into `track.raw_json`, so OpenSubsonic extensions (`contributors`, `replayGain`, future fields) survive without manual field mirroring. Internal `parse_envelope_body` extracts the validation + body-key lookup once; `parse_envelope` and the new `parse_envelope_with_raw` share it. Tests cover: `from_password` produces unique salt/token across two back-to-back calls (direct + over-the-wire via wiremock `received_requests`), static mode returns the same triple, `c` query param starts with `psysonic/` and equals `SUBSONIC_CLIENT_ID`, `get_song_with_raw` preserves untyped fields (`replayGain`, `contributors`) in the raw value, `get_album_with_raw` keeps per-track extensions in `raw.song[i]`, error 70 still maps to `NotFound` on the raw variant, and `Song` deserializes `musicBrainzId` and `mbid_recording` interchangeably. B9 fingerprint-verify glue and the wider raw-ingest call sites stay with PR-3 / C2 as the review's §5 / §7 checklist directs. * feat(library): capability probe + sync_state accessors (Phase C1+C7, PR-3a) (#795) First sub-PR of Phase C (sync orchestrator). Lands the foundation that PR-3b's InitialSyncRunner consumes — pure plumbing, no runners or background tasks yet. - C1 capability probe. `psysonic_library::sync::CapabilityProbe::run` drives the §6.1 probe chain: Subsonic ping (captures `ServerInfo` envelope metadata for server-type / OpenSubsonic detection), then best-effort probes for search3 / getScanStatus / getIndexes, plus an optional Navidrome native bulk probe (caller passes `NavidromeProbeCredentials`). `CapabilityFlags(u32)` matches the §6.1.1 bitfield: NavidromeNativeBulk / SubsonicSearch3Bulk / ScanStatusAvailable / OpenSubsonic / UnstableTrackIds / FileTreeBrowse. - C7 sync_state accessors. `SyncStateRepository` gains get/set capability_flags, get/set sync_phase (idle / probing / initial_sync / ready / error), and column-scoped setters for server_last_scan_iso, indexes_last_modified_ms, artists_last_modified_ms, library_tier. Every setter uses `ON CONFLICT … DO UPDATE` scoped to its own column so concurrent watermark writes don't clobber each other. - Supporting additions in `psysonic-integration`: - `subsonic::SubsonicClient::server_info()` extracts `ServerInfo` from the ping envelope (server_type, server_version, api_version, open_subsonic). Re-uses `send()` so auth lifecycle is the same. - `navidrome::probe::native_bulk_available(url, token)` does the `GET /api/song?_start=0&_end=1` Bearer-auth probe. Returns Ok(true) on 2xx, Ok(false) on 4xx (auth ok but endpoint missing), Err on 5xx. Probe-only — full nd_list_songs port is PR-3b. - `psysonic-library/Cargo.toml` gains a `psysonic-integration` dependency (sync calls into Subsonic + Navidrome probes). DAG stays acyclic: integration does not depend on library. Per cucadmuh's PR-3 kickoff answer (handoff `2026-05-19-pr3-kickoff.md`): - Crate placement: option A — sync lives in `psysonic-library/src/sync/`, no new psysonic-sync crate. - N1 gate: probe is `/api/song?_start=0&_end=1` only; `nd_list_artists_by_role` is NOT required (Q3 answer + N1 ingest port lands in PR-3b). - UnstableTrackIds: set for Navidrome via `ServerInfo.server_type`, cleared for generic Subsonic. Tests added: 23 across library/sync, library/repos/sync_state, integration/subsonic, integration/navidrome/probe. Cover bitfield contains/insert/remove + spec bit values, probe across mixed-capability servers (full Navidrome, minimal Subsonic, broken endpoints), ping-failure short-circuit, optional Navidrome creds gating N1, sync_state column-scoped upserts (capability_flags / sync_phase / watermarks / library_tier), cross-column independence (capability writes don't reset cursor), ServerInfo extraction from ping envelope, Navidrome bulk probe across 2xx/4xx/5xx. * feat(library): InitialSyncRunner + C12 backoff + C13 id remap (Phase C2/C12/C13, PR-3b) (#796) Second sub-PR of Phase C — wires the actual ingest path on top of PR-3a's capability + sync_state foundation. Runner is pure async Rust: PR-3d will spawn it inside a tokio task and emit Tauri progress events on top. - C2 InitialSyncRunner. Drives spec §6.3 IS-1 → IS-6: probe-derived IngestStrategy (enum N1/S1/S2/S3, selector picks N1 → S1 → S2 chain per kickoff Q3), per-page upsert loop, cursor flush after every successful batch, IS-4 best-effort getArtists watermark, IS-5 getScanStatus.lastScan capture, IS-6 phase=ready + cursor cleared. Resume is automatic: a non-empty initial_sync_cursor_json restarts at the persisted offset; a strategy mismatch between cursor and capability flags surfaces as SyncError::CursorIncompatible. - C12 backoff. sync::backoff::Backoff implements the §6.8 schedule (2s → 4s → … cap 120s) with ±25% jitter via deterministic salt. retry_with_backoff wraps every endpoint call: transport / Navidrome failures retry up to MAX_ATTEMPTS_PER_BATCH (5), the cursor never advances on failure, success resets the counter. Cancellation AtomicBool is checked between attempts. - C13 id remap. TrackRepository::upsert_batch_with_remap performs the §6.9 detect-and-rebind pass inside the same SQLite transaction as the upsert: a content_hash or server_path collision on a different existing id triggers UPDATE of child tables (track_offline, track_extension, track_fact, track_artifact, track_canonical_link), INSERT INTO track_id_history, DELETE old track row. Off when UnstableTrackIds is clear (generic Subsonic). New TrackIdHistoryRepository read-side helper for forward lookups (analysis cache reuse, Phase E). - IngestStrategy enum + selector (sync::strategy) — N1 → S1 → S2; N1 requires Navidrome bearer credentials at runtime (skipped when None). S3 is enumerated for future file-tree fallback but returns StrategyUnsupported in v1 per kickoff Q3. - InitialSyncCursor (sync::cursor) — JSON-serialisable { strategy, phase, library_scope, ingested_count, strategy_state }. StrategyState tagged enum: LinearOffset { offset } for N1/S1, AlbumCrawl { album_offset, current_album_id } for S2. - mapping::subsonic_song_to_track_row + navidrome_song_to_track_row centralise the JSON → TrackRow projection. Subsonic path also reads replayGain.{trackGain,albumGain} from the raw value so PR-3b doesn't drop the columns that PR-2b reserves on TrackRow. - Supporting bits in psysonic-integration: - subsonic types now derive Serialize so the runner can round-trip a typed Song back into raw JSON when feeding upsert. - navidrome::queries gains nd_list_songs_internal — pure async function (no #[tauri::command] decorator) that the N1 ingest loop calls directly. The existing Tauri command wraps it. Tests added across sync::* and repos::track_id_history. Wiremock covers S1 happy-path, mid-cursor resume from a persisted offset, strategy mismatch → CursorIncompatible, 503 transient → retry-then- succeed, AtomicBool cancellation → Cancelled, N1 paginated /api/song ingest, S2 album crawl, and §6.9 remap firing under UnstableTrackIds during an actual sync. Backoff schedule + jitter formula pinned. TrackRepository remap path covered by content_hash collision, server_path collision, hash+path-missing skip, identity-noop, and remap-off compatibility with the existing upsert_batch contract. Also fixes cucadmuh's PR-3a review minor 1: drops the dead `mount_ok` scaffolding from sync::capability tests. Out of scope per kickoff Q2: - DeltaSyncRunner + tombstones → PR-3c - Background task lifecycle, cancellation wiring, progress emit throttle, adaptive scheduler, request budget, bandwidth lane → PR-3d - Tauri command surface for "sync now" / progress events → PR-5 * feat(library): search3 raw envelope fidelity for S1 ingest (PR-3b follow-up) (#797) Picks up cucadmuh's PR-3b review minor 1: the S1 path in InitialSyncRunner was reserialising the typed `Song` for `track.raw_json`, dropping unknown OpenSubsonic extensions (`replayGain`, `contributors`, …). N1 and S2 already carry the raw sub-tree verbatim through `nd_list_songs_internal` and `get_album_with_raw`; S1 now matches via the new `SubsonicClient::search3_with_raw` mirror of the PR-2b pattern. - subsonic::SubsonicClient::search3_with_raw — returns `(SearchResult, serde_json::Value)`; uses the existing `parse_envelope_with_raw` so error 70 / `Api { code, .. }` mapping stays consistent. - sync::initial::run_s1 now calls `search3_with_raw` and feeds the per-song raw sub-tree (`raw_body.song[i]`) into `subsonic_song_to_track_row` instead of a typed reserialise. Tests cover `search3_with_raw` round-trip on a payload with `replayGain` + `contributors` (verifies the raw value preserves both) and the empty-result case where the body is `searchResult3: {}`. Plus an end-to-end S1 ingest test that asserts the persisted `track.raw_json` column contains the OpenSubsonic extensions after a full runner pass, and that `replay_gain_track_db` / `_album_db` still land on the typed columns via the mapping helper. Full review: psysonic-workdocs/internal/collaboration/handoffs/2026-05-19-pr-796-review.md * feat(library): DeltaSyncRunner + TombstoneReconciler (Phase C3/C4, PR-3c) (#798) Third sub-PR of Phase C — drives targeted delta passes on top of PR-3a/b's foundation. Pure async; PR-3d will spawn it inside the background scheduler. - C3 DeltaSyncRunner. Walks spec §6.4 DS-0 … DS-9: - DS-0/1/2/3 cheap probe via `getArtists` (small/medium tier) or `getScanStatus` (huge tier when `ScanStatusAvailable`). Server watermark match → up_to_date short-circuit, scan-in-progress → deferred_scanning report; zero further requests in either case. - DS-4 targeted ingest. Strategy from capability_flags: N1-delta when NavidromeNativeBulk is set, otherwise S2-delta. S1 has no delta semantic so it's not used here. - N1-delta: GET /api/song _sort=updated_at _order=DESC, pages until rows fall under the local `MAX(server_updated_at)` watermark; out-of-band rows in the same page are dropped. - S2-delta: getAlbumList2 type=newest then type=recent, up to a small page cap; getAlbum is fetched only for album_ids the local store doesn't already have. Known albums are skipped so a play-bump under "recent" doesn't re-ingest the whole tracklist. - DS-6 id remap reuses TrackRepository::upsert_batch_with_remap. - DS-9 stamps next watermark (artists_last_modified_ms or server_last_scan_iso) + last_delta_sync_at. - DS-5 canonical matcher (Phase H) and DS-7 starred delta are out of scope for PR-3c. - C4 TombstoneReconciler. Caller-driven streaming: each `reconcile_chunk(budget)` picks the next `budget` ids ordered by synced_at ASC, calls getSong, marks deleted=1 on code 70, and refreshes synced_at on every checked id so the queue rotates. Mode A (manual integrity) loops until checked == 0; Mode B (auto-threshold) tests `should_auto_reconcile(local, server, pct)` per delta tick and runs a small budgeted chunk. Memory bounded — no full local-id list ever held in RAM. - SyncStateRepository: new getters for artists_last_modified_ms, server_last_scan_iso, library_tier; new set_last_delta_sync_at stamp helper. All column-scoped upserts preserve neighbouring fields. Tests cover DS-2 short-circuit (watermark match), DS-3 defer (scanning=true), N1-delta watermark cutoff (3 fresh + 2 stale rows → only 3 upserted), S2-delta known-album skip (mock 404 on al_known guards the assertion), DS-9 watermark + last_delta stamping, should_auto_reconcile threshold cases (gap, tolerance, server=0, local<=server), reconcile_chunk code-70 → deleted=1, budget + ordering (oldest first, newest untouched), empty-store noop, and cancellation. PR-3d (background task, probe→flags wiring, progress emit, adaptive scheduler, request budget, bandwidth throttle) lands next on the same integration branch. * feat(library): sync supervisor + progress channel + DS-8 wiring (Phase C5/C6, PR-3d1) (#799) First half of PR-3d (cucadmuh-approved split per kickoff Q2). Pure-Rust lifecycle + progress infrastructure on top of the runners from PR-3a/b/c. Tauri events stay in the top crate (PR-5); this PR only ships the channel the top crate will subscribe to. - C5 SyncSupervisor. Spawns a sync workload inside a tokio task, owns the cancellation AtomicBool, and exposes a single-consumer mpsc receiver for ProgressEvent. join() returns the inner Result<(), SyncError>; panics surface as Storage so callers never need to know about tokio internals. - C6 progress channel. New sync::progress module: - ProgressEvent enum — lean variants (PhaseChanged / IngestPage / Remapped / Tombstoned / Completed / Error). Server / scope context lives on the channel side (one supervisor = one scope). - Progress trait + NoopProgress default + ChannelProgress forwarding through tokio mpsc. Throttle is the simple last-emit-timestamp gate; terminal events (Completed / Error) bypass it. - InitialSyncRunner + DeltaSyncRunner gain with_progress(...) builders. IS-1 / IS-6 emit PhaseChanged + Completed; delta emits PhaseChanged at strategy pick, Tombstoned at DS-8, and Completed at DS-9. Defaults to NoopProgress so existing call sites keep working. - DS-8 wired. DeltaSyncRunner::with_tombstone_budget(n) drives TombstoneReconciler::reconcile_chunk(n) after DS-4 ingest; shares the runner's cancellation flag + sleep override. The DeltaSyncReport gains tombstones_checked / tombstones_deleted so callers can act on the counts. - capability::probe_and_persist helper. Chains CapabilityProbe::run with sync_state writes: sets phase to "probing" before the probe, persists capability_flags, then drops back to "idle". PR-3d2 (the scheduler) will call this in front of every initial / delta run so the stored flags reflect the live server. Tests cover: ChannelProgress throttle (zero-interval pass-through, terminal bypass, non-terminal collapse, sender alive after receiver drop), SyncSupervisor task completion + cancel + panic-as-Storage + receiver-take-once, probe_and_persist round-trip through SyncStateRepository (flags persisted, phase ends at "idle"), DS-8 reconcile-after-ingest landing tombstones on code 70 returns. PR-3d2 follows with the adaptive scheduler (C8), request budget (C9), poll EWMA (C10), and the bandwidth / queue priority lane (C11). * feat(library): adaptive scheduler + request budget + EWMA poll + bandwidth (Phase C8/C9/C10/C11, PR-3d2) (#800) Second half of PR-3d per cucadmuh's kickoff-Q2 split. Wraps the runners + supervisor from PR-3a/b/c/d1 into a tick-driven background scheduler. Top crate (PR-5) plumbs the timer. - C8 BackgroundScheduler. Tick-based — caller drives the interval, scheduler decides whether the tick should run. is_due(now_ms) checks sync_state.next_poll_at; tick(now_ms) either skips (not due / PrefetchActive pause), or runs a DeltaSyncRunner with the right budget + tombstone trigger, then stamps the next poll_at via the adaptive formula. No tokio task ownership — tests stay deterministic, PR-5 plugs spawn behaviour to taste. - C9 RequestBudget. PassKind enum (PollTick / DeltaLight / DeltaMismatch / InitialSync) with caps per spec §6.2.5 (1 / 50 / 200 / unlimited). RequestBudget::has_room(used) gates the runner; PR-3d2 ships the data type, runner enforcement of the cap is a future tightening (DeltaSyncRunner already has its own page cap so the soft cap mostly informs Settings). - C10 PollStats EWMA. New sync::poll_stats with PollStats (artist_count, ewma_bytes, ewma_duration_ms, library_tier), observe()/set_artist_count()/reclassify() helpers, the §6.2.2 tier table (<2k / 2k-15k / >15k or ewma_bytes >2MB), and next_interval_ms following the spec formula (base * load_factor * artist_factor, load_factor clamped [1, 10]). - C11 PlaybackHint + ParallelismBudget. PlaybackHint enum (Idle / Playing / PrefetchActive) resolved to a ParallelismBudget { max_concurrent, min_request_gap_ms }. PrefetchActive pauses bulk (`max_concurrent = 0`) per §6.2.4; the scheduler honours it via tick short-circuit. - Auto-tombstone wire. Before running the DeltaSyncRunner the scheduler tests `should_auto_reconcile(local, server, pct)` against the persisted counts; on threshold trip it sets `with_tombstone_budget(200)` (the §6.2.5 DeltaMismatch cap). - SyncStateRepository gains poll_stats_json get/set, next_poll_at get/set, local_track_count get/set, and server_track_count get/set — all column-scoped upserts. Tests: ~30 new across poll_stats / budget / bandwidth / scheduler. EWMA seed + smoothing, tier-classification edges (artist + size overrides), next-interval formula bounds (idle base, slow-network load_factor clamp), RequestBudget caps per pass, ParallelismBudget resolution, scheduler is_due (no schedule / future schedule), tick short-circuit (not due, PrefetchActive pause), tick runs delta and persists next_poll_at, auto-tombstone trigger above 5 % threshold, PollStats round-trip through SQLite. Together with PR-3d1 this finishes Phase C — Tauri command surface (D1-D4) lands with PR-5. * feat(library): read-only Tauri command surface (Phase D1 part 1, PR-5a) (#801) First sub-PR of Phase D per cucadmuh's kickoff Q1 split. Lands the LibraryRuntime Tauri State plus the 8 read-only library commands from spec §7.1. No SyncSupervisor spawn, no sync lifecycle commands, no credentials store — those land in PR-5b. - New psysonic_library::runtime::LibraryRuntime — Tauri State wrapping Arc<LibraryStore>. Top crate's lib.rs setup() now calls LibraryStore::init(app), wraps the result in the runtime, and app.manage's it. Mirrors the AnalysisCache wiring above it. - New psysonic_library::dto module — camelCase wire DTOs per src-tauri/CLAUDE.md: SyncStateDto, LibraryTrackDto (flat projection over the track hot columns + raw_json sub-tree), LibraryTracksEnvelope, TrackArtifactDto, TrackFactDto, OfflinePathDto, TrackRefDto. local_tracks_max_updated_ms helper surfaces the implicit N1-delta watermark on the SyncStateDto. - New psysonic_library::payload module — pure ProgressEvent → LibrarySyncProgressPayload mapper (the payload Tauri events carry once PR-5b plugs the supervisor's mpsc receiver into AppHandle::emit). Constants for the event names too. Unit-testable without Tauri runtime. - New psysonic_library::commands module with 8 #[tauri::command] handlers: - library_get_status — joins the sync_state row + the track-watermark MAX query into one SyncStateDto. - library_search — FTS5 via the existing search_tracks helper, paginated; hydrates hits to full LibraryTrackDto. - library_get_track — single SELECT through new TrackRepository::find_one. - library_get_tracks_batch — capped at 100 refs/call per spec, preserves caller-supplied order, drops unknowns silently. - library_get_tracks_by_album — ordered by disc/track/id via new TrackRepository::find_by_album. - library_get_artifact — flexible WHERE over track_artifact (artifact_kind required, source/format optional), latest fetched_at wins. - library_get_facts — fact_kinds filter optional; returns all rows for the (server_id, track_id) pair when none specified, sorted by fact_kind + fetched_at DESC. - library_get_offline_path — returns local_path with a `missing: true` flag when the row is absent. - TrackRepository gains find_one / find_batch / find_by_album with a shared row-to-TrackRow mapper. SQL constants pinned next to the existing UPSERT_SQL so a schema change touches one file. - src-tauri/src/lib.rs: LibraryStore::init in setup(), the eight command handlers added to invoke_handler!. Tests cover: DTO field-name camelCase (IPC contract guard), LibraryTrackDto round-trip through TrackRow, raw_json fallback to Value::Null on bad input, local_tracks_max_updated_ms ignores deleted rows, TrackRepository::find_one / find_batch / find_by_album ordering + unknown-ref drop, ProgressEvent mapper across all six variants + serialization keys camelCase. Library tests at 166; workspace stays green. Out of scope per kickoff Q1: - Mutating commands (library_sync_*, library_patch_*, library_put_*, library_purge_*, library_delete_*) → PR-5b - SyncSupervisor spawn + background scheduler tick loop + progress emit → PR-5b - library_sync_bind_session / clear_session credentials → PR-5b - TS wrappers + Settings UI + server-remove modal → PR-5c - library_advanced_search / library_search_cross_server SQL builders → PR-5d * feat(library): sync lifecycle + mutate + purge Tauri surface (Phase D1 part 2, PR-5b) (#802) Second sub-PR of Phase D per cucadmuh's kickoff Q1 split. Adds the mutating side of §7.1 plus the SyncSession credentials store, the PlaybackHint setter, the orchestrator that runs InitialSyncRunner / DeltaSyncRunner under a Tauri AppHandle and emits library:sync-progress and library:sync-idle events, and the top-crate scheduler tick task that sweeps every bound session through BackgroundScheduler::tick. - LibraryRuntime extended per kickoff Q2: sync_sessions HashMap, playback_hint cell, current_job (cancel handle + identity), and scheduler_cancel flag the tick task watches. Kickoff sketch said Mutex<Option<SyncSupervisor>> — supervisor's join() consumes self, so holding it in the mutex would block library_sync_cancel behind the orchestrator's join; CurrentJob carries the Arc<AtomicBool> cancel + metadata instead, orchestrator task owns supervisor / receiver / join. - New commands (spec §7.1): - library_sync_bind_session — caches Subsonic creds in memory, tries navidrome_token once for bearer cache, runs probe_and_persist so capability_flags reflect the live server. - library_sync_clear_session — drops cached credentials. - library_set_playback_hint — JS pushes idle / playing / prefetch_active from existing audio listeners. - library_sync_start — dispatches InitialSyncRunner (mode='full') or DeltaSyncRunner (mode='delta', with auto-tombstone budget when local/server count gap exceeds threshold). Spawns runner + orchestrator task that drains the progress mpsc into library:sync-progress emits and emits library:sync-idle when the runner exits. - library_sync_cancel — trips the current job's cancel flag. - library_patch_track — sparse JSON patch (starredAt, userRating, playCount, playedAt) per §6.5. - library_put_artifact / library_put_fact — upserts with ON CONFLICT scoped to the PK so lyrics / BPM writes survive re-fetches. - library_purge_server — transactional DELETE across the v1 schema tables for this server_id. include_offline (default false) controls track_offline + bytes_freed. - library_delete_server_data — alias that always purges offline too (logout flow). - src-tauri/src/lib.rs setup() spawns a 30 s MissedTickBehavior::Skip task that snapshots bound sessions and drives BackgroundScheduler::tick(now_ms) for each. Honours runtime.scheduler_cancel + the current PlaybackHint. Background ticks stay silent (NoopProgress) — Tauri emit for the scheduler path lands when Settings (PR-5c) surfaces it. - psysonic-integration::navidrome re-exports navidrome_token so the bind_session command can drive the bearer cache without making the client module pub. Tests cover: LibraryRuntime session round-trip (set/get/clear scopes per server), playback_hint default + setter, snapshot returns clones so callers can mutate freely. Existing library tests stay green (171 → 171; new code paths under the Tauri command surface — devtools integration smoke is PR-5c's job). Out of scope per kickoff Q1: - src/library/ TS wrappers + Settings UI subsection + server-remove modal → PR-5c - library_advanced_search / library_search_cross_server SQL builders → PR-5d - Background-tick Tauri emit (NoopProgress today) → PR-5c - analysis_cache cross-purge in library_purge_server → PR-6 * feat(library): typed invoke wrappers + verify_integrity command (Phase D2 + part of D1, PR-5c) (#803) Frontend-facing slice of Phase D. Ships the typed src/api/library.ts wrapper layer that any Settings / browse code will import from, plus the manual-integrity backend command PR-5b's review §5 note 2 called out as missing. Scope cut from cucadmuh's PR-5 kickoff Q1 split: that proposal had PR-5c = D2 + D3 + D4 (wrappers + Settings subsection + server-remove modal). The Settings UI + server-remove + audio playback-hint wiring + authStore extensions + i18n strings turn into a thick frontend patch in their own right; landing them in one PR with the wrappers would mix Tauri-surface review with Settings UX review. The split: - PR-5c (this PR) — D2 wrappers + library_sync_verify_integrity. - PR-5c-ui (follow-up) — D3 Library Settings subsection, D4 server-remove modal contract, playback hint feed, authStore / i18n. Per kickoff exit clause ("Do not split 5c unless review size forces it"). Reviewable as a clean Tauri-surface vs UX boundary. - Backend: `library_sync_verify_integrity { serverId, libraryScope? }` command — same dispatch shape as `library_sync_start { mode:'delta' }` but always forces the full `DELTA_MISMATCH_CAP` tombstone budget regardless of the local/server count gap. Spec §6.7 Mode A user- initiated full reconcile bypasses the threshold check that governs background ticks. `library_sync_start` itself is refactored to delegate to a private `library_sync_start_inner(force_full_tombstone)` so both entry points share the runner-spawn + orchestrator + emit code. - Frontend `src/api/library.ts`: full typed wrapper layer over the 19 `library_*` Tauri commands. DTO mirrors carry the camelCase wire shape (`SyncStateDto`, `LibraryTrackDto`, `TrackArtifactDto`, `TrackFactDto`, `OfflinePathDto`, `PurgeReportDto`, `SyncJobDto`, `TrackRefDto`, `ArtifactInputDto`, `FactInputDto`). Plus the `LibrarySyncProgressPayload` / `LibrarySyncIdlePayload` interfaces and `subscribeLibrarySyncProgress` / `subscribeLibrarySyncIdle` helpers that wrap `@tauri-apps/api/event` listen. PlaybackHint literal type lives here too (`'idle' | 'playing' | 'prefetch_active'`) so the audio listeners in PR-5c-ui can import a single source of truth. - `src-tauri/src/lib.rs` adds the new verify_integrity handler to the `invoke_handler!` aggregate. Tests: library tests stay at 171 — verify_integrity is exercised through the existing `sync_start_inner` paths; the wrapper layer is trivial passthrough that TypeScript types already check. Vitest coverage for the typed wrappers belongs with PR-5c-ui where there are real consumers (LibraryTab) to drive integration tests. PR-5c-ui (next) lands: - Library Settings subsection (§7.3 minus advanced toggles) - ServerRemoveModal extension (keep vs delete local index per §5.6) - authStore: libraryIndexEnabledByServer + auto-reconcile toggle - src/store/audioListenerSetup audio:playing / ended / setDeferHotCachePrefetch → library_set_playback_hint - i18n keys for the new strings * feat(library): Settings library index UI + playback hint + purge-on-remove (Phase D3/D4, PR-5c-ui) (#804) * feat(library): Settings library index UI + playback hint + purge-on-remove (Phase D3/D4, PR-5c-ui) Frontend half of Phase D, on top of PR-5c's typed wrappers. Wires the Settings → Library subsection (§7.3), the audio playback-hint feed (§6.2.4), and the server-remove keep-vs-delete choice (§5.6). - New libraryIndexStore (Zustand, persisted) — per-server enable flag + auto-reconcile toggle. Kept out of authStore so the index feature evolves independently and the persisted blob stays small. - New LibraryIndexSection in Settings → Library: - Per-server "Enable local library index" toggle → binds / clears the Rust sync session with the active server's credentials. Off by default (P6). - Read-only status (Idle / Checking / Initial sync / Ready (n) / Error) polled from library_get_status every 3 s, overlaid with live library:sync-progress events. - Sync now / Verify integrity / Cancel buttons. Verify runs one §6.7 pass (budget 200) per click; the status line shows the checked/removed counts so large libraries can be continued with another click (auto-resume loop is a follow-up). - Auto-reconcile toggle. - Subscribes to library:sync-progress + library:sync-idle for the active server; errors surface as a toast. - Audio playback hint: handleAudioPlaying → 'playing', handleAudioEnded → 'idle' via notifyLibraryPlaybackHint, which gates on the per-server index toggle + dedupes repeated hints so the IPC boundary isn't spammed on every progress tick. - ServersTab delete flow: when a server with an enabled index is removed, a second confirm offers keep-vs-delete of the local library cache (OK = library_delete_server_data, Cancel = retain for offline). Always clears the sync session. - i18n: en + de keys for the new strings; other locales fall back to en via i18next (later sweep). Per PR-803 review §5: verify-integrity resume UX is one-pass-per-click with a visible counter; sync_start idempotency (replaces in-flight) is surfaced via the Cancel button appearing while busy. Out of scope: - VirtualSongList / playerStore local-mode consumers → PR-7 (F1/F3/F5) - library_advanced_search / cross-server UI → PR-5d + PR-7 F2 - Auto-resume loop for very large verify-integrity runs → follow-up - Search-all-servers + threshold input (advanced §7.3) → later * fix(library): normalize server base URL before bind probe The bind toggle threw "subsonic transport: builder error | relative URL without a base" — `server.url` is stored bare (e.g. `nas.example.com`) and reqwest needs a scheme. Two-sided fix: - Frontend: LibraryIndexSection passes `authStore.getBaseUrl()` (adds http:// + strips trailing slash) instead of the raw `server.url`, matching the existing `subsonic.ts` convention. - Backend: `library_sync_bind_session` normalizes the incoming `base_url` defensively so the stored session + every downstream caller (sync_start, scheduler tick, navidrome_token) gets a scheme-qualified URL regardless of what the WebView sends. Tests: normalize_base_url covers bare host, trailing slash, existing http/https scheme, and whitespace. * fix(library): re-bind sync session on startup + server switch "Library sync failed: no bound session" — the per-server index toggle persists in localStorage but the Rust sync session (credentials + bearer) lives in process memory and is gone after an app restart, so the toggle showed "on" while no session existed. Per PR-5 kickoff Q5 ("on server connect if index already on"). - New `ensureActiveServerSessionBound()` helper: re-binds the active server's session when its index toggle is enabled. Best-effort — silent on failure (Settings surfaces the real error on explicit toggle). - MainApp re-binds on every `activeServerId` change (covers app startup + server switch — `setActiveServer` drives the effect). - LibraryIndexSection re-binds on mount before the first status poll, so Sync now / Verify integrity work immediately even when the toggle was already on from a previous run. * fix(library): trigger initial full sync on first enable (PR-804 review §5.1) cucadmuh's PR-804 review flagged this as release-blocking: the toggle only bound the session and «Sync now» / the background tick ran delta-only, so a fresh enable left the index empty — delta can't populate a never-synced library. - On first enable, after bind, fetch status and dispatch `library_sync_start { mode: 'full' }` when `lastFullSyncAt` is null (matches spec §6.2 "initial sync always background"). - «Sync now» now picks mode adaptively: `full` until a full sync has completed, `delta` afterward — so the button works both for the initial population and incremental updates. Other PR-804 review notes (auto-reconcile toggle → backend wiring, prefetch_active hint, clear-old-session-on-switch) stay as documented non-blocking follow-ups. * feat(library): advanced search + cross-server SQL builders (Phase D-search, PR-5d) (#806) * feat(library): advanced search + cross-server SQL builders + commands (Phase D-search, PR-5d) - FilterFieldRegistry SQL resolution: SqlFragment, compare_fragment, validate_for_entity (§5.13.5) - Advanced Search builder: per-entity track/album/artist queries; genre (case-insensitive), year, starred, bpm filters; bpm dual-storage resolution (§5.13.4); libraryScope; sort allowlist; full-match totals - Cross-server FTS union (§5.5B / §5.9 A') with canonical-id dedup - library_advanced_search + library_search_cross_server commands, registered in the shell * feat(library): typed advanced search / cross-server invoke wrappers (PR-5d) Mirror request/response DTOs and add libraryAdvancedSearch / librarySearchCrossServer in src/api/library.ts. UI parity (AdvancedSearch.tsx) stays PR-7. * fix(library): self-heal stale/unreadable initial-sync cursor instead of bricking (#807) The initial-sync cursor records the ingest strategy it was created under. When a re-probe later selects a different strategy (e.g. the Navidrome native bearer is briefly unavailable, downgrading N1->S2), the cursor guard returned a hard error — and since nothing clears the cursor, every later full sync failed with no recovery path. Reset the stale (or unreadable) cursor and start fresh under the selected strategy instead of erroring. Re-ingest is idempotent (upsert); the tombstone pass reconciles leftovers. * fix(library): emit per-batch progress during initial sync (#808) The initial-sync runner only emitted PhaseChanged (start) and Completed (end), so the Settings status sat at "initial_sync" with no count for the entire ingest — looking stuck on large libraries even while rows landed. Emit IngestPage per batch from the N1/S1/S2 loops with the running ingested total; the existing <=2 Hz throttle paces it. The frontend already renders the count from these events. * feat(library): Advanced Search reads the local index when ready (Phase F2, PR-7a) (#811) When the active server's index is fully synced, Advanced Search serves query / genre / year / result-type from library_advanced_search (instant + offline) and pages songs locally. On not-ready or any failure it falls back to the existing network path unchanged (spec 5.13.6). Results map from each entity's stored Subsonic rawJson, with the flat hot columns as a fallback. * feat(library): canonical matcher — link tracks by ISRC/MBID on ingest (Phase H1/H2, PR-4a) (#812) Adds the strong-key cross-server matcher (spec §5.5A): on every track upsert, link (server_id, track_id) to a canonical id derived from its ISRC (preferred) or MBID recording. Deterministic id (`{kind}:{value}`) keeps it O(1) and idempotent — no lookup-then-create race, no fuzzy loop on the bulk path. Tracks without a strong key stay standalone (fuzzy/search-time matching is H3). * feat(library): cross-server fuzzy fallback in search (Phase H3, PR-4b) (#813) library_search_cross_server now returns a `fuzzy` list alongside the exact FTS `hits` (spec §5.9): per-server `title LIKE %query%` for matches the exact pass missed (diacritics, partial words), capped per server, excluding exact hits and deduped by canonical id against them. Shared `like_contains` moved to the `search` module. * feat(library): FactRepository with TTL + provenance rules (Phase E4, PR-6a) (#814) Typed CRUD over track_fact behind library_get_facts / library_put_fact (spec §5.12): get lazily deletes the track's expired facts then returns the survivors (no background GC, P34); a `user` bpm fact also writes the hot track.bpm column so the override wins and survives a resync (R6-3.4). The commands now delegate here instead of inlining the SQL. * feat(library): ArtifactRepository with TTL + 512KB cap (Phase E4, PR-6b) (#815) * fix(integration): decode OpenSubsonic isrc string-array on Song (#818) OpenSubsonic types `isrc` as `string[]`; Navidrome 0.61.2 ships it as `isrc: []` or `["USRC…"]`. The typed `Song.isrc: Option<String>` could not decode either form, which broke the S1 (`search3`) and S2 (`getAlbum`) ingest paths on real Navidrome libraries — initial sync could not complete past the first array-valued track. Add a tolerant `de_string_or_seq` deserializer: plain string → `Some`, non-empty array → first usable value (string element, or an object element's `name` for the `[{ "name": … }]` shape), `[]`/null → `None`. The full multi-value set still survives verbatim in `track.raw_json` (ADR-7). Applied to `Song.isrc`. Per maintainer policy R7-15 (workdocs question 2026-05-20-large-library-ingest-client-only, checklist item 1): treat Navidrome as a black box, harden the client decode. Tests cover `isrc: []` → None, populated array, and the legacy single-string form. * feat(library): large-library ingest strategy — S1 over N1 (R7-15) (#819) Per maintainer policy R7-15 (large-library ingest, client-only): very large Navidrome catalogs must not start initial sync on N1 — its native `/api/song` returns HTTP 500 beyond a deep offset and can never finish. S1 (`search3`) does not hit that wall. - Add `IngestStrategy::select_initial_strategy(flags, server_track_count, n1_bulk_unreliable)`. Large libraries (count > LARGE_LIBRARY_THRESHOLD, default 40_000) or servers flagged `n1_bulk_unreliable` route to S1 — or S2 when search3 bulk is absent. Normal-size libraries keep the cheapest N1 → S1 → S2 chain unchanged. - Persist the learned per-server `n1_bulk_unreliable` flag on `sync_state` (additive migration 002, DEFAULT 0). The mid-run N1→S1 fallback that sets it lands in a follow-up. - Capture `getScanStatus.count` in the capability probe and persist it as the `server_track_count` watermark, so the threshold applies from the first sync rather than only after N1 hits the wall once. A count-less probe never clobbers a watermark from a prior run. - The initial-sync runner now selects via the new policy. Tests: selector table (all branches incl. threshold boundary and the search3-absent fallback), repo flag roundtrip, probe count capture + watermark-preservation, migration head-version bookkeeping. * feat(library): freeze ingest strategy on resume (R7-15 Q3) (#820) A persisted initial-sync cursor that has already made progress must resume under its own strategy and ignore what a fresh capability probe would now pick. Previously any strategy mismatch reset the cursor to a fresh one — so a flapping Navidrome bearer (N1 flag toggling between probes) restarted ingest from offset 0 on every launch, which is why large initial syncs never completed across restarts. `load_or_init_cursor` now: - resumes the cursor's strategy when it has progress (`ingested_count > 0` or `phase != Ingest`), regardless of the re-selected strategy; - adopts the freshly-selected strategy only when there is no resumable progress (offset 0), where re-selecting costs nothing; - still resets a corrupt/unreadable cursor rather than hard-erroring. One guarded exception: a cursor still on N1 after the server was learned `n1_bulk_unreliable` is known-broken and re-selects onto the non-N1 path instead of resuming a wall-bound N1 loop (the mid-run N1→S1 fallback that preserves progress lands next). Tests: resume-with-progress freezes strategy and keeps the count; no-progress cursor adopts the re-selected strategy; known-broken N1 cursor re-selects; unreadable cursor still resets. * feat(library): one-way N1→S1 fallback on deep-offset 500 (R7-15 Q5) (#821) When the N1 ingest loop hits a persistent HTTP 500 at or beyond the deep-offset safety line (`N1_DEEP_OFFSET_SAFE`, 50_000) it now treats it as Navidrome's server-side deep-offset wall rather than a transient error: it learns `n1_bulk_unreliable` for the server and finishes the sync on S1. - `run_n1` catches the wall after retry exhaustion (`n1_hit_deep_offset_wall`: HTTP 500 AND offset >= the safety line) and hands off to `fall_back_n1_to_s1`. A 500 below the line stays a propagated error — no silent downgrade. - The fallback flags the server, then restarts S1 from offset 0. N1 (`id ASC`) and S1 (`search3` default order) don't share an offset space, so resuming from the N1 offset would skip songs; re-ingest is idempotent (PK upsert), duplicate work over the rows N1 already wrote is acceptable for v1. The cursor is rewritten in place, never zeroed. - One-way only: S1 never flips back to N1 mid-run. Combined with the persisted flag and the resume freeze, a future sync selects S1 directly. - `N1_DEEP_OFFSET_SAFE` is overridable on the runner so the fallback is testable without 50k rows of fixture data. Tests: deep-offset 500 falls back to S1, ingests the full set without duplicating N1's rows, and persists the flag; a shallow 500 propagates and does not flag the server. * feat(library): cache + retry Navidrome bearer, keep N1 flag on transient loss (R7-15 Q3) (#822) A flaky `/auth/login` previously stripped N1 for a whole bind: the bearer was fetched once, best-effort, and a single miss dropped to Subsonic-only. Per R7-15 Q3 a transient `navidrome_token` failure must not drop the `NavidromeNativeBulk` capability. - `bind_session` fetches the bearer with `navidrome_token_with_retry` (3 attempts, short backoff); if it still fails, it keeps the bearer cached from a prior bind instead of overwriting it with `None`. The token / credentials are never logged. - `probe_and_persist` preserves a previously-learned `NavidromeNativeBulk` flag when it probes without a token — the server still supports `/api/song`; only the bearer is missing this bind. The capability is a stable server property, so a token-less probe must not clear it. - `library_sync_start_inner` masks `NavidromeNativeBulk` from *this run's* strategy selection when the session has no token, so the run proceeds Subsonic-only (S1/S2) instead of selecting N1 with no creds. The persisted capability stays intact for a later bind that recovers the token. The in-flight cursor is already protected by the resume freeze. Tests: token retry yields the token on success and `None` after exhausting attempts; the probe keeps a learned N1 flag across a token-less re-probe. * feat(library): mid-run S1→S2 fallback on persistent S1 failure (R7-15 Q8) (#823) The N1→S1 fallback (#821) had no analogue when S1 itself fails on a server. Per R7-15 Q8, a persistent S1 failure (C12 retries already exhausted) must fall back to the universal S2 album crawl — no new artist-walk strategy. - `run_s1` catches a persistent fetch failure from the `search3` retry loop (`is_fetch_failure`: transport / HTTP / decode / Subsonic API / not-found) and hands off to `fall_back_s1_to_s2`. Cancellation and storage errors propagate untouched. - The fallback restarts S2 from scratch. S1 (`search3` order) and S2 (album-list order) don't share an offset space, so resuming from the S1 offset would skip songs; re-ingest is idempotent (PK upsert). The cursor is rewritten in place, never zeroed — the resume freeze then keeps the run on S2 across restarts. This completes the ingest fallback chain N1→S1→S2 from the §6.3 strategy order; the start-time "no search3 → S2" selection was already covered (#819). Tests: a persistent S1 500 falls back to S2 and the album crawl ingests the track. * fix(library): resume interrupted initial sync on startup (#824) * fix(library): resume interrupted initial sync on startup An initial sync killed mid-run (app restart) sat at `idle` until the user clicked «Sync now» — the background scheduler is delta-only and the auto-full-sync only fired on the index toggle, not on the startup re-bind. `resumeInitialSyncIfIncomplete` runs after the active server's session is re-bound (startup + server switch): if no full sync has completed yet (`!lastFullSyncAt`) it dispatches `library_sync_start { mode: 'full' }`, which resumes from the persisted cursor instead of restarting from zero. Once a full sync has landed it is a no-op, so delta stays the scheduler's job. Best-effort — errors stay silent (Settings surfaces them on explicit action). Tests: starts a full sync when none has completed, no-ops once a full sync has landed, stays silent when the status lookup fails. * fix(library): silence cancelled-sync toast, de-dupe startup resume Two rough edges from the startup resume: - A cancelled sync surfaced as «Library sync failed: sync cancelled». The orchestrator emitted the runner's `Cancelled` result as an error on the sync-idle event. Cancellation is expected — the user cancelled, or a newer `library_sync_start` superseded the job (server switch / startup resume) — and is documented as silent. `sync_outcome_to_result` now maps `SyncError::Cancelled` to a clean idle, only real errors toast. - `resumeInitialSyncIfIncomplete` is now de-duped per server. React StrictMode fires the startup effect twice, so a second `library_sync_start` cancelled the first (`set_current_job` is cancel-and-replace) — harmless with the fix above, but the dedupe avoids the wasted job + probe entirely. Tests: `sync_outcome_to_result` keeps `Cancelled` silent and forwards real errors; concurrent resume calls start a single full sync. * fix(library): run DB read commands off the main thread (async) (#825) The 10 library read commands were synchronous (`pub fn`). Per the Tauri v2 docs, commands without `async` run on the main thread — so a read that blocks freezes the UI. During an initial sync the runner holds the single `Mutex<Connection>` for a whole batch write (500 rows × per-row remap on Navidrome + upsert + FTS, one transaction), and the Settings library section polls `library_get_status` on an interval. Each batch write blocked that polled read on the main thread → the window greyed out until the batch finished, with the freeze growing as the DB grew. Make the DB-touching read commands `async` so they run off the main thread: `library_get_status`, `library_search`, `library_get_track`, `library_get_tracks_batch`, `library_get_tracks_by_album`, `library_get_artifact`, `library_get_facts`, `library_get_offline_path`, `library_advanced_search`, `library_search_cross_server`. Reads still serialize behind the writer (the single connection is intentional — the schema mirrors `analysis_cache`, spec §5.1), but the wait no longer blocks the UI. State-only commands stay sync. Invoke names / payloads are unchanged, so the frontend is unaffected. Spec §15 R7-15 follow-up — surfaced in live QA on a 170k library. * feat(library): scope analysis cache by server_id (E1, schema only) (#826) Add a versioned migration to audio-analysis.sqlite so waveform/loudness rows are keyed per server. This is the schema-only step (PR-6c-1): every existing row migrates to server_id='' and behaviour is unchanged. The server_id write/read wiring, legacy fallback and lazy re-tag follow in 6c-2. - migrations 001 (baseline = the pre-versioning schema) + 002 (rebuild the three tables with server_id; PK (server_id, track_id, md5_16kb), loudness + target_lufs) - versioned runner mirroring the library store; each migration commits its schema change and version marker in one transaction, so a failure or crash rolls the whole migration back and retries cleanly - VACUUM INTO snapshot before the table rewrite as a safety net beyond the transaction (disk-full at COMMIT, FS corruption) - TrackKey gains server_id; all callers pass "" for now * feat(library): analysis cache server_id wiring (E1, 6c-2) (#827) * feat(library): scope analysis cache writes/reads/deletes by server_id (E1 wiring) Build on the 6c-1 schema migration: thread the playback server scope (playbackServerId ?? activeServerId) through the analysis cache so a server switch can no longer surface another server's waveform/loudness for the same bare track_id. - Write: seed_from_bytes_* and the CPU-seed / HTTP-backfill queues carry a server_id; every audio write path (in-memory, ranged, legacy stream, local file, spill, preload), the syncfs offline/hot caches, and the backfill command write under the playback server (empty = legacy ''). - Read: get_latest_*_for_track and the exact-key lookup try the server scope first, then fall back to the legacy '' rows; a legacy hit is re-tagged onto the server scope (INSERT OR IGNORE, never clobbers a precise row). No bulk backfill — existing caches re-tag lazily on play, so they are not re-analysed wholesale. - The backend gain-resolution path (loudness normalization, replay-gain updates, device resume) is scoped via a pinned current_playback_server_id on the audio engine, so normalization keeps working for server-scoped rows. - Delete: delete_*_for_track_id scope to (server + legacy ''); reseed on one server no longer wipes another server's analysis. delete_all_waveforms stays global (Settings -> Storage). Tauri boundary: analysis_get_waveform(_for_track), analysis_get_loudness_for_track, analysis_delete_waveform/loudness_for_track and analysis_enqueue_seed_from_url gain an optional serverId; audio_play and audio_preload gain an optional serverId. All additive (absent = legacy ''). * feat(library): pass playback serverId to analysis IPC (E1 wiring) Send getPlaybackServerId() (queueServerId ?? activeServerId) with every analysis-cache call so reads/writes/deletes scope to the right server: - audio_play / audio_preload (playTrack, resume, queue-undo restore, gapless byte-preload) - analysis_get_waveform_for_track / analysis_get_loudness_for_track (waveform + loudness refresh) - analysis_delete_waveform/loudness_for_track + analysis_enqueue_seed_from_url (reseed + loudness backfill) Absent serverId stays backward-compatible (legacy '' scope). * feat(library): content_hash from playback (E2, 6d) (#828) * feat(library): record playback content_hash into the track store (E2) Bridge the playback-derived md5_16kb into library `track.content_hash` (R7-16 Q4) so id-remap can rebind a track when the server reassigns ids (§6.9). - New `ContentHashSink` port in psysonic-core (closure handle, mirrors PlaybackQueryHandle): keeps psysonic-analysis decoupled from psysonic-library. - `seed_from_bytes_into_cache` returns the computed md5; `seed_from_bytes_execute` fires the sink after a successful seed (Upserted or cache-hit) when a real server is known. The shell crate registers the sink to patch the library. - `patch_content_hash` + `library_patch_track`'s new optional `contentHash` field write it; both no-op when the library has no row for (server_id, id), i.e. the index is off for that server. - Sync upsert no longer clobbers it: `content_hash = COALESCE(NULLIF( excluded.content_hash,''), track.content_hash)` — a sync (which passes NULL) preserves the playback hash, a non-empty incoming hash still wins. No schema migration — the `content_hash` column already exists. Tauri boundary: `library_patch_track` gains optional `contentHash` (additive). * feat(library): expose contentHash on libraryPatchTrack wrapper (E2) Add optional `contentHash` to the `libraryPatchTrack` patch type so the TS contract matches the extended Rust command. Normally written by the Rust analysis bridge; exposed for completeness. * feat(library): enrichment summary on library_get_track (E3, 6e) (#829) * feat(library): enrichment summary on library_get_track (E3) Add an optional `enrichment { waveformReady, loudnessReady, lyricsCached }` to the single-track `library_get_track` read (R7-16 Q5). Read-only, per-server, never blocks on the network; list/batch projections leave it unset. - New `AnalysisReadinessQuery` port in psysonic-core (closure handle, mirrors ContentHashSink) keeps psysonic-library decoupled from psysonic-analysis. The shell crate registers it to probe the analysis cache by exact (server_id, track_id, content_hash) key with legacy '' fallback — read-only, no re-tag. waveform/loudness readiness is gated on a known content_hash (E2). - `lyricsCached` from a new pure-read `ArtifactRepository::lyrics_cached` (valid, non-expired, non-not_found lyrics row). - `library_purge_server`'s `includeAnalysis` documented as a deliberate v1 no-op (R7-16 Q7): analysis is never deleted on purge / server remove. Tauri boundary: `LibraryTrackDto` gains optional `enrichment` (additive). * feat(library): mirror enrichment on LibraryTrackDto wrapper (E3) Add `TrackEnrichmentDto` + optional `enrichment` to the TS `LibraryTrackDto` so the contract matches the extended `library_get_track` response. * feat(library): VirtualSongList browses the local index when ready (F1) (#830) The all-songs browse now serves pages from the local library index when it is ready for the active server, falling back to the unchanged network path otherwise. - `runLocalSongBrowse` (reuses the F2 local-read adapters): empty-query browse-all via `library_advanced_search`, whose default track order (`t.title COLLATE NOCASE ASC`) matches the network `ndListSongs('title','ASC')` path, so paging stays coherent across a local↔network boundary. - Gated per page on `libraryIsReady` + `source === 'local'`; any miss / failure returns null → VirtualSongList uses the existing browse path unchanged. - Search (non-empty query) stays on the network path for now; rich search is already covered by Advanced Search (F2). * feat(library): patch-on-use for star/rating/scrobble (PR-7 F3) (#831) * feat(library): library_patch_track clears nullable fields on explicit null (F3) Extract the patch logic into a testable `apply_track_patch`. Nullable integer fields (`starredAt` / `userRating` / `playCount` / `playedAt`) now distinguish an absent key (leave untouched) from an explicit `null` (clear the column), so `unstar` ({ starredAt: null }) actually un-stars the local row. `.map` keeps the present/absent distinction; `as_i64()` yields the value or `None` → bound as SQL NULL. F3 is the first caller that sends null, so no existing behaviour changes. * feat(library): patch-on-use wiring for star / rating / scrobble (F3) After a successful star/unstar, setRating, or play scrobble, mirror the change into the local library index via `library_patch_track` so its reads (browse F1, advanced search F2) reflect the action immediately — no stale list after a rate, no full resync. - `patchLibraryTrackOnUse` helper: fire-and-forget, gated on the index being enabled for the server; the Rust command additionally no-ops when no row matches (album/artist id, or index off). - Wired at the central API chokepoints: `star`/`unstar` (song only) → `starredAt`, `setRating` → `userRating`, `scrobbleSong` → `playedAt`. - `play_count` is left to the next sync (the patch sets absolute values; a correct increment needs the current base). F4 (deprecating the player-store override maps) is intentionally separate — removing them would break instant star feedback when the index is off. * feat(library): full-queue restore from the index on startup (PR-7 F5) (#832) Persist the whole queue as a lightweight ref list and rehydrate it from the local index on startup, so the entire queue survives a restart instead of only the windowed slice (R7-17 / §8.6). - Persist adds `queueRefs` (full ordered ids) + `queueRefsIndex` alongside the existing windowed `queue`. Ids are tiny; the windowed objects stay as the no-index fallback. - `hydrateQueueFromIndex` (startup, after session bind): when the library index is ready for the queue's server, hydrate the full queue via `library_get_tracks_batch` (batched ≤100), map `songToTrack ∘ trackToSong`, re-locate the current track so `queueIndex` stays aligned, then clear the refs. - Index not ready / missing rows / current track not found → keep the windowed fallback (queue never empty when the index is off, the P6 default). Old persisted shape without refs loads unchanged. - `trackToSong` exported from the F2 local-read adapters (one mapper). Kept the windowed-objects persist (did not drop the cap per R7-17 note): the index-off default needs the embedded fallback or the queue would restore empty. * feat(library): pending-sync for song star/rating (PR-7 F4) (#833) * feat(library): central pending-sync helper for song star/rating (PR-7 F4) `queueSongStar` / `queueSongRating` (spec §6.5 / R7-18): set the player-store override optimistically, retry the Subsonic API with exponential backoff (flush on `online` / window focus), and on success clear the override + patch the in-memory Track so the UI stays correct without it. The F3 index patch-on-use runs inside the API layer, unchanged. - No rollback on the first network error (the override survives until the retry succeeds or the app restarts; overrides are session-only, not persisted). - Latest-toggle-wins coalescing + an identity guard so a fast re-toggle while a request is in flight can't retire the newer task. - v1: songs only. * feat(library): route song star/rating through the pending-sync helper (PR-7 F4) Replace the scattered optimistic-set + API-call + rollback logic with the single `queueSongStar` / `queueSongRating` helper across cucadmuh's named v1 surfaces: PlayerBar, FullscreenPlayer, MobilePlayerView, both context menus (song + queue row), both shortcut paths, the song-rating hook + player-bar stars, skip→1★, and AlbumDetail (song star + rating). The 30+ override read sites are unchanged — they already read `override ?? track`, and the override now clears on success. Standalone page toggles (Favorites, RandomMix, NowPlaying star) and the separate mini-player webview keep their existing path — no regression (a non-migrated override simply lingers as before) — and move to a follow-up. * feat(library): route remaining song star/rating sites through pending-sync (F4 follow-up) (#834) Migrate the three standalone song write sites left out of #833 onto the central queueSongStar / queueSongRating helper: - Favorites: handleRate + removeSong (un-star) - RandomMix: toggleSongStar (drops local try/catch rollback per no-rollback policy) - useNowPlayingStarLove: toggleStar (keeps local view state, helper owns override + retried sync) MiniContextMenu stays on its direct path (separate webview, no shared store). No behaviour change for album/artist rating paths. * feat(library): route playlist song star/rating through pending-sync (F4 follow-up) (#835) The playlist-detail star/rating hook was the last shared-store song write site still calling the Subsonic API directly. Route handleRate + handleToggleStar through queueSongRating / queueSongStar, matching the Favorites and RandomMix follow-ups; keep the local ratings/starredSongs view state, drop the inline override. MiniContextMenu remains on its direct path (separate webview). * feat(library): BPM range filter UI in Advanced Search (PR-7 F6) (#836) * feat(library-sync): parallel initial ingest (S2 + N1/S1 prefetch) Wire C11 ParallelismBudget (max 4 when idle) into InitialSyncRunner: parallel getAlbum for S2, up to 4 in-flight pages for N1/S1, and persist S2 cursor once per album-list page instead of per album. * fix(library-sync): defer scheduler during initial sync and improve ingest diagnostics Background delta/tombstone ticks every 30s were competing with IS-3 bulk ingest for the write mutex (20–60s lock waits on large libraries). Skip scheduler while sync_phase is initial_sync/probing or bulk ingest is active. Serialize ingest batch metrics as camelCase for DevTools, add bulk-ingest FTS/index suspension, combined cursor persist, write-op tracing, live local search, and library dev logging helpers. * fix(library-search): scoped FTS, cancel stale live search, skip 1-char queries Use column-scoped FTS for artists/albums/songs, min two graphemes for local FTS, capped match counts in Advanced Search, and title browse index (m004). Live Search aborts superseded network requests, passes requestEpoch to drop stale Rust FTS, and avoids search3 fallback for too-short queries. * fix(library-search): prefix FTS, fast subquery joins, hide BPM in Advanced Search Live and Advanced Search now use FTS5 prefix tokens ("metal"*) and limit bm25 ranking inside rowid subqueries so large libraries stay in the ms range. Advanced Search BPM filter is removed from the UI until enrichment ships. * feat(library-search): race local index vs search3, show first result Live Search and Advanced Search text queries run library and network backends in parallel; the faster source wins. Adds searchRace helper and search_race dev logging. * feat(library-index): multi-server UI, serial sync queue, scoped local search Add master library index toggle with per-server rows, offline retry, and a frontend sync queue so initial ingest runs one server at a time. Scope Live Search and Advanced Search to the sidebar music library filter via library_id and raw_json fallbacks; coerce numeric libraryId on ingest. Promote idle sync state to ready when a full sync stamp exists and block cross-server initial sync starts in Rust. * chore(library-store): compliance — clippy, i18n, CHANGELOG Fix clippy/tsc blockers (request structs, IngestPageCtx, type aliases), add library index strings to all 9 locales, and document the preview feature in CHANGELOG [1.47.0] with Psychotoxical + cucadmuh attribution. * docs(credits): library index preview contributions Credit Psychotoxical for the local library store foundation and cucadmuh for multi-server UI, scoped search, and i18n. Drop removed scan-trigger wording from PR #780 entry. * docs(release): link library index preview to PR #846 * docs(changelog): sort [1.47.0] entries by ascending PR number * docs(changelog): mark library index as Added in [1.47.0] * docs(changelog): restructure [1.47.0] into Added/Changed/Fixed Match 1.46.0 layout: new features in Added (incl. library index), enhancements in Changed, bug fixes in Fixed — PR ascending within each block. * fix(library): address PR #846 review — delta guard + FTS order Skip background scheduler delta when LibraryRuntime already has a foreground sync job for the same server. Preserve bm25 rowid ordering in live search track/artist/album fetches. * fix(library): address remaining PR #846 review items S2 resume persists current_album_id per album; same-server resync awaits the previous runner. N1 delta watermark uses strict less-than; Navidrome HTTP 500 detection is structured. Adds genre/year indexes, backoff jitter salt, LiveSearch failure toast, and user-facing search badge copy. * fix(library): close resync notify race and tighten FTS trigger test Use notify_one() so an early runner completion cannot lose the drain signal before same-server full resync awaits. FTS test now compares normalized trigger bodies from migration vs suspend/restore roundtrip. --------- Co-authored-by: Frank Stellmacher <171614930+Psychotoxical@users.noreply.github.com>
This commit is contained in:
cucadmuh
GitHub
@@ -0,0 +1,151 @@
|
||||
//! C12 — HTTP backoff for ingest batches (spec §6.8).
|
||||
//!
|
||||
//! The runner does not advance its cursor on transient failure; it
|
||||
//! sleeps via `Backoff::next_delay`, retries the same batch, and
|
||||
//! resets the counter on success. The cap is 120 s — the user is
|
||||
//! waiting for sync to finish, longer hangs become indistinguishable
|
||||
//! from "stuck" without progress events to reassure them.
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
/// Exponential schedule per §6.8: `2s → 4s → 8s → … cap 120s`. The
|
||||
/// caller adds jitter on top to avoid herd-sync against a recovering
|
||||
/// upstream proxy.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Backoff {
|
||||
attempt: u32,
|
||||
base: Duration,
|
||||
cap: Duration,
|
||||
}
|
||||
|
||||
impl Default for Backoff {
|
||||
fn default() -> Self {
|
||||
Self::new(Duration::from_secs(2), Duration::from_secs(120))
|
||||
}
|
||||
}
|
||||
|
||||
impl Backoff {
|
||||
pub fn new(base: Duration, cap: Duration) -> Self {
|
||||
Self { attempt: 0, base, cap }
|
||||
}
|
||||
|
||||
/// Reset after a successful batch — the next failure starts at
|
||||
/// the base delay again.
|
||||
pub fn reset(&mut self) {
|
||||
self.attempt = 0;
|
||||
}
|
||||
|
||||
/// Compute the next sleep duration and bump the attempt counter.
|
||||
/// Doubles each call (2 → 4 → 8 → …) and clamps to `cap`. Caller
|
||||
/// adds jitter via `with_jitter` if desired.
|
||||
pub fn next_delay(&mut self) -> Duration {
|
||||
let n = self.attempt;
|
||||
self.attempt = self.attempt.saturating_add(1);
|
||||
let scaled = self.base.saturating_mul(1u32 << n.min(31));
|
||||
scaled.min(self.cap)
|
||||
}
|
||||
|
||||
pub fn attempt(&self) -> u32 {
|
||||
self.attempt
|
||||
}
|
||||
}
|
||||
|
||||
/// Salt for production jitter: attempt plus sub-second clock noise so
|
||||
/// concurrent retries don't share the same jitter slot.
|
||||
pub fn jitter_salt(attempt: u32) -> u64 {
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
let nanos = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.map(|d| d.subsec_nanos() as u64)
|
||||
.unwrap_or(0);
|
||||
u64::from(attempt).saturating_add(nanos)
|
||||
}
|
||||
|
||||
/// Add ±25% jitter to a planned sleep — deterministic per `salt` so
|
||||
/// tests can pin the result without pulling in `rand`. Production
|
||||
/// code passes `jitter_salt(attempt)` as the salt; tests pass a fixed
|
||||
/// value to assert the formula.
|
||||
pub fn with_jitter(base: Duration, salt: u64) -> Duration {
|
||||
let millis = base.as_millis().min(u128::from(u64::MAX)) as u64;
|
||||
if millis == 0 {
|
||||
return base;
|
||||
}
|
||||
let span = millis / 2; // ±25% = total span of 50% of base
|
||||
if span == 0 {
|
||||
return base;
|
||||
}
|
||||
// map salt into [-span/2, +span/2]
|
||||
let pct = (salt % span) as i64 - (span as i64 / 2);
|
||||
let jittered = (millis as i64).saturating_add(pct).max(1) as u64;
|
||||
Duration::from_millis(jittered)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn next_delay_doubles_until_cap() {
|
||||
let mut b = Backoff::default();
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(2));
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(4));
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(8));
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(16));
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(32));
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(64));
|
||||
// Next would be 128 — clamps to 120 cap.
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(120));
|
||||
// Stays at cap from here on.
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(120));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reset_brings_schedule_back_to_base() {
|
||||
let mut b = Backoff::default();
|
||||
b.next_delay();
|
||||
b.next_delay();
|
||||
assert!(b.attempt() > 0);
|
||||
b.reset();
|
||||
assert_eq!(b.attempt(), 0);
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(2));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn next_delay_handles_custom_base_and_cap() {
|
||||
let mut b = Backoff::new(Duration::from_millis(50), Duration::from_secs(1));
|
||||
assert_eq!(b.next_delay(), Duration::from_millis(50));
|
||||
assert_eq!(b.next_delay(), Duration::from_millis(100));
|
||||
assert_eq!(b.next_delay(), Duration::from_millis(200));
|
||||
assert_eq!(b.next_delay(), Duration::from_millis(400));
|
||||
assert_eq!(b.next_delay(), Duration::from_millis(800));
|
||||
// 1600 would exceed 1s cap.
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(1));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn next_delay_does_not_overflow_at_extreme_attempts() {
|
||||
let mut b = Backoff::default();
|
||||
for _ in 0..100 {
|
||||
// Should saturate at the cap, never panic on shift overflow.
|
||||
let _ = b.next_delay();
|
||||
}
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(120));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn jitter_stays_within_plus_minus_half_of_span() {
|
||||
let base = Duration::from_secs(8);
|
||||
let span_ms = base.as_millis() as u64 / 2; // 4000 ms
|
||||
let lo = base.as_millis() as u64 - span_ms / 2; // 6000 ms
|
||||
let hi = base.as_millis() as u64 + span_ms / 2; // 10000 ms
|
||||
for salt in 0u64..1000 {
|
||||
let j = with_jitter(base, salt).as_millis() as u64;
|
||||
assert!(j >= lo && j <= hi, "salt {salt} → {j}ms outside [{lo},{hi}]");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn jitter_with_zero_base_returns_zero() {
|
||||
assert_eq!(with_jitter(Duration::ZERO, 12345), Duration::ZERO);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
//! C11 — bandwidth priority lane (spec §6.2.4).
|
||||
//!
|
||||
//! PR-3d2 ships the data types + parallelism resolver. The signal
|
||||
//! itself is pushed in from the top crate (PR-5 hooks audio engine
|
||||
//! events from `psysonic-audio` into a shared `PlaybackHint` cell);
|
||||
//! the library side just consumes it.
|
||||
|
||||
/// What the player is currently doing — drives crawl parallelism.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
|
||||
pub enum PlaybackHint {
|
||||
/// Nothing playing, queue idle → bulk crawl runs at normal
|
||||
/// parallelism.
|
||||
#[default]
|
||||
Idle,
|
||||
/// Active stream (user listening). Bulk drops to single-request
|
||||
/// crawling and increases inter-request delay.
|
||||
Playing,
|
||||
/// Queue prefetch / waveform analysis hot — priority lane only;
|
||||
/// bulk pauses entirely.
|
||||
PrefetchActive,
|
||||
}
|
||||
|
||||
/// Parallelism budget the bulk crawl uses for this tick. `0` means
|
||||
/// bulk is suspended this tick (PR-3d2 returns the value; the runner
|
||||
/// honoring it is wired in PR-3d2 too via `BackgroundScheduler`).
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct ParallelismBudget {
|
||||
pub max_concurrent: u32,
|
||||
/// Minimum gap between successive bulk requests, in ms.
|
||||
pub min_request_gap_ms: u32,
|
||||
}
|
||||
|
||||
impl ParallelismBudget {
|
||||
pub fn resolve(hint: PlaybackHint) -> Self {
|
||||
match hint {
|
||||
PlaybackHint::Idle => Self {
|
||||
max_concurrent: 4,
|
||||
min_request_gap_ms: 0,
|
||||
},
|
||||
PlaybackHint::Playing => Self {
|
||||
max_concurrent: 1,
|
||||
min_request_gap_ms: 250,
|
||||
},
|
||||
PlaybackHint::PrefetchActive => Self {
|
||||
max_concurrent: 0,
|
||||
min_request_gap_ms: 0,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
pub fn bulk_paused(&self) -> bool {
|
||||
self.max_concurrent == 0
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn idle_resolves_to_normal_parallelism() {
|
||||
let b = ParallelismBudget::resolve(PlaybackHint::Idle);
|
||||
assert_eq!(b.max_concurrent, 4);
|
||||
assert_eq!(b.min_request_gap_ms, 0);
|
||||
assert!(!b.bulk_paused());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn playing_caps_parallelism_to_one_with_inter_request_gap() {
|
||||
let b = ParallelismBudget::resolve(PlaybackHint::Playing);
|
||||
assert_eq!(b.max_concurrent, 1);
|
||||
assert!(b.min_request_gap_ms >= 100, "playing must space requests out");
|
||||
assert!(!b.bulk_paused());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prefetch_active_pauses_bulk_crawl_entirely() {
|
||||
let b = ParallelismBudget::resolve(PlaybackHint::PrefetchActive);
|
||||
assert!(b.bulk_paused());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn playback_hint_default_is_idle() {
|
||||
assert_eq!(PlaybackHint::default(), PlaybackHint::Idle);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
//! C9 — request budget per spec §6.2.5.
|
||||
//!
|
||||
//! Soft caps the scheduler declares before each pass; runners
|
||||
//! consume them to decide when to defer the remainder to the next
|
||||
//! tick. PR-3d2 ships the data type + lookups; PR-5 / the runner
|
||||
//! wires the actual consumption (delta runner can already be capped
|
||||
//! via batch_size — the budget is a higher-level abstraction).
|
||||
|
||||
/// Per-pass HTTP call budgets matching the spec table.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum PassKind {
|
||||
/// `poll_tick (unchanged watermark)` — 1 call (`getArtists` only).
|
||||
PollTick,
|
||||
/// `delta_sync (light)` — small targeted delta, 50 calls.
|
||||
DeltaLight,
|
||||
/// `delta_sync (count mismatch / tombstone)` — 200 calls,
|
||||
/// split across ticks if exhausted.
|
||||
DeltaMismatch,
|
||||
/// Initial sync — unlimited (only user cancel stops it).
|
||||
InitialSync,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct RequestBudget {
|
||||
pub kind: PassKind,
|
||||
/// `None` = unlimited.
|
||||
pub cap: Option<u32>,
|
||||
}
|
||||
|
||||
impl RequestBudget {
|
||||
pub const POLL_TICK_CAP: u32 = 1;
|
||||
pub const DELTA_LIGHT_CAP: u32 = 50;
|
||||
pub const DELTA_MISMATCH_CAP: u32 = 200;
|
||||
|
||||
pub fn for_pass(kind: PassKind) -> Self {
|
||||
let cap = match kind {
|
||||
PassKind::PollTick => Some(Self::POLL_TICK_CAP),
|
||||
PassKind::DeltaLight => Some(Self::DELTA_LIGHT_CAP),
|
||||
PassKind::DeltaMismatch => Some(Self::DELTA_MISMATCH_CAP),
|
||||
PassKind::InitialSync => None,
|
||||
};
|
||||
Self { kind, cap }
|
||||
}
|
||||
|
||||
pub fn is_unlimited(&self) -> bool {
|
||||
self.cap.is_none()
|
||||
}
|
||||
|
||||
/// Returns `true` when `used` requests still fit inside the cap.
|
||||
/// Unlimited budgets always return `true`.
|
||||
pub fn has_room(&self, used: u32) -> bool {
|
||||
match self.cap {
|
||||
None => true,
|
||||
Some(cap) => used < cap,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn poll_tick_cap_is_one_request() {
|
||||
let b = RequestBudget::for_pass(PassKind::PollTick);
|
||||
assert_eq!(b.cap, Some(1));
|
||||
assert!(b.has_room(0));
|
||||
assert!(!b.has_room(1));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delta_light_caps_at_fifty() {
|
||||
let b = RequestBudget::for_pass(PassKind::DeltaLight);
|
||||
assert_eq!(b.cap, Some(50));
|
||||
assert!(b.has_room(49));
|
||||
assert!(!b.has_room(50));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delta_mismatch_caps_at_two_hundred() {
|
||||
let b = RequestBudget::for_pass(PassKind::DeltaMismatch);
|
||||
assert_eq!(b.cap, Some(200));
|
||||
assert!(b.has_room(199));
|
||||
assert!(!b.has_room(200));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn initial_sync_is_unlimited() {
|
||||
let b = RequestBudget::for_pass(PassKind::InitialSync);
|
||||
assert!(b.is_unlimited());
|
||||
assert!(b.has_room(u32::MAX));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,680 @@
|
||||
//! C1 — Capability probe (spec §6.1 / §6.1.1).
|
||||
//!
|
||||
//! Drives the Subsonic client + an optional Navidrome native probe to
|
||||
//! populate `sync_state.capability_flags` before initial sync picks its
|
||||
//! ingest strategy (§6.3). PR-3a only writes flags from the responses;
|
||||
//! interpretation lives in PR-3b's `IngestStrategy` selector.
|
||||
|
||||
use psysonic_integration::navidrome::probe::native_bulk_available;
|
||||
use psysonic_integration::subsonic::{ServerInfo, SubsonicClient, SubsonicError};
|
||||
|
||||
/// Bitfield matching spec §6.1.1. `u32` storage so the `sync_state`
|
||||
/// table can keep it as a single integer column (`capability_flags
|
||||
/// INTEGER NOT NULL DEFAULT 0`).
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
pub struct CapabilityFlags(u32);
|
||||
|
||||
impl CapabilityFlags {
|
||||
/// N1 — Navidrome native `/api/song` paginated ingest.
|
||||
pub const NAVIDROME_NATIVE_BULK: u32 = 0x001;
|
||||
/// S1 — Subsonic `search3` empty-query bulk ingest.
|
||||
pub const SUBSONIC_SEARCH3_BULK: u32 = 0x002;
|
||||
/// `getScanStatus` available (Subsonic 1.15+); cheap-poll tier signal.
|
||||
pub const SCAN_STATUS_AVAILABLE: u32 = 0x004;
|
||||
/// Server advertises OpenSubsonic extensions (`isrc`, `played`,
|
||||
/// `bpm`, contributor arrays, …).
|
||||
pub const OPEN_SUBSONIC: u32 = 0x008;
|
||||
/// Track ids may shift across server re-indexing — sync engine must
|
||||
/// run the `track_id_history` remap pass (§6.9, P33). Always set
|
||||
/// for Navidrome.
|
||||
pub const UNSTABLE_TRACK_IDS: u32 = 0x010;
|
||||
/// S3 — `getIndexes` / `getMusicDirectory` available (file-tree
|
||||
/// fallback when ID3 endpoints are missing entirely).
|
||||
pub const FILE_TREE_BROWSE: u32 = 0x020;
|
||||
|
||||
pub fn new(bits: u32) -> Self {
|
||||
Self(bits)
|
||||
}
|
||||
|
||||
pub fn bits(self) -> u32 {
|
||||
self.0
|
||||
}
|
||||
|
||||
pub fn contains(self, flag: u32) -> bool {
|
||||
self.0 & flag == flag
|
||||
}
|
||||
|
||||
pub fn insert(&mut self, flag: u32) {
|
||||
self.0 |= flag;
|
||||
}
|
||||
|
||||
pub fn remove(&mut self, flag: u32) {
|
||||
self.0 &= !flag;
|
||||
}
|
||||
}
|
||||
|
||||
/// Optional input for `CapabilityProbe::run` — Navidrome native API
|
||||
/// needs its own bearer token (separate from the Subsonic salted-md5
|
||||
/// auth). When `None`, the `NavidromeNativeBulk` bit stays clear and
|
||||
/// sync falls back to Subsonic strategies.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct NavidromeProbeCredentials {
|
||||
pub server_url: String,
|
||||
pub bearer_token: String,
|
||||
}
|
||||
|
||||
/// Outcome of the capability probe — both the bitfield (stored in
|
||||
/// `sync_state.capability_flags`) and the raw `ServerInfo` envelope
|
||||
/// metadata (callers may want to log `serverVersion` etc.).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CapabilityProbeResult {
|
||||
pub flags: CapabilityFlags,
|
||||
pub server_info: ServerInfo,
|
||||
/// Server-reported track count from `getScanStatus.count`, when the
|
||||
/// server exposes it. `None` when `getScanStatus` is unavailable or
|
||||
/// reports no count. Persisted as the `server_track_count` watermark so
|
||||
/// the strategy selector can route large catalogs to S1 at IS-1 without
|
||||
/// first hitting N1's deep-offset wall (R7-15 Q4).
|
||||
pub server_track_count: Option<i64>,
|
||||
}
|
||||
|
||||
/// Run `CapabilityProbe::run` and persist the resulting flags +
|
||||
/// transition `sync_phase` from whatever it was to `probing` →
|
||||
/// `idle` (caller is responsible for advancing to `initial_sync` /
|
||||
/// `ready` once the appropriate runner starts).
|
||||
///
|
||||
/// PR-3d wires this in front of every initial / delta run so the
|
||||
/// stored `capability_flags` always reflects the current server.
|
||||
/// Returns the freshly resolved `(flags, server_info)` so callers
|
||||
/// can pick their `IngestStrategy` without re-reading SQLite.
|
||||
pub async fn probe_and_persist(
|
||||
store: &crate::store::LibraryStore,
|
||||
subsonic: &psysonic_integration::subsonic::SubsonicClient,
|
||||
navidrome: Option<&NavidromeProbeCredentials>,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
) -> Result<CapabilityProbeResult, psysonic_integration::subsonic::SubsonicError> {
|
||||
let sync_state = crate::repos::SyncStateRepository::new(store);
|
||||
sync_state
|
||||
.ensure(server_id, library_scope)
|
||||
.map_err(psysonic_integration::subsonic::SubsonicError::Transport)?;
|
||||
let phase_before = sync_state
|
||||
.get_sync_phase(server_id, library_scope)
|
||||
.map_err(psysonic_integration::subsonic::SubsonicError::Transport)?;
|
||||
sync_state
|
||||
.set_sync_phase(server_id, library_scope, "probing")
|
||||
.map_err(psysonic_integration::subsonic::SubsonicError::Transport)?;
|
||||
|
||||
let existing_flags = sync_state
|
||||
.get_capability_flags(server_id, library_scope)
|
||||
.map_err(psysonic_integration::subsonic::SubsonicError::Transport)?
|
||||
.unwrap_or(0);
|
||||
|
||||
let mut result = CapabilityProbe::run(subsonic, navidrome).await?;
|
||||
|
||||
// R7-15 Q3: a probe run without a Navidrome bearer can't test N1, so it
|
||||
// must not drop a previously-learned NavidromeNativeBulk capability — the
|
||||
// server still supports `/api/song`; only the token is missing this bind.
|
||||
// Token availability gates actual N1 use per run (see library_sync_start).
|
||||
if navidrome.is_none()
|
||||
&& existing_flags & CapabilityFlags::NAVIDROME_NATIVE_BULK != 0
|
||||
{
|
||||
result.flags.insert(CapabilityFlags::NAVIDROME_NATIVE_BULK);
|
||||
}
|
||||
|
||||
sync_state
|
||||
.set_capability_flags(server_id, library_scope, result.flags.bits())
|
||||
.map_err(psysonic_integration::subsonic::SubsonicError::Transport)?;
|
||||
// Refresh the track-count watermark only when the probe learned one — a
|
||||
// missing `getScanStatus.count` must not clobber a count from a prior run.
|
||||
if let Some(count) = result.server_track_count {
|
||||
sync_state
|
||||
.set_server_track_count(server_id, library_scope, count)
|
||||
.map_err(psysonic_integration::subsonic::SubsonicError::Transport)?;
|
||||
}
|
||||
sync_state
|
||||
.set_sync_phase(
|
||||
server_id,
|
||||
library_scope,
|
||||
match phase_before.as_deref() {
|
||||
// Re-bind on app restart must not clobber a finished index —
|
||||
// callers gate local search on `ready` (§9.3 / P8).
|
||||
Some("ready") => "ready",
|
||||
Some("initial_sync") => "initial_sync",
|
||||
Some("error") => "error",
|
||||
_ => {
|
||||
if sync_state
|
||||
.has_last_full_sync_at(server_id, library_scope)
|
||||
.map_err(psysonic_integration::subsonic::SubsonicError::Transport)?
|
||||
{
|
||||
"ready"
|
||||
} else {
|
||||
"idle"
|
||||
}
|
||||
}
|
||||
},
|
||||
)
|
||||
.map_err(psysonic_integration::subsonic::SubsonicError::Transport)?;
|
||||
|
||||
Ok(result)
|
||||
}
|
||||
|
||||
pub struct CapabilityProbe;
|
||||
|
||||
impl CapabilityProbe {
|
||||
/// Run the §6.1 probe chain. Returns the resolved flags plus the
|
||||
/// envelope metadata captured from the Subsonic ping.
|
||||
///
|
||||
/// The Subsonic ping is the only failure-blocking probe — if it
|
||||
/// returns `Err`, the server is unreachable / wrong creds / wrong
|
||||
/// URL, and no other capability can be determined. Every other
|
||||
/// probe is best-effort: it sets its flag on success and leaves it
|
||||
/// clear on any error.
|
||||
pub async fn run(
|
||||
subsonic: &SubsonicClient,
|
||||
navidrome: Option<&NavidromeProbeCredentials>,
|
||||
) -> Result<CapabilityProbeResult, SubsonicError> {
|
||||
let server_info = subsonic.server_info().await?;
|
||||
|
||||
let mut flags = CapabilityFlags::default();
|
||||
|
||||
if server_info.open_subsonic {
|
||||
flags.insert(CapabilityFlags::OPEN_SUBSONIC);
|
||||
}
|
||||
// Navidrome rebuilds its track id space on full re-scan; spec
|
||||
// §6.9 / P33 makes the remap pass mandatory for those servers.
|
||||
if matches!(server_info.server_type.as_deref(), Some("navidrome")) {
|
||||
flags.insert(CapabilityFlags::UNSTABLE_TRACK_IDS);
|
||||
}
|
||||
|
||||
// `search3` with songCount=1 is the cheapest way to confirm the
|
||||
// bulk-ingest endpoint is usable on this server (Navidrome
|
||||
// accepts empty query; some forks reject it).
|
||||
if subsonic.search3("", 1, 0, None).await.is_ok() {
|
||||
flags.insert(CapabilityFlags::SUBSONIC_SEARCH3_BULK);
|
||||
}
|
||||
|
||||
let mut server_track_count = None;
|
||||
if let Ok(scan) = subsonic.get_scan_status().await {
|
||||
flags.insert(CapabilityFlags::SCAN_STATUS_AVAILABLE);
|
||||
// Only a positive count is a usable watermark; a scan in progress
|
||||
// can report 0, which we treat as "unknown" rather than "empty".
|
||||
server_track_count = scan.count.filter(|&c| c > 0);
|
||||
}
|
||||
|
||||
if subsonic.get_indexes(None, None).await.is_ok() {
|
||||
flags.insert(CapabilityFlags::FILE_TREE_BROWSE);
|
||||
}
|
||||
|
||||
if let Some(creds) = navidrome {
|
||||
match native_bulk_available(&creds.server_url, &creds.bearer_token).await {
|
||||
Ok(true) => flags.insert(CapabilityFlags::NAVIDROME_NATIVE_BULK),
|
||||
Ok(false) => {}
|
||||
Err(_) => {
|
||||
// Probe transport failed but Subsonic ping worked —
|
||||
// assume the native endpoint is unavailable for this
|
||||
// setup and let sync fall back to S1/S2.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Ok(CapabilityProbeResult {
|
||||
flags,
|
||||
server_info,
|
||||
server_track_count,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use psysonic_integration::subsonic::{SubsonicClient, SubsonicCredentials};
|
||||
use serde_json::json;
|
||||
use wiremock::matchers::{header, method as wm_method, path as wm_path};
|
||||
use wiremock::{Mock, MockServer, ResponseTemplate};
|
||||
|
||||
// ── CapabilityFlags bitfield ─────────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn capability_flags_contains_respects_individual_bits() {
|
||||
let mut f = CapabilityFlags::default();
|
||||
assert!(!f.contains(CapabilityFlags::OPEN_SUBSONIC));
|
||||
f.insert(CapabilityFlags::OPEN_SUBSONIC);
|
||||
assert!(f.contains(CapabilityFlags::OPEN_SUBSONIC));
|
||||
assert!(!f.contains(CapabilityFlags::NAVIDROME_NATIVE_BULK));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn capability_flags_insert_is_idempotent() {
|
||||
let mut f = CapabilityFlags::default();
|
||||
f.insert(CapabilityFlags::SUBSONIC_SEARCH3_BULK);
|
||||
let after_first = f.bits();
|
||||
f.insert(CapabilityFlags::SUBSONIC_SEARCH3_BULK);
|
||||
assert_eq!(f.bits(), after_first);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn capability_flags_remove_clears_only_the_named_bit() {
|
||||
let mut f = CapabilityFlags::new(
|
||||
CapabilityFlags::OPEN_SUBSONIC | CapabilityFlags::UNSTABLE_TRACK_IDS,
|
||||
);
|
||||
f.remove(CapabilityFlags::OPEN_SUBSONIC);
|
||||
assert!(!f.contains(CapabilityFlags::OPEN_SUBSONIC));
|
||||
assert!(f.contains(CapabilityFlags::UNSTABLE_TRACK_IDS));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn capability_flags_bit_values_match_spec_table() {
|
||||
// Spec §6.1.1 hex values — pin the wire format so future
|
||||
// schema-migration writers don't shift them silently.
|
||||
assert_eq!(CapabilityFlags::NAVIDROME_NATIVE_BULK, 0x001);
|
||||
assert_eq!(CapabilityFlags::SUBSONIC_SEARCH3_BULK, 0x002);
|
||||
assert_eq!(CapabilityFlags::SCAN_STATUS_AVAILABLE, 0x004);
|
||||
assert_eq!(CapabilityFlags::OPEN_SUBSONIC, 0x008);
|
||||
assert_eq!(CapabilityFlags::UNSTABLE_TRACK_IDS, 0x010);
|
||||
assert_eq!(CapabilityFlags::FILE_TREE_BROWSE, 0x020);
|
||||
}
|
||||
|
||||
// ── CapabilityProbe wiremock harness ─────────────────────────────────
|
||||
|
||||
fn test_subsonic_client(uri: &str) -> SubsonicClient {
|
||||
SubsonicClient::with_static_credentials(
|
||||
uri,
|
||||
SubsonicCredentials::with_static("user", "tok", "salt"),
|
||||
reqwest::Client::new(),
|
||||
)
|
||||
}
|
||||
|
||||
fn ok_envelope(body_key: &str, body: serde_json::Value) -> serde_json::Value {
|
||||
json!({
|
||||
"subsonic-response": {
|
||||
"status": "ok",
|
||||
"version": "1.16.1",
|
||||
body_key: body,
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
async fn mount_subsonic_full_navidrome(server: &MockServer) {
|
||||
// ping → navidrome + openSubsonic
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/ping.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": {
|
||||
"status": "ok",
|
||||
"version": "1.16.1",
|
||||
"type": "navidrome",
|
||||
"serverVersion": "0.55.2",
|
||||
"openSubsonic": true
|
||||
}
|
||||
})))
|
||||
.mount(server)
|
||||
.await;
|
||||
// search3 empty query
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/search3.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(ok_envelope(
|
||||
"searchResult3",
|
||||
json!({ "song": [{ "id": "x", "title": "y" }] }),
|
||||
)))
|
||||
.mount(server)
|
||||
.await;
|
||||
// getScanStatus
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getScanStatus.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(ok_envelope(
|
||||
"scanStatus",
|
||||
json!({ "scanning": false }),
|
||||
)))
|
||||
.mount(server)
|
||||
.await;
|
||||
// getIndexes
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getIndexes.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(ok_envelope(
|
||||
"indexes",
|
||||
json!({ "lastModified": 0, "ignoredArticles": "", "index": [] }),
|
||||
)))
|
||||
.mount(server)
|
||||
.await;
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_sets_all_subsonic_bits_on_a_fully_capable_navidrome_server() {
|
||||
let server = MockServer::start().await;
|
||||
mount_subsonic_full_navidrome(&server).await;
|
||||
|
||||
let result = CapabilityProbe::run(&test_subsonic_client(&server.uri()), None)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(result.flags.contains(CapabilityFlags::SUBSONIC_SEARCH3_BULK));
|
||||
assert!(result.flags.contains(CapabilityFlags::SCAN_STATUS_AVAILABLE));
|
||||
assert!(result.flags.contains(CapabilityFlags::FILE_TREE_BROWSE));
|
||||
assert!(result.flags.contains(CapabilityFlags::OPEN_SUBSONIC));
|
||||
assert!(result.flags.contains(CapabilityFlags::UNSTABLE_TRACK_IDS));
|
||||
// No navidrome probe creds passed → N1 stays clear.
|
||||
assert!(!result.flags.contains(CapabilityFlags::NAVIDROME_NATIVE_BULK));
|
||||
assert_eq!(result.server_info.server_type.as_deref(), Some("navidrome"));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_returns_err_when_subsonic_ping_fails() {
|
||||
let server = MockServer::start().await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/ping.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": {
|
||||
"status": "failed",
|
||||
"error": { "code": 40, "message": "Wrong credentials" }
|
||||
}
|
||||
})))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
let err = CapabilityProbe::run(&test_subsonic_client(&server.uri()), None)
|
||||
.await
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, SubsonicError::Api { code: 40, .. }));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_keeps_optional_bits_clear_when_their_endpoint_fails() {
|
||||
// Minimal Subsonic-like server: ping ok, search3 ok, but
|
||||
// scanStatus + getIndexes 4xx. UnstableTrackIds + OpenSubsonic
|
||||
// stay clear because the ping envelope omits them.
|
||||
let server = MockServer::start().await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/ping.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": { "status": "ok", "version": "1.13" }
|
||||
})))
|
||||
.mount(&server)
|
||||
.await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/search3.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(ok_envelope(
|
||||
"searchResult3",
|
||||
json!({}),
|
||||
)))
|
||||
.mount(&server)
|
||||
.await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getScanStatus.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": {
|
||||
"status": "failed",
|
||||
"error": { "code": 30, "message": "Method not available" }
|
||||
}
|
||||
})))
|
||||
.mount(&server)
|
||||
.await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getIndexes.view"))
|
||||
.respond_with(ResponseTemplate::new(500))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
let result = CapabilityProbe::run(&test_subsonic_client(&server.uri()), None)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(result.flags.contains(CapabilityFlags::SUBSONIC_SEARCH3_BULK));
|
||||
assert!(!result.flags.contains(CapabilityFlags::SCAN_STATUS_AVAILABLE));
|
||||
assert!(!result.flags.contains(CapabilityFlags::FILE_TREE_BROWSE));
|
||||
assert!(!result.flags.contains(CapabilityFlags::OPEN_SUBSONIC));
|
||||
assert!(!result.flags.contains(CapabilityFlags::UNSTABLE_TRACK_IDS));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_sets_navidrome_native_bulk_when_credentials_succeed() {
|
||||
let server = MockServer::start().await;
|
||||
mount_subsonic_full_navidrome(&server).await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/api/song"))
|
||||
.and(header("X-ND-Authorization", "Bearer nd-tok"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!([])))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
let nav = NavidromeProbeCredentials {
|
||||
server_url: server.uri(),
|
||||
bearer_token: "nd-tok".into(),
|
||||
};
|
||||
let result = CapabilityProbe::run(&test_subsonic_client(&server.uri()), Some(&nav))
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(result.flags.contains(CapabilityFlags::NAVIDROME_NATIVE_BULK));
|
||||
}
|
||||
|
||||
// ── probe_and_persist round-trip ──────────────────────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_and_persist_writes_flags_and_resets_phase_to_idle() {
|
||||
use crate::repos::SyncStateRepository;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
let server = MockServer::start().await;
|
||||
mount_subsonic_full_navidrome(&server).await;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let result = super::probe_and_persist(
|
||||
&store,
|
||||
&test_subsonic_client(&server.uri()),
|
||||
None,
|
||||
"s1",
|
||||
"",
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
let flags = sync_state.get_capability_flags("s1", "").unwrap().unwrap();
|
||||
assert_eq!(flags, result.flags.bits());
|
||||
assert!(flags & CapabilityFlags::OPEN_SUBSONIC != 0);
|
||||
assert!(flags & CapabilityFlags::UNSTABLE_TRACK_IDS != 0);
|
||||
|
||||
// Fresh server ends at `idle` so the caller can transition to
|
||||
// `initial_sync` / `ready` based on whether a sync is needed.
|
||||
assert_eq!(
|
||||
sync_state.get_sync_phase("s1", "").unwrap().as_deref(),
|
||||
Some("idle")
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_and_persist_preserves_ready_phase_on_rebind() {
|
||||
use crate::repos::SyncStateRepository;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
let server = MockServer::start().await;
|
||||
mount_subsonic_full_navidrome(&server).await;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
sync_state.ensure("s1", "").unwrap();
|
||||
sync_state.set_sync_phase("s1", "", "ready").unwrap();
|
||||
|
||||
super::probe_and_persist(
|
||||
&store,
|
||||
&test_subsonic_client(&server.uri()),
|
||||
None,
|
||||
"s1",
|
||||
"",
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(
|
||||
sync_state.get_sync_phase("s1", "").unwrap().as_deref(),
|
||||
Some("ready")
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_and_persist_promotes_idle_to_ready_when_full_sync_stamped() {
|
||||
use crate::repos::SyncStateRepository;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
let server = MockServer::start().await;
|
||||
mount_subsonic_full_navidrome(&server).await;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
sync_state.ensure("s1", "").unwrap();
|
||||
sync_state.set_sync_phase("s1", "", "idle").unwrap();
|
||||
sync_state
|
||||
.set_last_full_sync_at("s1", "", 1_716_000_000_000)
|
||||
.unwrap();
|
||||
|
||||
super::probe_and_persist(
|
||||
&store,
|
||||
&test_subsonic_client(&server.uri()),
|
||||
None,
|
||||
"s1",
|
||||
"",
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(
|
||||
sync_state.get_sync_phase("s1", "").unwrap().as_deref(),
|
||||
Some("ready")
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_captures_and_persists_scan_status_track_count() {
|
||||
use crate::repos::SyncStateRepository;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
let server = MockServer::start().await;
|
||||
// ping + search3 + getIndexes from the shared helper, then override
|
||||
// getScanStatus with a populated `count` (large library).
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/ping.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": { "status": "ok", "version": "1.16.1", "type": "navidrome" }
|
||||
})))
|
||||
.mount(&server)
|
||||
.await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/search3.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(ok_envelope(
|
||||
"searchResult3",
|
||||
json!({ "song": [{ "id": "x", "title": "y" }] }),
|
||||
)))
|
||||
.mount(&server)
|
||||
.await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getScanStatus.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(ok_envelope(
|
||||
"scanStatus",
|
||||
json!({ "scanning": false, "count": 170_000 }),
|
||||
)))
|
||||
.mount(&server)
|
||||
.await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getIndexes.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(ok_envelope(
|
||||
"indexes",
|
||||
json!({ "lastModified": 0, "ignoredArticles": "", "index": [] }),
|
||||
)))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let result =
|
||||
super::probe_and_persist(&store, &test_subsonic_client(&server.uri()), None, "s1", "")
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(result.server_track_count, Some(170_000));
|
||||
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
assert_eq!(
|
||||
sync_state.get_server_track_count("s1", "").unwrap(),
|
||||
Some(170_000)
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_preserves_navidrome_native_bulk_when_no_token_supplied() {
|
||||
use crate::repos::SyncStateRepository;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
// A prior bind (with a working bearer) already learned N1.
|
||||
sync_state
|
||||
.set_capability_flags("s1", "", CapabilityFlags::NAVIDROME_NATIVE_BULK)
|
||||
.unwrap();
|
||||
|
||||
let server = MockServer::start().await;
|
||||
mount_subsonic_full_navidrome(&server).await;
|
||||
|
||||
// Re-probe without a Navidrome token (transient /auth/login failure).
|
||||
// R7-15 Q3: the server still supports /api/song — the flag must stay.
|
||||
let result = super::probe_and_persist(
|
||||
&store,
|
||||
&test_subsonic_client(&server.uri()),
|
||||
None,
|
||||
"s1",
|
||||
"",
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(
|
||||
result.flags.contains(CapabilityFlags::NAVIDROME_NATIVE_BULK),
|
||||
"result must keep the previously-learned N1 capability"
|
||||
);
|
||||
let persisted = sync_state.get_capability_flags("s1", "").unwrap().unwrap();
|
||||
assert!(
|
||||
persisted & CapabilityFlags::NAVIDROME_NATIVE_BULK != 0,
|
||||
"persisted flags must keep N1 across a token-less probe"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_does_not_clobber_track_count_when_scan_status_omits_it() {
|
||||
use crate::repos::SyncStateRepository;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
// A prior run already learned the count.
|
||||
sync_state.set_server_track_count("s1", "", 52_000).unwrap();
|
||||
|
||||
let server = MockServer::start().await;
|
||||
mount_subsonic_full_navidrome(&server).await; // scanStatus has no count
|
||||
|
||||
let result =
|
||||
super::probe_and_persist(&store, &test_subsonic_client(&server.uri()), None, "s1", "")
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(result.server_track_count, None);
|
||||
// Watermark from the prior run survives the count-less probe.
|
||||
assert_eq!(
|
||||
sync_state.get_server_track_count("s1", "").unwrap(),
|
||||
Some(52_000)
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_leaves_navidrome_native_bulk_clear_when_endpoint_404s() {
|
||||
let server = MockServer::start().await;
|
||||
mount_subsonic_full_navidrome(&server).await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/api/song"))
|
||||
.respond_with(ResponseTemplate::new(404))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
let nav = NavidromeProbeCredentials {
|
||||
server_url: server.uri(),
|
||||
bearer_token: "nd-tok".into(),
|
||||
};
|
||||
let result = CapabilityProbe::run(&test_subsonic_client(&server.uri()), Some(&nav))
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(!result.flags.contains(CapabilityFlags::NAVIDROME_NATIVE_BULK));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,157 @@
|
||||
//! Cursor shape persisted in `sync_state.initial_sync_cursor_json`.
|
||||
//! Resume after a restart, kill, or app crash deserializes this value
|
||||
//! and tells the runner where to pick up. Strategy is recorded so a
|
||||
//! cap-flag change between runs is detected and the stale cursor is
|
||||
//! reset — the runner restarts ingest under the newly-selected strategy
|
||||
//! rather than resuming the wrong loop.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use super::strategy::IngestStrategy;
|
||||
|
||||
/// Top-level cursor. `phase` advances `Ingest → ArtistPass → Watermarks
|
||||
/// → Done`; each phase only reads the fields it owns.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
|
||||
pub struct InitialSyncCursor {
|
||||
/// Ingest strategy in flight — stored as the tag string so the JSON
|
||||
/// is human-readable on disk.
|
||||
pub strategy: String,
|
||||
pub phase: CursorPhase,
|
||||
/// Scope this run operates on (Navidrome `library_id` / Subsonic
|
||||
/// `musicFolderId`). `None` means "all libraries on this server".
|
||||
#[serde(default)]
|
||||
pub library_scope: Option<String>,
|
||||
/// Tracks ingested across the entire run so far — informational,
|
||||
/// matches the §6 progress event payload.
|
||||
#[serde(default)]
|
||||
pub ingested_count: u32,
|
||||
/// Per-strategy offset state. Discriminated by `strategy` so a
|
||||
/// future field add doesn't break old cursors.
|
||||
#[serde(default)]
|
||||
pub strategy_state: StrategyState,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum CursorPhase {
|
||||
Ingest,
|
||||
ArtistPass,
|
||||
Watermarks,
|
||||
Done,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, Default)]
|
||||
#[serde(tag = "kind", rename_all = "snake_case")]
|
||||
pub enum StrategyState {
|
||||
/// N1 / S1 — single linear offset.
|
||||
LinearOffset { offset: u32 },
|
||||
/// S2 — outer album-list offset plus optional in-flight album we
|
||||
/// were halfway through when interrupted.
|
||||
AlbumCrawl {
|
||||
album_offset: u32,
|
||||
#[serde(default)]
|
||||
current_album_id: Option<String>,
|
||||
},
|
||||
/// Fresh cursor with no progress yet.
|
||||
#[default]
|
||||
Empty,
|
||||
}
|
||||
|
||||
impl InitialSyncCursor {
|
||||
pub fn fresh(strategy: IngestStrategy, library_scope: Option<String>) -> Self {
|
||||
let strategy_state = match strategy {
|
||||
IngestStrategy::N1 | IngestStrategy::S1 => StrategyState::LinearOffset { offset: 0 },
|
||||
IngestStrategy::S2 => StrategyState::AlbumCrawl {
|
||||
album_offset: 0,
|
||||
current_album_id: None,
|
||||
},
|
||||
IngestStrategy::S3 => StrategyState::Empty,
|
||||
};
|
||||
Self {
|
||||
strategy: strategy.as_tag().to_string(),
|
||||
phase: CursorPhase::Ingest,
|
||||
library_scope,
|
||||
ingested_count: 0,
|
||||
strategy_state,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn strategy_tag(&self) -> &str {
|
||||
&self.strategy
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn fresh_n1_starts_at_offset_zero() {
|
||||
let c = InitialSyncCursor::fresh(IngestStrategy::N1, None);
|
||||
assert_eq!(c.phase, CursorPhase::Ingest);
|
||||
assert_eq!(c.ingested_count, 0);
|
||||
match c.strategy_state {
|
||||
StrategyState::LinearOffset { offset } => assert_eq!(offset, 0),
|
||||
other => panic!("expected LinearOffset, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fresh_s2_uses_album_crawl_state() {
|
||||
let c = InitialSyncCursor::fresh(IngestStrategy::S2, Some("lib-1".into()));
|
||||
assert_eq!(c.library_scope.as_deref(), Some("lib-1"));
|
||||
match c.strategy_state {
|
||||
StrategyState::AlbumCrawl { album_offset, current_album_id } => {
|
||||
assert_eq!(album_offset, 0);
|
||||
assert!(current_album_id.is_none());
|
||||
}
|
||||
other => panic!("expected AlbumCrawl, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cursor_roundtrips_through_json() {
|
||||
// The cursor lives as TEXT in `sync_state.initial_sync_cursor_json`;
|
||||
// serde must keep the round trip stable for resume to work.
|
||||
let c = InitialSyncCursor {
|
||||
strategy: "n1".into(),
|
||||
phase: CursorPhase::Ingest,
|
||||
library_scope: Some("lib-1".into()),
|
||||
ingested_count: 2500,
|
||||
strategy_state: StrategyState::LinearOffset { offset: 2500 },
|
||||
};
|
||||
let json = serde_json::to_value(&c).unwrap();
|
||||
let back: InitialSyncCursor = serde_json::from_value(json.clone()).unwrap();
|
||||
assert_eq!(c, back);
|
||||
|
||||
// strategy_state internally tagged with `"kind"`.
|
||||
assert_eq!(
|
||||
json.get("strategy_state").and_then(|s| s.get("kind")),
|
||||
Some(&json!("linear_offset"))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cursor_deserialize_tolerates_omitted_defaults() {
|
||||
// Minimal cursor produced by a much older client must still
|
||||
// deserialize as a "fresh ingest" state.
|
||||
let raw = json!({
|
||||
"strategy": "s1",
|
||||
"phase": "ingest"
|
||||
});
|
||||
let c: InitialSyncCursor = serde_json::from_value(raw).unwrap();
|
||||
assert_eq!(c.ingested_count, 0);
|
||||
assert_eq!(c.library_scope, None);
|
||||
assert!(matches!(c.strategy_state, StrategyState::Empty));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_object_does_not_parse_as_cursor() {
|
||||
// sync_state.initial_sync_cursor_json default is `'{}'`; that
|
||||
// value is not a valid cursor (no strategy / phase). Runner
|
||||
// must treat it as "no cursor → start fresh".
|
||||
let raw = json!({});
|
||||
assert!(serde_json::from_value::<InitialSyncCursor>(raw).is_err());
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,149 @@
|
||||
//! Errors surfaced by the sync orchestrator. Designed so callers
|
||||
//! (PR-5 Tauri command surface) can pattern-match on the variant for
|
||||
//! UI affordances — `Cancelled` is silent, `Transport`/`HttpStatus`
|
||||
//! retry, `Subsonic { code: 70, .. }` flows into the tombstone path,
|
||||
//! and `StrategyUnsupported` surfaces a Settings hint.
|
||||
|
||||
use std::fmt;
|
||||
|
||||
use psysonic_integration::subsonic::SubsonicError;
|
||||
|
||||
#[derive(Debug)]
|
||||
pub enum SyncError {
|
||||
/// Network / TLS / DNS failure surfaced by the Subsonic or
|
||||
/// Navidrome HTTP client. Retryable per §6.8.
|
||||
Transport(String),
|
||||
|
||||
/// Subsonic-level error after the envelope parsed cleanly. The
|
||||
/// dedicated `NotFound` (error code 70) is kept inline rather
|
||||
/// than collapsed into a string so the tombstone path can match.
|
||||
Subsonic { code: i32, message: String },
|
||||
|
||||
/// Subsonic returned error code 70 — track missing from the
|
||||
/// server. Tombstone reconciler matches on this variant directly.
|
||||
NotFound,
|
||||
|
||||
/// Navidrome native REST returned a non-success status; carries
|
||||
/// the flattened HTTP error string from `nd_err`.
|
||||
Navidrome(String),
|
||||
|
||||
/// Persistence layer (SQLite) failure.
|
||||
Storage(String),
|
||||
|
||||
/// Strategy is enumerated but not implemented for v1
|
||||
/// (currently only `S3`).
|
||||
StrategyUnsupported {
|
||||
strategy: &'static str,
|
||||
},
|
||||
|
||||
/// Cancellation token tripped — caller asked us to abort.
|
||||
/// Cursor stays where it was so the next run resumes the batch.
|
||||
Cancelled,
|
||||
|
||||
/// Cursor JSON in `sync_state` is from an incompatible strategy
|
||||
/// (e.g. switching from N1 to S2 mid-sync). Caller should clear
|
||||
/// the cursor and restart initial sync.
|
||||
CursorIncompatible {
|
||||
expected: &'static str,
|
||||
actual: String,
|
||||
},
|
||||
}
|
||||
|
||||
impl fmt::Display for SyncError {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
match self {
|
||||
Self::Transport(m) => write!(f, "sync transport: {m}"),
|
||||
Self::Subsonic { code, message } => write!(f, "subsonic {code}: {message}"),
|
||||
Self::NotFound => write!(f, "subsonic: not found (code 70)"),
|
||||
Self::Navidrome(m) => write!(f, "navidrome: {m}"),
|
||||
Self::Storage(m) => write!(f, "storage: {m}"),
|
||||
Self::StrategyUnsupported { strategy } => {
|
||||
write!(f, "ingest strategy not supported: {strategy}")
|
||||
}
|
||||
Self::Cancelled => write!(f, "sync cancelled"),
|
||||
Self::CursorIncompatible { expected, actual } => write!(
|
||||
f,
|
||||
"sync cursor strategy mismatch: cursor says `{actual}`, runner is `{expected}`"
|
||||
),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for SyncError {}
|
||||
|
||||
impl SyncError {
|
||||
/// Parsed HTTP status when this is a Navidrome native REST failure
|
||||
/// shaped like `HTTP 500` or `HTTP 500: body`.
|
||||
pub fn navidrome_http_status(&self) -> Option<u16> {
|
||||
let SyncError::Navidrome(msg) = self else {
|
||||
return None;
|
||||
};
|
||||
let rest = msg.strip_prefix("HTTP ")?.split(':').next()?.trim();
|
||||
rest.split_whitespace().next()?.parse().ok()
|
||||
}
|
||||
}
|
||||
|
||||
impl From<SubsonicError> for SyncError {
|
||||
fn from(e: SubsonicError) -> Self {
|
||||
match e {
|
||||
SubsonicError::Transport(m) => Self::Transport(m),
|
||||
SubsonicError::HttpStatus(s) => Self::Transport(format!("http {s}")),
|
||||
SubsonicError::Api { code, message } => Self::Subsonic { code, message },
|
||||
SubsonicError::NotFound => Self::NotFound,
|
||||
SubsonicError::Decode(m) => Self::Transport(format!("decode: {m}")),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn subsonic_not_found_maps_to_sync_not_found() {
|
||||
let e: SyncError = SubsonicError::NotFound.into();
|
||||
assert!(matches!(e, SyncError::NotFound));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn subsonic_api_error_carries_code_through() {
|
||||
let e: SyncError = SubsonicError::Api { code: 40, message: "bad creds".into() }.into();
|
||||
match e {
|
||||
SyncError::Subsonic { code, message } => {
|
||||
assert_eq!(code, 40);
|
||||
assert!(message.contains("bad creds"));
|
||||
}
|
||||
other => panic!("expected Subsonic, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn http_status_collapses_into_transport() {
|
||||
let e: SyncError = SubsonicError::HttpStatus(reqwest::StatusCode::SERVICE_UNAVAILABLE).into();
|
||||
assert!(matches!(e, SyncError::Transport(ref m) if m.contains("503")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn navidrome_http_status_parses_status_line() {
|
||||
let e = SyncError::Navidrome("HTTP 500".into());
|
||||
assert_eq!(e.navidrome_http_status(), Some(500));
|
||||
let with_body = SyncError::Navidrome("HTTP 503: upstream timeout".into());
|
||||
assert_eq!(with_body.navidrome_http_status(), Some(503));
|
||||
let with_reason = SyncError::Navidrome("HTTP 500 Internal Server Error".into());
|
||||
assert_eq!(with_reason.navidrome_http_status(), Some(500));
|
||||
assert_eq!(
|
||||
SyncError::Transport("http 500".into()).navidrome_http_status(),
|
||||
None
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cursor_incompatible_renders_both_strategies() {
|
||||
let s = SyncError::CursorIncompatible {
|
||||
expected: "n1",
|
||||
actual: "s2".into(),
|
||||
}
|
||||
.to_string();
|
||||
assert!(s.contains("n1") && s.contains("s2"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,226 @@
|
||||
//! Concurrent fetch helpers for initial-sync ingest (C11 parallelism budget).
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::future::Future;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use psysonic_integration::subsonic::{Album, SubsonicClient};
|
||||
use serde_json::Value;
|
||||
use tokio::sync::Semaphore;
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use super::backoff::{jitter_salt, with_jitter, Backoff};
|
||||
use super::bandwidth::ParallelismBudget;
|
||||
use super::error::SyncError;
|
||||
|
||||
const MAX_ATTEMPTS_PER_BATCH: u32 = 5;
|
||||
|
||||
pub fn check_cancel_flag(cancel: &Option<Arc<std::sync::atomic::AtomicBool>>) -> Result<(), SyncError> {
|
||||
if cancel.as_ref().is_some_and(|f| f.load(Ordering::SeqCst)) {
|
||||
return Err(SyncError::Cancelled);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn is_retryable(e: &SyncError) -> bool {
|
||||
matches!(e, SyncError::Transport(_) | SyncError::Navidrome(_))
|
||||
}
|
||||
|
||||
/// How many linear pages to keep in flight (`1` = sequential).
|
||||
pub fn linear_prefetch_depth(budget: &ParallelismBudget) -> usize {
|
||||
if budget.bulk_paused() {
|
||||
return 1;
|
||||
}
|
||||
budget.max_concurrent.max(1) as usize
|
||||
}
|
||||
|
||||
pub async fn sleep_request_gap(budget: &ParallelismBudget, sleep_enabled: bool) {
|
||||
if sleep_enabled && budget.min_request_gap_ms > 0 {
|
||||
tokio::time::sleep(Duration::from_millis(budget.min_request_gap_ms as u64)).await;
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn wait_while_bulk_paused(
|
||||
budget: &ParallelismBudget,
|
||||
sleep_enabled: bool,
|
||||
mut check_cancel: impl FnMut() -> Result<(), SyncError>,
|
||||
) -> Result<(), SyncError> {
|
||||
while budget.bulk_paused() {
|
||||
check_cancel()?;
|
||||
if sleep_enabled {
|
||||
tokio::time::sleep(Duration::from_millis(500)).await;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Standalone retry loop for spawned prefetch tasks (no `InitialSyncRunner` ref).
|
||||
pub async fn retry_fetch<T, F, Fut, E>(
|
||||
sleep_enabled: bool,
|
||||
mut check_cancel: impl FnMut() -> Result<(), SyncError>,
|
||||
mut build: F,
|
||||
map_err: impl Fn(E) -> SyncError,
|
||||
) -> Result<T, SyncError>
|
||||
where
|
||||
F: FnMut() -> Fut,
|
||||
Fut: Future<Output = Result<T, E>>,
|
||||
{
|
||||
let mut backoff = Backoff::default();
|
||||
let mut attempt = 0u32;
|
||||
loop {
|
||||
check_cancel()?;
|
||||
attempt += 1;
|
||||
match build().await {
|
||||
Ok(v) => return Ok(v),
|
||||
Err(e) => {
|
||||
let mapped = map_err(e);
|
||||
if !is_retryable(&mapped) || attempt >= MAX_ATTEMPTS_PER_BATCH {
|
||||
return Err(mapped);
|
||||
}
|
||||
let delay = backoff.next_delay();
|
||||
let jittered = with_jitter(delay, jitter_salt(attempt));
|
||||
if sleep_enabled && !jittered.is_zero() {
|
||||
tokio::time::sleep(jittered).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Ordered in-flight page queue for N1/S1 linear ingest.
|
||||
pub struct LinearPrefetchQueue<T> {
|
||||
depth: usize,
|
||||
batch_size: u32,
|
||||
next_enqueue_offset: u32,
|
||||
exhausted: bool,
|
||||
inflight: BTreeMap<u32, JoinHandle<Result<T, SyncError>>>,
|
||||
}
|
||||
|
||||
impl<T: Send + 'static> LinearPrefetchQueue<T> {
|
||||
pub fn new(budget: &ParallelismBudget, batch_size: u32, start_offset: u32) -> Self {
|
||||
Self {
|
||||
depth: linear_prefetch_depth(budget),
|
||||
batch_size,
|
||||
next_enqueue_offset: start_offset,
|
||||
exhausted: false,
|
||||
inflight: BTreeMap::new(),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn mark_exhausted(&mut self) {
|
||||
self.exhausted = true;
|
||||
}
|
||||
|
||||
pub fn pump<F>(
|
||||
&mut self,
|
||||
mut check_cancel: impl FnMut() -> Result<(), SyncError>,
|
||||
mut spawn: F,
|
||||
) -> Result<(), SyncError>
|
||||
where
|
||||
F: FnMut(u32) -> JoinHandle<Result<T, SyncError>>,
|
||||
{
|
||||
while !self.exhausted && self.inflight.len() < self.depth {
|
||||
check_cancel()?;
|
||||
let offset = self.next_enqueue_offset;
|
||||
self.next_enqueue_offset = offset.saturating_add(self.batch_size);
|
||||
let handle = spawn(offset);
|
||||
self.inflight.insert(offset, handle);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub async fn take_at(
|
||||
&mut self,
|
||||
offset: u32,
|
||||
mut check_cancel: impl FnMut() -> Result<(), SyncError>,
|
||||
) -> Result<Option<T>, SyncError> {
|
||||
check_cancel()?;
|
||||
let Some(handle) = self.inflight.remove(&offset) else {
|
||||
return Ok(None);
|
||||
};
|
||||
let value = handle
|
||||
.await
|
||||
.map_err(|e| SyncError::Transport(format!("prefetch join: {e}")))??;
|
||||
Ok(Some(value))
|
||||
}
|
||||
}
|
||||
|
||||
/// Fetch `getAlbum` bodies for a page with at most `budget.max_concurrent`
|
||||
/// requests in flight. Results preserve input order.
|
||||
#[derive(Clone)]
|
||||
pub struct ParallelAlbumFetchOpts {
|
||||
pub budget: ParallelismBudget,
|
||||
pub sleep_enabled: bool,
|
||||
pub cancel: Option<Arc<AtomicBool>>,
|
||||
}
|
||||
|
||||
pub async fn fetch_albums_parallel(
|
||||
subsonic: &SubsonicClient,
|
||||
album_ids: &[String],
|
||||
opts: ParallelAlbumFetchOpts,
|
||||
) -> Result<Vec<(Album, Value)>, SyncError> {
|
||||
if album_ids.is_empty() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
wait_while_bulk_paused(&opts.budget, opts.sleep_enabled, || check_cancel_flag(&opts.cancel)).await?;
|
||||
let max = opts.budget.max_concurrent.max(1) as usize;
|
||||
let client = subsonic.clone();
|
||||
let sem = Arc::new(Semaphore::new(max));
|
||||
let mut handles: Vec<JoinHandle<Result<(Album, Value), SyncError>>> =
|
||||
Vec::with_capacity(album_ids.len());
|
||||
|
||||
for id in album_ids {
|
||||
check_cancel_flag(&opts.cancel)?;
|
||||
sleep_request_gap(&opts.budget, opts.sleep_enabled).await;
|
||||
let permit = sem
|
||||
.clone()
|
||||
.acquire_owned()
|
||||
.await
|
||||
.map_err(|_| SyncError::Transport("parallel fetch semaphore closed".into()))?;
|
||||
let client = client.clone();
|
||||
let id = id.clone();
|
||||
let cancel = opts.cancel.clone();
|
||||
let sleep_enabled = opts.sleep_enabled;
|
||||
handles.push(tokio::spawn(async move {
|
||||
let _permit = permit;
|
||||
retry_fetch(
|
||||
sleep_enabled,
|
||||
|| check_cancel_flag(&cancel),
|
||||
|| async {
|
||||
client
|
||||
.get_album_with_raw(&id)
|
||||
.await
|
||||
.map_err(SyncError::from)
|
||||
},
|
||||
|e| e,
|
||||
)
|
||||
.await
|
||||
}));
|
||||
}
|
||||
|
||||
let mut out = Vec::with_capacity(handles.len());
|
||||
for handle in handles {
|
||||
check_cancel_flag(&opts.cancel)?;
|
||||
out.push(
|
||||
handle
|
||||
.await
|
||||
.map_err(|e| SyncError::Transport(format!("parallel album fetch join: {e}")))??
|
||||
);
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn linear_prefetch_depth_respects_budget() {
|
||||
let idle = ParallelismBudget::resolve(super::super::bandwidth::PlaybackHint::Idle);
|
||||
assert_eq!(linear_prefetch_depth(&idle), 4);
|
||||
let playing = ParallelismBudget::resolve(super::super::bandwidth::PlaybackHint::Playing);
|
||||
assert_eq!(linear_prefetch_depth(&playing), 1);
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,303 @@
|
||||
//! Subsonic / Navidrome song JSON → `TrackRow`. PR-3b's ingest paths
|
||||
//! all feed the same upsert API, so the projection happens here once.
|
||||
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::repos::TrackRow;
|
||||
use psysonic_integration::subsonic::Song;
|
||||
|
||||
/// Project a Subsonic `Song` plus its raw JSON sub-tree into a
|
||||
/// `TrackRow`. `raw_value` is what `track.raw_json` stores verbatim so
|
||||
/// OpenSubsonic extensions survive (spec §5.1 / ADR-7).
|
||||
pub fn subsonic_song_to_track_row(
|
||||
server_id: &str,
|
||||
song: &Song,
|
||||
raw_value: &Value,
|
||||
synced_at: i64,
|
||||
library_id_fallback: Option<&str>,
|
||||
) -> TrackRow {
|
||||
TrackRow {
|
||||
server_id: server_id.to_string(),
|
||||
id: song.id.clone(),
|
||||
title: song.title.clone(),
|
||||
title_sort: None,
|
||||
artist: song.artist.clone(),
|
||||
artist_id: song.artist_id.clone(),
|
||||
album: song.album.clone().unwrap_or_default(),
|
||||
album_id: song.album_id.clone(),
|
||||
album_artist: song.album_artist.clone(),
|
||||
duration_sec: song.duration.unwrap_or(0),
|
||||
track_number: song.track_number,
|
||||
disc_number: song.disc_number,
|
||||
year: song.year,
|
||||
genre: song.genre.clone(),
|
||||
suffix: song.suffix.clone(),
|
||||
bit_rate: song.bit_rate,
|
||||
size_bytes: song.size,
|
||||
cover_art_id: song.cover_art.clone(),
|
||||
starred_at: parse_iso_ms(song.starred.as_deref()),
|
||||
user_rating: song.user_rating,
|
||||
play_count: song.play_count,
|
||||
played_at: parse_iso_ms(song.played.as_deref()),
|
||||
server_path: song.path.clone(),
|
||||
library_id: song.library_id.clone().or_else(|| library_id_fallback.map(String::from)),
|
||||
isrc: song.isrc.clone(),
|
||||
mbid_recording: song.mbid_recording.clone(),
|
||||
bpm: song.bpm,
|
||||
replay_gain_track_db: raw_value
|
||||
.get("replayGain")
|
||||
.and_then(|rg| rg.get("trackGain"))
|
||||
.and_then(|v| v.as_f64()),
|
||||
replay_gain_album_db: raw_value
|
||||
.get("replayGain")
|
||||
.and_then(|rg| rg.get("albumGain"))
|
||||
.and_then(|v| v.as_f64()),
|
||||
content_hash: None,
|
||||
server_updated_at: None,
|
||||
server_created_at: None,
|
||||
deleted: false,
|
||||
synced_at,
|
||||
raw_json: raw_value.to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Project a Navidrome `/api/song` row (native REST shape) into a
|
||||
/// `TrackRow`. Field names mostly overlap with Subsonic but use
|
||||
/// snake_case JSON aliases — we read fields by `get(name)` rather
|
||||
/// than reusing the Subsonic `Song` deserializer so a server-side
|
||||
/// rename doesn't silently zero out hot columns.
|
||||
pub fn navidrome_song_to_track_row(
|
||||
server_id: &str,
|
||||
raw: &Value,
|
||||
synced_at: i64,
|
||||
library_id_fallback: Option<&str>,
|
||||
) -> Option<TrackRow> {
|
||||
let id = raw.get("id").and_then(|v| v.as_str())?.to_string();
|
||||
let title = raw
|
||||
.get("title")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or_default()
|
||||
.to_string();
|
||||
let server_updated_at = raw
|
||||
.get("updatedAt")
|
||||
.and_then(|v| v.as_str())
|
||||
.and_then(parse_iso_ms_str);
|
||||
let library_id = json_string_field(raw, "libraryId")
|
||||
.or_else(|| json_string_field(raw, "library_id"))
|
||||
.or_else(|| json_string_field(raw, "musicFolderId"))
|
||||
.or_else(|| library_id_fallback.map(String::from));
|
||||
Some(TrackRow {
|
||||
server_id: server_id.to_string(),
|
||||
id,
|
||||
title,
|
||||
title_sort: string_field(raw, "sortTitle").or_else(|| string_field(raw, "orderTitle")),
|
||||
artist: string_field(raw, "artist"),
|
||||
artist_id: string_field(raw, "artistId"),
|
||||
album: string_field(raw, "album").unwrap_or_default(),
|
||||
album_id: string_field(raw, "albumId"),
|
||||
album_artist: string_field(raw, "albumArtist"),
|
||||
duration_sec: raw.get("duration").and_then(|v| v.as_i64()).unwrap_or(0),
|
||||
track_number: raw.get("trackNumber").and_then(|v| v.as_i64()),
|
||||
disc_number: raw.get("discNumber").and_then(|v| v.as_i64()),
|
||||
year: raw.get("year").and_then(|v| v.as_i64()),
|
||||
genre: string_field(raw, "genre"),
|
||||
suffix: string_field(raw, "suffix"),
|
||||
bit_rate: raw.get("bitRate").and_then(|v| v.as_i64()),
|
||||
size_bytes: raw.get("size").and_then(|v| v.as_i64()),
|
||||
cover_art_id: string_field(raw, "coverArtId").or_else(|| string_field(raw, "coverArt")),
|
||||
starred_at: raw.get("starredAt").and_then(|v| v.as_str()).and_then(parse_iso_ms_str),
|
||||
user_rating: raw.get("rating").and_then(|v| v.as_i64()),
|
||||
play_count: raw.get("playCount").and_then(|v| v.as_i64()),
|
||||
played_at: raw.get("playedAt").and_then(|v| v.as_str()).and_then(parse_iso_ms_str),
|
||||
server_path: string_field(raw, "path"),
|
||||
library_id,
|
||||
isrc: string_field(raw, "isrc"),
|
||||
mbid_recording: string_field(raw, "mbzTrackId").or_else(|| string_field(raw, "musicBrainzId")),
|
||||
bpm: raw.get("bpm").and_then(|v| v.as_i64()),
|
||||
replay_gain_track_db: raw.get("rgTrackGain").and_then(|v| v.as_f64()),
|
||||
replay_gain_album_db: raw.get("rgAlbumGain").and_then(|v| v.as_f64()),
|
||||
content_hash: None,
|
||||
server_updated_at,
|
||||
server_created_at: raw
|
||||
.get("createdAt")
|
||||
.and_then(|v| v.as_str())
|
||||
.and_then(parse_iso_ms_str),
|
||||
deleted: false,
|
||||
synced_at,
|
||||
raw_json: raw.to_string(),
|
||||
})
|
||||
}
|
||||
|
||||
fn json_string_field(raw: &Value, key: &str) -> Option<String> {
|
||||
match raw.get(key)? {
|
||||
Value::String(s) => Some(s.clone()),
|
||||
Value::Number(n) => Some(n.to_string()),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
fn string_field(raw: &Value, key: &str) -> Option<String> {
|
||||
json_string_field(raw, key)
|
||||
}
|
||||
|
||||
fn parse_iso_ms(s: Option<&str>) -> Option<i64> {
|
||||
s.and_then(parse_iso_ms_str)
|
||||
}
|
||||
|
||||
/// Lightweight ISO-8601 → epoch-ms parser. Supports the Navidrome /
|
||||
/// OpenSubsonic shape (`2024-06-01T12:00:00Z` or
|
||||
/// `2024-06-01T12:00:00.123+02:00`). Falls back to `None` on parse
|
||||
/// failure — sync code never panics on a bad timestamp.
|
||||
fn parse_iso_ms_str(s: &str) -> Option<i64> {
|
||||
// Strip fractional + timezone before doing the manual parse —
|
||||
// SQLite stores starred_at / played_at as integer ms, so we only
|
||||
// need second precision rounded up from the offset.
|
||||
let trimmed = s.trim();
|
||||
if trimmed.is_empty() {
|
||||
return None;
|
||||
}
|
||||
// Accept either `Z`, `+HH:MM`, or no suffix. Reduce to a flat
|
||||
// `YYYY-MM-DDTHH:MM:SS` core for parsing — server-side timestamps
|
||||
// are already in UTC for Navidrome, and we don't track timezone
|
||||
// in the schema column.
|
||||
let core = trimmed
|
||||
.find(|c: char| c == '.' || c == 'Z' || c == '+' || (c == '-' && trimmed.find('T').is_some_and(|t| trimmed[t..].contains(c))))
|
||||
.map(|i| &trimmed[..i])
|
||||
.unwrap_or(trimmed);
|
||||
let mut parts = core.split(['T', '-', ':']);
|
||||
let year: i64 = parts.next()?.parse().ok()?;
|
||||
let month: i64 = parts.next()?.parse().ok()?;
|
||||
let day: i64 = parts.next()?.parse().ok()?;
|
||||
let hour: i64 = parts.next().unwrap_or("0").parse().ok()?;
|
||||
let minute: i64 = parts.next().unwrap_or("0").parse().ok()?;
|
||||
let second: i64 = parts.next().unwrap_or("0").parse().ok()?;
|
||||
if !(1970..=2100).contains(&year)
|
||||
|| !(1..=12).contains(&month)
|
||||
|| !(1..=31).contains(&day)
|
||||
|| !(0..=23).contains(&hour)
|
||||
|| !(0..=59).contains(&minute)
|
||||
|| !(0..=60).contains(&second)
|
||||
{
|
||||
return None;
|
||||
}
|
||||
// Days since 1970-01-01 — Howard Hinnant's civil_from_days inverse.
|
||||
let y = if month <= 2 { year - 1 } else { year };
|
||||
let era = y.div_euclid(400);
|
||||
let yoe = y - era * 400; // [0, 399]
|
||||
let m = if month > 2 { month - 3 } else { month + 9 };
|
||||
let doy = (153 * m + 2) / 5 + day - 1;
|
||||
let doe = yoe * 365 + yoe / 4 - yoe / 100 + doy;
|
||||
let days = era * 146_097 + doe - 719_468;
|
||||
let seconds = days * 86_400 + hour * 3600 + minute * 60 + second;
|
||||
Some(seconds.saturating_mul(1000))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn parse_iso_handles_zulu_suffix() {
|
||||
// 2024-01-01T00:00:00Z = 1704067200000 ms.
|
||||
let ms = parse_iso_ms_str("2024-01-01T00:00:00Z").unwrap();
|
||||
assert_eq!(ms, 1_704_067_200_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_iso_handles_fractional_and_offset() {
|
||||
let ms = parse_iso_ms_str("2024-01-01T00:00:00.123+02:00").unwrap();
|
||||
// Truncated before offset → same epoch-second as Zulu of the
|
||||
// wall-clock value. Good enough for the schema's integer-ms
|
||||
// column.
|
||||
assert_eq!(ms, 1_704_067_200_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_iso_rejects_garbage() {
|
||||
assert!(parse_iso_ms_str("").is_none());
|
||||
assert!(parse_iso_ms_str("not-a-date").is_none());
|
||||
assert!(parse_iso_ms_str("9999-99-99").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn subsonic_song_maps_hot_columns_and_keeps_raw_json() {
|
||||
let raw = json!({
|
||||
"id": "tr_1",
|
||||
"title": "Hello",
|
||||
"artist": "World",
|
||||
"albumId": "al_1",
|
||||
"duration": 240,
|
||||
"track": 3,
|
||||
"year": 2024,
|
||||
"musicBrainzId": "mb-1",
|
||||
"replayGain": { "trackGain": -1.2, "albumGain": -0.8 }
|
||||
});
|
||||
let song: Song = serde_json::from_value(raw.clone()).unwrap();
|
||||
let row = subsonic_song_to_track_row("s1", &song, &raw, 1_000, Some("lib-fb"));
|
||||
assert_eq!(row.id, "tr_1");
|
||||
assert_eq!(row.album_id.as_deref(), Some("al_1"));
|
||||
assert_eq!(row.duration_sec, 240);
|
||||
assert_eq!(row.mbid_recording.as_deref(), Some("mb-1"));
|
||||
assert_eq!(row.replay_gain_track_db, Some(-1.2));
|
||||
assert_eq!(row.replay_gain_album_db, Some(-0.8));
|
||||
// Fallback library_id kicks in when the song didn't ship one.
|
||||
assert_eq!(row.library_id.as_deref(), Some("lib-fb"));
|
||||
assert!(row.raw_json.contains("replayGain"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn navidrome_song_maps_native_field_shape() {
|
||||
let raw = json!({
|
||||
"id": "tr_1",
|
||||
"title": "Hello",
|
||||
"artist": "World",
|
||||
"artistId": "ar_1",
|
||||
"album": "An Album",
|
||||
"albumId": "al_1",
|
||||
"albumArtist": "World",
|
||||
"duration": 240,
|
||||
"trackNumber": 3,
|
||||
"discNumber": 1,
|
||||
"year": 2024,
|
||||
"genre": "Ambient",
|
||||
"suffix": "flac",
|
||||
"bitRate": 1000,
|
||||
"size": 32_000_000_i64,
|
||||
"path": "World/An Album/03.flac",
|
||||
"libraryId": "1",
|
||||
"isrc": "USRC17607839",
|
||||
"mbzTrackId": "mb-1",
|
||||
"bpm": 128,
|
||||
"rgTrackGain": -1.2,
|
||||
"rgAlbumGain": -0.8,
|
||||
"createdAt": "2024-01-01T00:00:00Z",
|
||||
"updatedAt": "2024-06-01T00:00:00Z"
|
||||
});
|
||||
let row = navidrome_song_to_track_row("s1", &raw, 9_999, None).unwrap();
|
||||
assert_eq!(row.id, "tr_1");
|
||||
assert_eq!(row.track_number, Some(3));
|
||||
assert_eq!(row.isrc.as_deref(), Some("USRC17607839"));
|
||||
assert_eq!(row.mbid_recording.as_deref(), Some("mb-1"));
|
||||
assert_eq!(row.replay_gain_track_db, Some(-1.2));
|
||||
assert_eq!(row.library_id.as_deref(), Some("1"));
|
||||
assert!(row.server_updated_at.unwrap_or(0) > 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn navidrome_song_maps_numeric_library_id() {
|
||||
let raw = json!({
|
||||
"id": "tr_1",
|
||||
"title": "Hello",
|
||||
"libraryId": 3
|
||||
});
|
||||
let row = navidrome_song_to_track_row("s1", &raw, 1, None).unwrap();
|
||||
assert_eq!(row.library_id.as_deref(), Some("3"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn navidrome_song_skips_rows_without_id() {
|
||||
let row = navidrome_song_to_track_row("s1", &json!({"title": "no id"}), 1, None);
|
||||
assert!(row.is_none());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
//! Sync orchestrator.
|
||||
//!
|
||||
//! PR-3a landed the foundation (`CapabilityProbe` C1 +
|
||||
//! `SyncStateRepository` C7); PR-3b adds the initial-sync runner,
|
||||
//! ingest-strategy selection, backoff, and the §6.9 id remap path.
|
||||
//! `DeltaSyncRunner` / background scheduler / Tauri surface follow in
|
||||
//! PR-3c / PR-3d / PR-5.
|
||||
|
||||
pub mod backoff;
|
||||
pub mod bandwidth;
|
||||
pub mod budget;
|
||||
pub mod capability;
|
||||
pub mod cursor;
|
||||
pub mod delta;
|
||||
pub mod error;
|
||||
pub mod ingest_parallel;
|
||||
pub mod initial;
|
||||
pub mod mapping;
|
||||
pub mod poll_stats;
|
||||
pub mod progress;
|
||||
pub mod scheduler;
|
||||
pub mod strategy;
|
||||
pub mod supervisor;
|
||||
pub mod tombstone;
|
||||
|
||||
pub use backoff::{with_jitter, Backoff};
|
||||
pub use bandwidth::{ParallelismBudget, PlaybackHint};
|
||||
pub use budget::{PassKind, RequestBudget};
|
||||
pub use capability::{CapabilityFlags, CapabilityProbe, NavidromeProbeCredentials};
|
||||
pub use cursor::{CursorPhase, InitialSyncCursor, StrategyState};
|
||||
pub use delta::{DeltaSyncReport, DeltaSyncRunner};
|
||||
pub use error::SyncError;
|
||||
pub use initial::{InitialSyncReport, InitialSyncRunner};
|
||||
pub use mapping::{navidrome_song_to_track_row, subsonic_song_to_track_row};
|
||||
pub use poll_stats::{classify_tier, next_interval_ms, LibraryTier, PollStats};
|
||||
pub use progress::{ChannelProgress, NoopProgress, Progress, ProgressEvent};
|
||||
pub use scheduler::{BackgroundScheduler, SchedulerTickReport, DEFAULT_TOMBSTONE_THRESHOLD_PCT};
|
||||
pub use strategy::IngestStrategy;
|
||||
pub use supervisor::SyncSupervisor;
|
||||
pub use tombstone::{should_auto_reconcile, TombstoneReconciler, TombstoneReport};
|
||||
@@ -0,0 +1,250 @@
|
||||
//! C10 — adaptive poll interval (spec §6.2.2).
|
||||
//!
|
||||
//! Rolling EWMA over the last delta pass's HTTP response stats plus
|
||||
//! the artist-count signal classifies the server into a `LibraryTier`,
|
||||
//! which then drives the next poll interval. Persisted in
|
||||
//! `sync_state.poll_stats_json` so the scheduler picks up where it
|
||||
//! left off across restarts.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// EWMA smoothing factor — higher values weight recent samples more.
|
||||
/// 0.3 matches the §6.2.2 "rolling EWMA" target without over-reacting
|
||||
/// to a single slow response.
|
||||
pub const EWMA_ALPHA: f64 = 0.3;
|
||||
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum LibraryTier {
|
||||
/// `<2k` artists.
|
||||
Small,
|
||||
/// `2k-15k` artists.
|
||||
Medium,
|
||||
/// `>15k` artists OR `ewma_bytes > 2 MB`.
|
||||
Huge,
|
||||
#[default]
|
||||
Unknown,
|
||||
}
|
||||
|
||||
impl LibraryTier {
|
||||
pub fn as_tag(self) -> &'static str {
|
||||
match self {
|
||||
Self::Small => "small",
|
||||
Self::Medium => "medium",
|
||||
Self::Huge => "huge",
|
||||
Self::Unknown => "unknown",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Serialize, Deserialize)]
|
||||
pub struct PollStats {
|
||||
/// Last successful `getArtists.index` cardinality, or
|
||||
/// `getScanStatus.count` estimate. Used as the primary tier
|
||||
/// signal.
|
||||
#[serde(default)]
|
||||
pub artist_count: u64,
|
||||
/// EWMA of the most recent poll response sizes, in bytes.
|
||||
/// Decompressed (post-gzip) per §2.2.1.
|
||||
#[serde(default)]
|
||||
pub ewma_bytes: f64,
|
||||
/// EWMA of the most recent poll wall-clock durations, in ms.
|
||||
#[serde(default)]
|
||||
pub ewma_duration_ms: f64,
|
||||
/// Resolved tier from the inputs above.
|
||||
#[serde(default)]
|
||||
pub library_tier: LibraryTier,
|
||||
}
|
||||
|
||||
impl PollStats {
|
||||
/// Fold a new sample into the EWMAs. First sample seeds the
|
||||
/// averages directly so a single fresh poll yields a meaningful
|
||||
/// tier classification.
|
||||
pub fn observe(&mut self, bytes: u64, duration_ms: u64) {
|
||||
let b = bytes as f64;
|
||||
let d = duration_ms as f64;
|
||||
self.ewma_bytes = if self.ewma_bytes == 0.0 {
|
||||
b
|
||||
} else {
|
||||
EWMA_ALPHA * b + (1.0 - EWMA_ALPHA) * self.ewma_bytes
|
||||
};
|
||||
self.ewma_duration_ms = if self.ewma_duration_ms == 0.0 {
|
||||
d
|
||||
} else {
|
||||
EWMA_ALPHA * d + (1.0 - EWMA_ALPHA) * self.ewma_duration_ms
|
||||
};
|
||||
}
|
||||
|
||||
/// Update `artist_count` and recompute the resolved tier.
|
||||
pub fn set_artist_count(&mut self, count: u64) {
|
||||
self.artist_count = count;
|
||||
self.library_tier = classify_tier(self.artist_count, self.ewma_bytes);
|
||||
}
|
||||
|
||||
/// Force a tier re-classification — call after a fresh
|
||||
/// `observe()` pair if a borderline `ewma_bytes` may have tipped
|
||||
/// the threshold.
|
||||
pub fn reclassify(&mut self) {
|
||||
self.library_tier = classify_tier(self.artist_count, self.ewma_bytes);
|
||||
}
|
||||
}
|
||||
|
||||
/// §6.2.2 classification table.
|
||||
pub fn classify_tier(artist_count: u64, ewma_bytes: f64) -> LibraryTier {
|
||||
if artist_count == 0 && ewma_bytes == 0.0 {
|
||||
return LibraryTier::Unknown;
|
||||
}
|
||||
if artist_count > 15_000 || ewma_bytes > 2_000_000.0 {
|
||||
return LibraryTier::Huge;
|
||||
}
|
||||
if artist_count >= 2_000 {
|
||||
return LibraryTier::Medium;
|
||||
}
|
||||
LibraryTier::Small
|
||||
}
|
||||
|
||||
/// Spec §6.2.2 formula. Returns the delta in milliseconds — caller
|
||||
/// stamps `next_poll_at = now + this`. All arithmetic in `f64` so the
|
||||
/// `load_factor` clamp produces a smooth curve.
|
||||
pub fn next_interval_ms(stats: &PollStats) -> u64 {
|
||||
let base_ms: u64 = match stats.library_tier {
|
||||
LibraryTier::Huge => 15 * 60 * 1000,
|
||||
LibraryTier::Medium => 10 * 60 * 1000,
|
||||
LibraryTier::Small => 5 * 60 * 1000,
|
||||
// No data yet — start short so the first real tick lands
|
||||
// quickly and re-classifies.
|
||||
LibraryTier::Unknown => 60 * 1000,
|
||||
};
|
||||
let load_factor = if stats.ewma_duration_ms <= 0.0 {
|
||||
1.0
|
||||
} else {
|
||||
(stats.ewma_duration_ms / 3000.0).clamp(1.0, 10.0)
|
||||
};
|
||||
let artist_factor: f64 = if matches!(stats.library_tier, LibraryTier::Huge) {
|
||||
3.0
|
||||
} else {
|
||||
1.0
|
||||
};
|
||||
let scaled = (base_ms as f64) * load_factor * artist_factor;
|
||||
scaled.min(u64::MAX as f64) as u64
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn classify_returns_unknown_with_zero_signals() {
|
||||
assert_eq!(classify_tier(0, 0.0), LibraryTier::Unknown);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn classify_small_under_2k_artists() {
|
||||
assert_eq!(classify_tier(500, 1000.0), LibraryTier::Small);
|
||||
assert_eq!(classify_tier(1_999, 1000.0), LibraryTier::Small);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn classify_medium_in_range() {
|
||||
assert_eq!(classify_tier(2_000, 1000.0), LibraryTier::Medium);
|
||||
assert_eq!(classify_tier(15_000, 1000.0), LibraryTier::Medium);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn classify_huge_above_15k_artists() {
|
||||
assert_eq!(classify_tier(15_001, 1000.0), LibraryTier::Huge);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn classify_huge_when_ewma_bytes_exceed_2mb_even_with_few_artists() {
|
||||
// Slow-network signal — the spec's `ewma_bytes > 2MB` override.
|
||||
assert_eq!(classify_tier(500, 2_500_000.0), LibraryTier::Huge);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn observe_first_sample_seeds_ewmas_directly() {
|
||||
let mut s = PollStats::default();
|
||||
s.observe(100_000, 1500);
|
||||
assert_eq!(s.ewma_bytes, 100_000.0);
|
||||
assert_eq!(s.ewma_duration_ms, 1500.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn observe_subsequent_samples_apply_alpha_smoothing() {
|
||||
let mut s = PollStats::default();
|
||||
s.observe(100_000, 1000);
|
||||
s.observe(200_000, 2000);
|
||||
// 0.3 * 200_000 + 0.7 * 100_000 = 60_000 + 70_000 = 130_000
|
||||
assert!((s.ewma_bytes - 130_000.0).abs() < 0.001);
|
||||
assert!((s.ewma_duration_ms - 1300.0).abs() < 0.001);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn set_artist_count_triggers_reclassification() {
|
||||
let mut s = PollStats::default();
|
||||
s.observe(50_000, 1500);
|
||||
s.set_artist_count(5_000);
|
||||
assert_eq!(s.library_tier, LibraryTier::Medium);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn next_interval_unknown_tier_starts_short() {
|
||||
let s = PollStats::default();
|
||||
assert_eq!(next_interval_ms(&s), 60_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn next_interval_small_base_5min_at_idle_load() {
|
||||
let mut s = PollStats::default();
|
||||
s.set_artist_count(1000);
|
||||
// load_factor clamps at 1.0 when ewma_duration_ms = 0.
|
||||
assert_eq!(next_interval_ms(&s), 5 * 60_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn next_interval_huge_uses_artist_factor_3x() {
|
||||
let mut s = PollStats::default();
|
||||
s.set_artist_count(20_000);
|
||||
// 15 min * 3 = 45 min on idle load.
|
||||
assert_eq!(next_interval_ms(&s), 45 * 60_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn next_interval_load_factor_stretches_with_slow_network() {
|
||||
let mut s = PollStats::default();
|
||||
s.set_artist_count(1000);
|
||||
s.observe(50_000, 9_000); // 3× the 3000ms target
|
||||
// 5 min * 3 (load_factor) = 15 min.
|
||||
let ms = next_interval_ms(&s);
|
||||
assert!(
|
||||
(14 * 60_000..=16 * 60_000).contains(&ms),
|
||||
"expected ~15min, got {ms}ms"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn next_interval_load_factor_clamped_at_10x() {
|
||||
let mut s = PollStats::default();
|
||||
s.set_artist_count(1000);
|
||||
s.observe(50_000, 60_000); // 20× target → clamps at 10
|
||||
assert_eq!(next_interval_ms(&s), 10 * 5 * 60_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn poll_stats_round_trips_through_json() {
|
||||
let mut s = PollStats::default();
|
||||
s.observe(123_456, 789);
|
||||
s.set_artist_count(3_500);
|
||||
let json = serde_json::to_value(s).unwrap();
|
||||
let back: PollStats = serde_json::from_value(json).unwrap();
|
||||
assert_eq!(back, s);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn poll_stats_deserialize_tolerates_missing_fields() {
|
||||
// Stored default is `'{}'` per spec §5.1; runner must accept it
|
||||
// as a fresh stats object.
|
||||
let s: PollStats = serde_json::from_str("{}").unwrap();
|
||||
assert_eq!(s, PollStats::default());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,329 @@
|
||||
//! C6 — progress channel for the sync runners (spec §6 emit limit
|
||||
//! `≤2 events/s`). The runners call `Progress::emit` at phase
|
||||
//! transitions and per-batch checkpoints; the supervisor wraps an
|
||||
//! `mpsc::UnboundedSender` so the top crate (PR-5) can forward events
|
||||
//! to Tauri's emit surface.
|
||||
//!
|
||||
//! Non-terminal events are rate-limited. `IngestPage` updates are
|
||||
//! **coalesced** (latest `ingested_total` wins) so fast S1/N1 batches
|
||||
//! do not leave the UI stuck on a stale count between throttled emits.
|
||||
|
||||
use std::sync::Mutex;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::repos::RemapEntry;
|
||||
|
||||
/// Per-batch ingest timings (DevTools + terminal diagnosis).
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct IngestBatchMetrics {
|
||||
pub offset: u32,
|
||||
pub strategy: String,
|
||||
pub fetch_ms: u32,
|
||||
pub write_ms: u32,
|
||||
pub lock_wait_ms: u32,
|
||||
pub sql_exec_ms: u32,
|
||||
pub persist_ms: u32,
|
||||
pub row_count: u32,
|
||||
pub bulk_ingest_active: bool,
|
||||
}
|
||||
|
||||
/// Lean event union — server_id / library_scope context lives on the
|
||||
/// channel side (one supervisor = one scope). Top-crate code wraps
|
||||
/// these into Tauri events with their own envelope.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum ProgressEvent {
|
||||
PhaseChanged { phase: String },
|
||||
IngestPage {
|
||||
ingested_total: u32,
|
||||
batch_count: u32,
|
||||
metrics: Option<IngestBatchMetrics>,
|
||||
},
|
||||
Remapped { entries: Vec<RemapEntry> },
|
||||
Tombstoned { deleted_count: u32, checked_count: u32 },
|
||||
Completed { kind: String },
|
||||
Error { message: String },
|
||||
}
|
||||
|
||||
impl ProgressEvent {
|
||||
/// Terminal events always bypass the throttle so callers never
|
||||
/// miss a "we're done" / "we crashed" signal.
|
||||
pub fn always_emit(&self) -> bool {
|
||||
matches!(self, Self::Completed { .. } | Self::Error { .. })
|
||||
}
|
||||
}
|
||||
|
||||
pub trait Progress: Send + Sync {
|
||||
fn emit(&self, event: ProgressEvent);
|
||||
}
|
||||
|
||||
/// No-op implementation. Used as the default when runners are called
|
||||
/// outside a supervisor (tests, future ad-hoc invocations).
|
||||
pub struct NoopProgress;
|
||||
|
||||
impl Progress for NoopProgress {
|
||||
fn emit(&self, _event: ProgressEvent) {}
|
||||
}
|
||||
|
||||
/// `Progress` impl that forwards through a tokio mpsc channel,
|
||||
/// throttling non-terminal events to the configured `min_interval`.
|
||||
pub struct ChannelProgress {
|
||||
sender: tokio::sync::mpsc::UnboundedSender<ProgressEvent>,
|
||||
min_interval: Duration,
|
||||
last_emit: Mutex<Option<Instant>>,
|
||||
/// Latest ingest checkpoint held back while the throttle gate is closed.
|
||||
pending_ingest: Mutex<Option<(u32, u32, Option<IngestBatchMetrics>)>>,
|
||||
}
|
||||
|
||||
impl ChannelProgress {
|
||||
/// 500 ms gate ≈ 2 events/s per spec §6.
|
||||
pub const DEFAULT_INTERVAL: Duration = Duration::from_millis(500);
|
||||
|
||||
pub fn new(sender: tokio::sync::mpsc::UnboundedSender<ProgressEvent>) -> Self {
|
||||
Self::with_interval(sender, Self::DEFAULT_INTERVAL)
|
||||
}
|
||||
|
||||
pub fn with_interval(
|
||||
sender: tokio::sync::mpsc::UnboundedSender<ProgressEvent>,
|
||||
min_interval: Duration,
|
||||
) -> Self {
|
||||
Self {
|
||||
sender,
|
||||
min_interval,
|
||||
last_emit: Mutex::new(None),
|
||||
pending_ingest: Mutex::new(None),
|
||||
}
|
||||
}
|
||||
|
||||
fn flush_pending_ingest(&self) {
|
||||
let pending = self
|
||||
.pending_ingest
|
||||
.lock()
|
||||
.expect("progress pending lock poisoned")
|
||||
.take();
|
||||
if let Some((ingested_total, batch_count, metrics)) = pending {
|
||||
let _ = self.sender.send(ProgressEvent::IngestPage {
|
||||
ingested_total,
|
||||
batch_count,
|
||||
metrics,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
fn throttle_open(&self) -> bool {
|
||||
if self.min_interval.is_zero() {
|
||||
return true;
|
||||
}
|
||||
let last = self.last_emit.lock().expect("progress lock poisoned");
|
||||
match *last {
|
||||
None => true,
|
||||
Some(prev) if prev.elapsed() >= self.min_interval => true,
|
||||
Some(_) => false,
|
||||
}
|
||||
}
|
||||
|
||||
fn mark_emitted(&self) {
|
||||
if self.min_interval.is_zero() {
|
||||
return;
|
||||
}
|
||||
*self
|
||||
.last_emit
|
||||
.lock()
|
||||
.expect("progress lock poisoned") = Some(Instant::now());
|
||||
}
|
||||
}
|
||||
|
||||
impl Progress for ChannelProgress {
|
||||
fn emit(&self, event: ProgressEvent) {
|
||||
if event.always_emit() {
|
||||
self.flush_pending_ingest();
|
||||
let _ = self.sender.send(event);
|
||||
return;
|
||||
}
|
||||
|
||||
if let ProgressEvent::IngestPage {
|
||||
ingested_total,
|
||||
batch_count,
|
||||
metrics,
|
||||
} = event
|
||||
{
|
||||
let gate_open = self.throttle_open();
|
||||
{
|
||||
let mut pending = self
|
||||
.pending_ingest
|
||||
.lock()
|
||||
.expect("progress pending lock poisoned");
|
||||
if gate_open {
|
||||
if let Some((total, batch, m)) = pending.take() {
|
||||
let _ = self.sender.send(ProgressEvent::IngestPage {
|
||||
ingested_total: total,
|
||||
batch_count: batch,
|
||||
metrics: m,
|
||||
});
|
||||
}
|
||||
}
|
||||
*pending = Some((ingested_total, batch_count, metrics));
|
||||
}
|
||||
if gate_open {
|
||||
self.mark_emitted();
|
||||
self.flush_pending_ingest();
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if !self.min_interval.is_zero() && !self.throttle_open() {
|
||||
return;
|
||||
}
|
||||
self.flush_pending_ingest();
|
||||
self.mark_emitted();
|
||||
let _ = self.sender.send(event);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::thread;
|
||||
use std::time::Duration;
|
||||
use tokio::sync::mpsc;
|
||||
|
||||
#[test]
|
||||
fn noop_progress_swallows_events_without_panicking() {
|
||||
let p = NoopProgress;
|
||||
p.emit(ProgressEvent::PhaseChanged { phase: "ingest".into() });
|
||||
p.emit(ProgressEvent::Completed { kind: "initial_sync".into() });
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn zero_interval_channel_emits_every_event() {
|
||||
let (tx, mut rx) = mpsc::unbounded_channel();
|
||||
let p = ChannelProgress::with_interval(tx, Duration::ZERO);
|
||||
for i in 0..10 {
|
||||
p.emit(ProgressEvent::IngestPage {
|
||||
ingested_total: i,
|
||||
batch_count: 1,
|
||||
metrics: None,
|
||||
});
|
||||
}
|
||||
let mut received = 0;
|
||||
while rx.try_recv().is_ok() {
|
||||
received += 1;
|
||||
}
|
||||
assert_eq!(received, 10, "ZERO interval must not drop anything");
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn terminal_events_bypass_throttle() {
|
||||
let (tx, mut rx) = mpsc::unbounded_channel();
|
||||
let p = ChannelProgress::with_interval(tx, Duration::from_secs(60));
|
||||
p.emit(ProgressEvent::IngestPage {
|
||||
ingested_total: 999,
|
||||
batch_count: 3,
|
||||
metrics: None,
|
||||
});
|
||||
p.emit(ProgressEvent::Completed { kind: "delta_sync".into() });
|
||||
p.emit(ProgressEvent::Error { message: "boom".into() });
|
||||
assert!(matches!(
|
||||
rx.try_recv(),
|
||||
Ok(ProgressEvent::IngestPage {
|
||||
ingested_total: 999,
|
||||
..
|
||||
})
|
||||
));
|
||||
assert!(matches!(
|
||||
rx.try_recv(),
|
||||
Ok(ProgressEvent::Completed { .. })
|
||||
));
|
||||
assert!(matches!(rx.try_recv(), Ok(ProgressEvent::Error { .. })));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn non_terminal_events_collapse_under_throttle() {
|
||||
let (tx, mut rx) = mpsc::unbounded_channel();
|
||||
let p = ChannelProgress::with_interval(tx, Duration::from_millis(100));
|
||||
p.emit(ProgressEvent::PhaseChanged { phase: "a".into() });
|
||||
p.emit(ProgressEvent::PhaseChanged { phase: "b".into() });
|
||||
assert!(matches!(
|
||||
rx.try_recv(),
|
||||
Ok(ProgressEvent::PhaseChanged { ref phase }) if phase == "a"
|
||||
));
|
||||
assert!(rx.try_recv().is_err(), "second emit must have been dropped");
|
||||
|
||||
thread::sleep(Duration::from_millis(120));
|
||||
p.emit(ProgressEvent::PhaseChanged { phase: "c".into() });
|
||||
assert!(matches!(
|
||||
rx.try_recv(),
|
||||
Ok(ProgressEvent::PhaseChanged { ref phase }) if phase == "c"
|
||||
));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn ingest_pages_coalesce_to_latest_within_throttle_window() {
|
||||
let (tx, mut rx) = mpsc::unbounded_channel();
|
||||
let p = ChannelProgress::with_interval(tx, Duration::from_millis(100));
|
||||
p.emit(ProgressEvent::IngestPage {
|
||||
ingested_total: 500,
|
||||
batch_count: 1,
|
||||
metrics: None,
|
||||
});
|
||||
p.emit(ProgressEvent::IngestPage {
|
||||
ingested_total: 2500,
|
||||
batch_count: 5,
|
||||
metrics: None,
|
||||
});
|
||||
p.emit(ProgressEvent::IngestPage {
|
||||
ingested_total: 5000,
|
||||
batch_count: 10,
|
||||
metrics: None,
|
||||
});
|
||||
assert!(matches!(
|
||||
rx.try_recv(),
|
||||
Ok(ProgressEvent::IngestPage {
|
||||
ingested_total: 500,
|
||||
..
|
||||
})
|
||||
));
|
||||
assert!(rx.try_recv().is_err(), "bursts must coalesce, not stack");
|
||||
|
||||
thread::sleep(Duration::from_millis(120));
|
||||
p.emit(ProgressEvent::IngestPage {
|
||||
ingested_total: 5500,
|
||||
batch_count: 11,
|
||||
metrics: None,
|
||||
});
|
||||
assert!(
|
||||
matches!(
|
||||
rx.try_recv(),
|
||||
Ok(ProgressEvent::IngestPage {
|
||||
ingested_total: 5000,
|
||||
..
|
||||
})
|
||||
),
|
||||
"latest pending count must flush when the gate opens"
|
||||
);
|
||||
assert!(matches!(
|
||||
rx.try_recv(),
|
||||
Ok(ProgressEvent::IngestPage {
|
||||
ingested_total: 5500,
|
||||
..
|
||||
})
|
||||
));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn closed_receiver_does_not_panic_the_sender() {
|
||||
let (tx, rx) = mpsc::unbounded_channel();
|
||||
let p = ChannelProgress::with_interval(tx, Duration::ZERO);
|
||||
drop(rx);
|
||||
p.emit(ProgressEvent::PhaseChanged { phase: "x".into() });
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn always_emit_true_for_terminal_events() {
|
||||
assert!(ProgressEvent::Completed { kind: "k".into() }.always_emit());
|
||||
assert!(ProgressEvent::Error { message: "m".into() }.always_emit());
|
||||
assert!(!ProgressEvent::PhaseChanged { phase: "p".into() }.always_emit());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,619 @@
|
||||
//! C8 — background scheduler (spec §6.2).
|
||||
//!
|
||||
//! Tick-based: the top crate (PR-5) drives the actual timer; PR-3d2
|
||||
//! ships the logic that decides "is it time?", picks the budget +
|
||||
//! tombstone trigger, runs the DeltaSyncRunner, and writes back the
|
||||
//! adaptive interval.
|
||||
//!
|
||||
//! Owns no tokio task itself — keeps testability high and lets the
|
||||
//! caller decide spawn behaviour (Supervisor or inline).
|
||||
|
||||
use std::sync::atomic::AtomicBool;
|
||||
use std::sync::Arc;
|
||||
|
||||
use psysonic_integration::subsonic::SubsonicClient;
|
||||
|
||||
use super::bandwidth::{ParallelismBudget, PlaybackHint};
|
||||
use super::budget::{PassKind, RequestBudget};
|
||||
use super::capability::{CapabilityFlags, NavidromeProbeCredentials};
|
||||
use super::delta::{DeltaSyncReport, DeltaSyncRunner};
|
||||
use super::error::SyncError;
|
||||
use super::poll_stats::{next_interval_ms, PollStats};
|
||||
use super::progress::{NoopProgress, Progress};
|
||||
use super::tombstone::should_auto_reconcile;
|
||||
use crate::repos::SyncStateRepository;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
/// Default Mode B threshold per §6.7 (5 % gap before auto reconcile).
|
||||
pub const DEFAULT_TOMBSTONE_THRESHOLD_PCT: u32 = 5;
|
||||
|
||||
/// Outcome of one scheduler tick — what happened plus the resolved
|
||||
/// `next_poll_at` so the caller can re-schedule its timer.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct SchedulerTickReport {
|
||||
pub skipped_not_due: bool,
|
||||
pub skipped_bulk_paused: bool,
|
||||
/// Delta/tombstone pass deferred while initial sync or capability probe
|
||||
/// holds `sync_phase`, IS-3 bulk ingest is active, or a foreground sync
|
||||
/// job (`LibraryRuntime::current_job`) is running for this server.
|
||||
pub skipped_sync_pass_active: bool,
|
||||
pub delta: Option<DeltaSyncReport>,
|
||||
pub next_poll_at_ms: i64,
|
||||
}
|
||||
|
||||
pub struct BackgroundScheduler<'a> {
|
||||
store: &'a LibraryStore,
|
||||
subsonic: &'a SubsonicClient,
|
||||
navidrome: Option<NavidromeProbeCredentials>,
|
||||
server_id: String,
|
||||
library_scope: String,
|
||||
capability_flags: CapabilityFlags,
|
||||
playback_hint: PlaybackHint,
|
||||
cancel: Option<Arc<AtomicBool>>,
|
||||
progress: Arc<dyn Progress + Send + Sync>,
|
||||
tombstone_threshold_pct: u32,
|
||||
sleep_enabled: bool,
|
||||
/// When true, a user-triggered sync job (delta / verify / full resync)
|
||||
/// already owns this server — skip the background delta pass.
|
||||
foreground_sync_job_active: bool,
|
||||
}
|
||||
|
||||
impl<'a> BackgroundScheduler<'a> {
|
||||
pub fn new(
|
||||
store: &'a LibraryStore,
|
||||
subsonic: &'a SubsonicClient,
|
||||
server_id: impl Into<String>,
|
||||
library_scope: impl Into<String>,
|
||||
capability_flags: CapabilityFlags,
|
||||
) -> Self {
|
||||
Self {
|
||||
store,
|
||||
subsonic,
|
||||
navidrome: None,
|
||||
server_id: server_id.into(),
|
||||
library_scope: library_scope.into(),
|
||||
capability_flags,
|
||||
playback_hint: PlaybackHint::Idle,
|
||||
cancel: None,
|
||||
progress: Arc::new(NoopProgress),
|
||||
tombstone_threshold_pct: DEFAULT_TOMBSTONE_THRESHOLD_PCT,
|
||||
sleep_enabled: true,
|
||||
foreground_sync_job_active: false,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn with_navidrome_credentials(mut self, creds: NavidromeProbeCredentials) -> Self {
|
||||
self.navidrome = Some(creds);
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_playback_hint(mut self, hint: PlaybackHint) -> Self {
|
||||
self.playback_hint = hint;
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_cancellation(mut self, flag: Arc<AtomicBool>) -> Self {
|
||||
self.cancel = Some(flag);
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_progress(mut self, progress: Arc<dyn Progress + Send + Sync>) -> Self {
|
||||
self.progress = progress;
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_tombstone_threshold_pct(mut self, pct: u32) -> Self {
|
||||
self.tombstone_threshold_pct = pct;
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_sleep_disabled(mut self) -> Self {
|
||||
self.sleep_enabled = false;
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_foreground_sync_job_active(mut self, active: bool) -> Self {
|
||||
self.foreground_sync_job_active = active;
|
||||
self
|
||||
}
|
||||
|
||||
/// `true` when `next_poll_at` has passed (or no value yet). Caller
|
||||
/// short-circuits its timer when this returns `false`.
|
||||
pub fn is_due(&self, now_ms: i64) -> Result<bool, SyncError> {
|
||||
let sync_state = SyncStateRepository::new(self.store);
|
||||
let next = sync_state
|
||||
.get_next_poll_at(&self.server_id, &self.library_scope)
|
||||
.map_err(SyncError::Storage)?;
|
||||
Ok(next.map(|n| now_ms >= n).unwrap_or(true))
|
||||
}
|
||||
|
||||
/// Resolve the parallelism budget for the current playback state.
|
||||
/// Bulk-paused state means the scheduler skips the tick entirely
|
||||
/// and just re-schedules.
|
||||
pub fn parallelism_budget(&self) -> ParallelismBudget {
|
||||
ParallelismBudget::resolve(self.playback_hint)
|
||||
}
|
||||
|
||||
/// Run one tick — runs a delta sync if due and bulk isn't paused
|
||||
/// by the playback signal, then writes the new `next_poll_at`.
|
||||
pub async fn tick(&self, now_ms: i64) -> Result<SchedulerTickReport, SyncError> {
|
||||
let sync_state = SyncStateRepository::new(self.store);
|
||||
sync_state
|
||||
.ensure(&self.server_id, &self.library_scope)
|
||||
.map_err(SyncError::Storage)?;
|
||||
|
||||
let mut report = SchedulerTickReport {
|
||||
skipped_not_due: false,
|
||||
skipped_bulk_paused: false,
|
||||
skipped_sync_pass_active: false,
|
||||
delta: None,
|
||||
next_poll_at_ms: now_ms,
|
||||
};
|
||||
|
||||
if self.sync_pass_active(&sync_state)? {
|
||||
report.skipped_sync_pass_active = true;
|
||||
report.next_poll_at_ms = now_ms + 30_000;
|
||||
sync_state
|
||||
.set_next_poll_at(&self.server_id, &self.library_scope, report.next_poll_at_ms)
|
||||
.map_err(SyncError::Storage)?;
|
||||
crate::app_eprintln!(
|
||||
"[library-sync] scheduler tick skipped: sync pass active (phase={:?}, bulk={})",
|
||||
sync_state
|
||||
.get_sync_phase(&self.server_id, &self.library_scope)
|
||||
.ok()
|
||||
.flatten(),
|
||||
self.store.bulk_ingest_active()
|
||||
);
|
||||
return Ok(report);
|
||||
}
|
||||
|
||||
if !self.is_due(now_ms)? {
|
||||
report.skipped_not_due = true;
|
||||
let stats = self.load_poll_stats(&sync_state)?;
|
||||
report.next_poll_at_ms = now_ms + next_interval_ms(&stats) as i64;
|
||||
return Ok(report);
|
||||
}
|
||||
|
||||
let parallelism = self.parallelism_budget();
|
||||
if parallelism.bulk_paused() {
|
||||
// §6.2.4 PrefetchActive — skip this tick entirely, re-poll
|
||||
// soon so we can catch the prefetch finishing.
|
||||
report.skipped_bulk_paused = true;
|
||||
report.next_poll_at_ms = now_ms + 30_000; // ~30s short retry
|
||||
sync_state
|
||||
.set_next_poll_at(&self.server_id, &self.library_scope, report.next_poll_at_ms)
|
||||
.map_err(SyncError::Storage)?;
|
||||
return Ok(report);
|
||||
}
|
||||
|
||||
// Decide budget + tombstone trigger.
|
||||
let mut tombstone_budget: u32 = 0;
|
||||
if let (Some(local), Some(server)) = (
|
||||
sync_state
|
||||
.get_local_track_count(&self.server_id, &self.library_scope)
|
||||
.map_err(SyncError::Storage)?,
|
||||
sync_state
|
||||
.get_server_track_count(&self.server_id, &self.library_scope)
|
||||
.map_err(SyncError::Storage)?,
|
||||
) {
|
||||
let (local_u, server_u) = (local.max(0) as u32, server.max(0) as u32);
|
||||
if should_auto_reconcile(local_u, server_u, self.tombstone_threshold_pct) {
|
||||
tombstone_budget = RequestBudget::DELTA_MISMATCH_CAP;
|
||||
}
|
||||
}
|
||||
let _pass_budget = if tombstone_budget > 0 {
|
||||
RequestBudget::for_pass(PassKind::DeltaMismatch)
|
||||
} else {
|
||||
RequestBudget::for_pass(PassKind::DeltaLight)
|
||||
};
|
||||
// PR-3d2 doesn't enforce pass_budget against the runner yet —
|
||||
// delta runner is already small (1 probe + ≤8 album-list
|
||||
// pages); the budget value is recorded so PR-5 can surface it
|
||||
// in Settings. Wire actual cap in the runner when DS-7
|
||||
// starred delta or other request-heavy paths land.
|
||||
|
||||
// Run the delta pass.
|
||||
let mut runner = DeltaSyncRunner::new(
|
||||
self.store,
|
||||
self.subsonic,
|
||||
&self.server_id,
|
||||
&self.library_scope,
|
||||
self.capability_flags,
|
||||
)
|
||||
.with_progress(Arc::clone(&self.progress));
|
||||
if let Some(creds) = &self.navidrome {
|
||||
runner = runner.with_navidrome_credentials(creds.clone());
|
||||
}
|
||||
if let Some(flag) = &self.cancel {
|
||||
runner = runner.with_cancellation(Arc::clone(flag));
|
||||
}
|
||||
if !self.sleep_enabled {
|
||||
runner = runner.with_sleep_disabled();
|
||||
}
|
||||
if tombstone_budget > 0 {
|
||||
runner = runner.with_tombstone_budget(tombstone_budget);
|
||||
}
|
||||
let delta_report = runner.run().await?;
|
||||
|
||||
// Update poll_stats: nothing measured per-request yet in
|
||||
// PR-3d2 (PR-5 will plumb byte/duration via a custom HTTP
|
||||
// wrapper). For now the tier signal updates from artist_count
|
||||
// when the next probe lands; we just persist the artist_count
|
||||
// we know from the local DB so the tier classifier has data.
|
||||
let mut stats = self.load_poll_stats(&sync_state)?;
|
||||
if delta_report.changed_count > 0 {
|
||||
// Re-stamp the local count snapshot so the next tick's
|
||||
// threshold check has fresh data.
|
||||
if let Ok(local) = self.count_local_tracks() {
|
||||
sync_state
|
||||
.set_local_track_count(&self.server_id, &self.library_scope, local)
|
||||
.map_err(SyncError::Storage)?;
|
||||
}
|
||||
}
|
||||
stats.reclassify();
|
||||
sync_state
|
||||
.set_library_tier(
|
||||
&self.server_id,
|
||||
&self.library_scope,
|
||||
stats.library_tier.as_tag(),
|
||||
)
|
||||
.map_err(SyncError::Storage)?;
|
||||
sync_state
|
||||
.set_poll_stats_json(
|
||||
&self.server_id,
|
||||
&self.library_scope,
|
||||
&serde_json::to_value(stats).unwrap_or_default(),
|
||||
)
|
||||
.map_err(SyncError::Storage)?;
|
||||
|
||||
report.next_poll_at_ms = now_ms + next_interval_ms(&stats) as i64;
|
||||
sync_state
|
||||
.set_next_poll_at(&self.server_id, &self.library_scope, report.next_poll_at_ms)
|
||||
.map_err(SyncError::Storage)?;
|
||||
|
||||
report.delta = Some(delta_report);
|
||||
Ok(report)
|
||||
}
|
||||
|
||||
fn load_poll_stats(
|
||||
&self,
|
||||
sync_state: &SyncStateRepository<'_>,
|
||||
) -> Result<PollStats, SyncError> {
|
||||
let raw = sync_state
|
||||
.get_poll_stats_json(&self.server_id, &self.library_scope)
|
||||
.map_err(SyncError::Storage)?;
|
||||
match raw {
|
||||
None => Ok(PollStats::default()),
|
||||
Some(v) => serde_json::from_value(v).map_err(|e| SyncError::Storage(e.to_string())),
|
||||
}
|
||||
}
|
||||
|
||||
/// True while initial sync, capability probe, IS-3 bulk ingest, or a
|
||||
/// foreground sync job for this server is in flight — background delta
|
||||
/// must not compete for HTTP budget or tombstone probes.
|
||||
fn sync_pass_active(&self, sync_state: &SyncStateRepository<'_>) -> Result<bool, SyncError> {
|
||||
if self.foreground_sync_job_active {
|
||||
return Ok(true);
|
||||
}
|
||||
if self.store.bulk_ingest_active() {
|
||||
return Ok(true);
|
||||
}
|
||||
let phase = sync_state
|
||||
.get_sync_phase(&self.server_id, &self.library_scope)
|
||||
.map_err(SyncError::Storage)?;
|
||||
Ok(matches!(
|
||||
phase.as_deref(),
|
||||
Some("initial_sync") | Some("probing")
|
||||
))
|
||||
}
|
||||
|
||||
fn count_local_tracks(&self) -> Result<i64, SyncError> {
|
||||
self.store
|
||||
.with_conn("scheduler.count_local_tracks", |c| {
|
||||
c.query_row(
|
||||
"SELECT COUNT(*) FROM track WHERE server_id = ?1 AND deleted = 0",
|
||||
rusqlite::params![self.server_id],
|
||||
|row| row.get(0),
|
||||
)
|
||||
})
|
||||
.map_err(SyncError::Storage)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use psysonic_integration::subsonic::{SubsonicClient, SubsonicCredentials};
|
||||
use serde_json::json;
|
||||
use wiremock::matchers::{method as wm_method, path as wm_path};
|
||||
use wiremock::{Mock, MockServer, ResponseTemplate};
|
||||
|
||||
fn test_subsonic(uri: &str) -> SubsonicClient {
|
||||
SubsonicClient::with_static_credentials(
|
||||
uri,
|
||||
SubsonicCredentials::with_static("user", "tok", "salt"),
|
||||
reqwest::Client::new(),
|
||||
)
|
||||
}
|
||||
|
||||
fn flags(bits: u32) -> CapabilityFlags {
|
||||
CapabilityFlags::new(bits)
|
||||
}
|
||||
|
||||
async fn empty_probe_and_albumlist(server: &MockServer, last_modified: i64) {
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getArtists.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": {
|
||||
"status": "ok",
|
||||
"artists": {
|
||||
"lastModified": last_modified,
|
||||
"ignoredArticles": "",
|
||||
"index": []
|
||||
}
|
||||
}
|
||||
})))
|
||||
.mount(server)
|
||||
.await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getAlbumList2.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": {
|
||||
"status": "ok",
|
||||
"albumList2": { "album": [] }
|
||||
}
|
||||
})))
|
||||
.mount(server)
|
||||
.await;
|
||||
}
|
||||
|
||||
// ── is_due ────────────────────────────────────────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn is_due_returns_true_when_no_schedule_yet() {
|
||||
let server = MockServer::start().await;
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let sched = BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
);
|
||||
assert!(sched.is_due(0).unwrap());
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn is_due_false_when_next_poll_in_future() {
|
||||
let server = MockServer::start().await;
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
sync_state.ensure("s1", "").unwrap();
|
||||
sync_state.set_next_poll_at("s1", "", 5_000_000).unwrap();
|
||||
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let sched = BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
);
|
||||
assert!(!sched.is_due(1_000_000).unwrap());
|
||||
assert!(sched.is_due(5_000_001).unwrap());
|
||||
}
|
||||
|
||||
// ── tick skips when not due ──────────────────────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn tick_skips_while_initial_sync_phase_active() {
|
||||
let server = MockServer::start().await;
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
sync_state.ensure("s1", "").unwrap();
|
||||
sync_state
|
||||
.set_sync_phase("s1", "", "initial_sync")
|
||||
.unwrap();
|
||||
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
)
|
||||
.with_sleep_disabled()
|
||||
.tick(0)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert!(report.skipped_sync_pass_active);
|
||||
assert!(report.delta.is_none());
|
||||
assert_eq!(report.next_poll_at_ms, 30_000);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn tick_skips_when_foreground_sync_job_active() {
|
||||
let server = MockServer::start().await;
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
sync_state.ensure("s1", "").unwrap();
|
||||
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
)
|
||||
.with_sleep_disabled()
|
||||
.with_foreground_sync_job_active(true)
|
||||
.tick(0)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert!(report.skipped_sync_pass_active);
|
||||
assert!(report.delta.is_none());
|
||||
assert_eq!(report.next_poll_at_ms, 30_000);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn tick_skips_when_not_due_and_reports_next_poll() {
|
||||
let server = MockServer::start().await;
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
sync_state.ensure("s1", "").unwrap();
|
||||
sync_state.set_next_poll_at("s1", "", 1_000_000_000).unwrap();
|
||||
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
)
|
||||
.with_sleep_disabled()
|
||||
.tick(500)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert!(report.skipped_not_due);
|
||||
assert!(report.delta.is_none());
|
||||
assert!(report.next_poll_at_ms > 500);
|
||||
}
|
||||
|
||||
// ── tick pauses when PrefetchActive ──────────────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn tick_pauses_when_playback_hint_is_prefetch_active() {
|
||||
let server = MockServer::start().await;
|
||||
let store = LibraryStore::open_in_memory();
|
||||
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
)
|
||||
.with_playback_hint(PlaybackHint::PrefetchActive)
|
||||
.with_sleep_disabled()
|
||||
.tick(0)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert!(report.skipped_bulk_paused);
|
||||
assert!(report.delta.is_none());
|
||||
// Re-scheduled soon (≤ 60s after now) so we catch the
|
||||
// prefetch finishing.
|
||||
assert!(report.next_poll_at_ms > 0);
|
||||
assert!(report.next_poll_at_ms <= 60_000);
|
||||
}
|
||||
|
||||
// ── tick runs delta and stamps next_poll_at ──────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn tick_runs_delta_and_persists_next_poll_at() {
|
||||
let server = MockServer::start().await;
|
||||
empty_probe_and_albumlist(&server, 1_716_840_000_000).await;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
)
|
||||
.with_sleep_disabled()
|
||||
.tick(1_000)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert!(!report.skipped_not_due);
|
||||
assert!(!report.skipped_bulk_paused);
|
||||
assert!(report.delta.is_some());
|
||||
let next = SyncStateRepository::new(&store)
|
||||
.get_next_poll_at("s1", "")
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
assert_eq!(next, report.next_poll_at_ms);
|
||||
assert!(next > 1_000);
|
||||
}
|
||||
|
||||
// ── auto-tombstone trigger ──────────────────────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn tick_auto_tombstones_when_count_gap_exceeds_threshold() {
|
||||
let server = MockServer::start().await;
|
||||
empty_probe_and_albumlist(&server, 1_716_840_000_000).await;
|
||||
// Tombstone probe — empty store has nothing to probe, so we
|
||||
// only need to know the runner *would* have called getSong if
|
||||
// there were rows. For this test it's enough that no panic
|
||||
// occurs and the delta report's tombstone counters are zero.
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
sync_state.ensure("s1", "").unwrap();
|
||||
// 110 local vs 100 server → 10 % gap, threshold 5 % default.
|
||||
sync_state.set_local_track_count("s1", "", 110).unwrap();
|
||||
sync_state.set_server_track_count("s1", "", 100).unwrap();
|
||||
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
)
|
||||
.with_sleep_disabled()
|
||||
.tick(0)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let delta = report.delta.expect("delta ran");
|
||||
// Tombstone budget was set (200), but no local tracks exist →
|
||||
// nothing to probe, both counters stay at 0. The important
|
||||
// signal is that the runner accepted the trigger.
|
||||
assert_eq!(delta.tombstones_checked, 0);
|
||||
assert_eq!(delta.tombstones_deleted, 0);
|
||||
}
|
||||
|
||||
// ── PollStats persistence round trip ────────────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn poll_stats_persist_round_trip_through_tick() {
|
||||
let server = MockServer::start().await;
|
||||
empty_probe_and_albumlist(&server, 1_716_840_000_000).await;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
)
|
||||
.with_sleep_disabled()
|
||||
.tick(0)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let stored = SyncStateRepository::new(&store)
|
||||
.get_poll_stats_json("s1", "")
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
// tier is recorded — runner reclassifies even with no
|
||||
// observations yet, so this is "unknown" on a fresh store.
|
||||
let stats: PollStats = serde_json::from_value(stored).unwrap();
|
||||
assert_eq!(stats.library_tier.as_tag(), "unknown");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,216 @@
|
||||
//! C2 ingest strategy selection. Per the PR-3 kickoff answer (workdocs
|
||||
//! `2026-05-19-pr3-kickoff.md` Q3) the choice is made once at initial
|
||||
//! sync start from the probed capability flags; the runner does not
|
||||
//! auto-switch on transient failure (C12 retries the same batch).
|
||||
|
||||
use super::capability::CapabilityFlags;
|
||||
|
||||
/// Server track count above which N1 is skipped in favour of S1 at initial
|
||||
/// sync start (R7-15 Q4). N1's native `/api/song` deep-offset 500 wall makes
|
||||
/// it unable to finish very large catalogs; S1 (`search3`) does not hit it.
|
||||
/// Tunable constant — the live wall was observed past ~50k.
|
||||
pub const LARGE_LIBRARY_THRESHOLD: i64 = 40_000;
|
||||
|
||||
/// Spec §6.3 IS-3 strategies. Names match §6.1.1 capability bits where
|
||||
/// applicable; S2 has no flag of its own — it's the universal
|
||||
/// album-crawl fallback assumed available whenever the Subsonic ping
|
||||
/// succeeds.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
|
||||
pub enum IngestStrategy {
|
||||
/// N1 — Navidrome native `GET /api/song` paginated. Cheapest at
|
||||
/// 500k; requires `NavidromeNativeBulk` flag set by the probe.
|
||||
N1,
|
||||
/// S1 — Subsonic `search3` empty query, songOffset paged. Requires
|
||||
/// `SubsonicSearch3Bulk`.
|
||||
S1,
|
||||
/// S2 — `getAlbumList2` + `getAlbum` per album. Universal Subsonic
|
||||
/// fallback — assumed available whenever the ping returns ok.
|
||||
S2,
|
||||
/// S3 — `getIndexes` + `getMusicDirectory` recursive file-tree
|
||||
/// crawl. Last resort; PR-3b does not auto-select it.
|
||||
S3,
|
||||
}
|
||||
|
||||
impl IngestStrategy {
|
||||
/// Pick the cheapest strategy supported by `flags`. Per kickoff Q3:
|
||||
/// `N1 → S1 → S2`. S3 is enumerated for completeness but never
|
||||
/// auto-selected — when neither N1 nor S1 is available, S2 is
|
||||
/// always tried first because every Subsonic-compliant server
|
||||
/// exposes `getAlbumList2` + `getAlbum`.
|
||||
pub fn select_from_flags(flags: CapabilityFlags) -> Self {
|
||||
if flags.contains(CapabilityFlags::NAVIDROME_NATIVE_BULK) {
|
||||
Self::N1
|
||||
} else if flags.contains(CapabilityFlags::SUBSONIC_SEARCH3_BULK) {
|
||||
Self::S1
|
||||
} else {
|
||||
Self::S2
|
||||
}
|
||||
}
|
||||
|
||||
/// Pick the initial-sync strategy with the large-library policy on top
|
||||
/// of `select_from_flags` (R7-15 Q4). A large catalog — or one already
|
||||
/// flagged `n1_bulk_unreliable` after an N1 deep-offset 500 — must avoid
|
||||
/// N1, which cannot finish past the wall. Prefer S1 (`search3`) and fall
|
||||
/// back to S2 (universal album crawl) when search3 bulk is unavailable.
|
||||
///
|
||||
/// `server_track_count` is best-effort at IS-1 (probe `getScanStatus`
|
||||
/// count or a prior watermark); `None` means unknown, in which case only
|
||||
/// the `n1_bulk_unreliable` flag forces the non-N1 path (a first run with
|
||||
/// no count still tries the cheapest strategy and learns from a 500).
|
||||
pub fn select_initial_strategy(
|
||||
flags: CapabilityFlags,
|
||||
server_track_count: Option<i64>,
|
||||
n1_bulk_unreliable: bool,
|
||||
) -> Self {
|
||||
let is_large = server_track_count
|
||||
.map(|c| c > LARGE_LIBRARY_THRESHOLD)
|
||||
.unwrap_or(false);
|
||||
if n1_bulk_unreliable || is_large {
|
||||
if flags.contains(CapabilityFlags::SUBSONIC_SEARCH3_BULK) {
|
||||
return Self::S1;
|
||||
}
|
||||
return Self::S2;
|
||||
}
|
||||
Self::select_from_flags(flags)
|
||||
}
|
||||
|
||||
/// String tag stored in `initial_sync_cursor_json` so the runner
|
||||
/// can resume after restart without re-running capability probe.
|
||||
pub fn as_tag(self) -> &'static str {
|
||||
match self {
|
||||
Self::N1 => "n1",
|
||||
Self::S1 => "s1",
|
||||
Self::S2 => "s2",
|
||||
Self::S3 => "s3",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn from_tag(tag: &str) -> Option<Self> {
|
||||
match tag {
|
||||
"n1" => Some(Self::N1),
|
||||
"s1" => Some(Self::S1),
|
||||
"s2" => Some(Self::S2),
|
||||
"s3" => Some(Self::S3),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn select_prefers_navidrome_native_when_both_n1_and_s1_present() {
|
||||
// At 500k, N1 cuts request count by 10× vs S1 — selector must
|
||||
// pick it whenever the flag is set, regardless of S1.
|
||||
let flags = CapabilityFlags::new(
|
||||
CapabilityFlags::NAVIDROME_NATIVE_BULK | CapabilityFlags::SUBSONIC_SEARCH3_BULK,
|
||||
);
|
||||
assert_eq!(IngestStrategy::select_from_flags(flags), IngestStrategy::N1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn select_falls_back_to_s1_without_n1() {
|
||||
let flags = CapabilityFlags::new(CapabilityFlags::SUBSONIC_SEARCH3_BULK);
|
||||
assert_eq!(IngestStrategy::select_from_flags(flags), IngestStrategy::S1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn select_falls_back_to_s2_when_no_bulk_flag_set() {
|
||||
// Generic Subsonic server without search3 bulk → universal
|
||||
// album crawl. S3 is not auto-selected even with FileTreeBrowse.
|
||||
let flags = CapabilityFlags::new(CapabilityFlags::FILE_TREE_BROWSE);
|
||||
assert_eq!(IngestStrategy::select_from_flags(flags), IngestStrategy::S2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn select_falls_back_to_s2_with_no_flags() {
|
||||
// Default-flag (`0x000`) — fresh DB before probe runs, or a
|
||||
// truly minimal Subsonic implementation. Still resolves to a
|
||||
// strategy; runner surfaces errors if S2 endpoints then fail.
|
||||
assert_eq!(
|
||||
IngestStrategy::select_from_flags(CapabilityFlags::default()),
|
||||
IngestStrategy::S2
|
||||
);
|
||||
}
|
||||
|
||||
// ── select_initial_strategy (R7-15 Q4 large-library policy) ──────────
|
||||
|
||||
fn navidrome_full() -> CapabilityFlags {
|
||||
CapabilityFlags::new(
|
||||
CapabilityFlags::NAVIDROME_NATIVE_BULK | CapabilityFlags::SUBSONIC_SEARCH3_BULK,
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn initial_strategy_small_library_keeps_cheapest_n1() {
|
||||
// Below threshold, not flagged unreliable → unchanged N1-first chain.
|
||||
let s = IngestStrategy::select_initial_strategy(navidrome_full(), Some(1_000), false);
|
||||
assert_eq!(s, IngestStrategy::N1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn initial_strategy_large_library_avoids_n1_for_s1() {
|
||||
// Over threshold → S1 even though N1 is advertised (deep-offset wall).
|
||||
let s = IngestStrategy::select_initial_strategy(navidrome_full(), Some(170_000), false);
|
||||
assert_eq!(s, IngestStrategy::S1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn initial_strategy_unreliable_flag_avoids_n1_regardless_of_size() {
|
||||
// Learned `n1_bulk_unreliable` forces the non-N1 path even when the
|
||||
// count is small/unknown (covers R7-15 "unknown → large if N1 failed").
|
||||
let small =
|
||||
IngestStrategy::select_initial_strategy(navidrome_full(), Some(500), true);
|
||||
assert_eq!(small, IngestStrategy::S1);
|
||||
let unknown = IngestStrategy::select_initial_strategy(navidrome_full(), None, true);
|
||||
assert_eq!(unknown, IngestStrategy::S1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn initial_strategy_large_without_search3_falls_back_to_s2() {
|
||||
// Avoid-N1 path but no search3 bulk → universal album crawl, not N1.
|
||||
let flags = CapabilityFlags::new(CapabilityFlags::NAVIDROME_NATIVE_BULK);
|
||||
let s = IngestStrategy::select_initial_strategy(flags, Some(170_000), false);
|
||||
assert_eq!(s, IngestStrategy::S2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn initial_strategy_unknown_count_uses_cheapest_when_not_flagged() {
|
||||
// First run, no count yet, N1 never failed → try cheapest (N1); the
|
||||
// mid-run N1→S1 fallback (R7-15 Q5) handles the wall if hit.
|
||||
let s = IngestStrategy::select_initial_strategy(navidrome_full(), None, false);
|
||||
assert_eq!(s, IngestStrategy::N1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn initial_strategy_threshold_is_strictly_greater_than() {
|
||||
// Exactly at the threshold is not "large"; one above is.
|
||||
let at = IngestStrategy::select_initial_strategy(
|
||||
navidrome_full(),
|
||||
Some(LARGE_LIBRARY_THRESHOLD),
|
||||
false,
|
||||
);
|
||||
assert_eq!(at, IngestStrategy::N1);
|
||||
let over = IngestStrategy::select_initial_strategy(
|
||||
navidrome_full(),
|
||||
Some(LARGE_LIBRARY_THRESHOLD + 1),
|
||||
false,
|
||||
);
|
||||
assert_eq!(over, IngestStrategy::S1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tag_roundtrip_is_stable_for_cursor_persistence() {
|
||||
for s in [
|
||||
IngestStrategy::N1,
|
||||
IngestStrategy::S1,
|
||||
IngestStrategy::S2,
|
||||
IngestStrategy::S3,
|
||||
] {
|
||||
assert_eq!(IngestStrategy::from_tag(s.as_tag()), Some(s));
|
||||
}
|
||||
assert_eq!(IngestStrategy::from_tag("unknown"), None);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,166 @@
|
||||
//! C5 — `SyncSupervisor`. Wraps the spawn / cancel / join lifecycle
|
||||
//! the top crate (PR-5) will use to run an `InitialSyncRunner` or
|
||||
//! `DeltaSyncRunner` from a Tauri command. The supervisor owns the
|
||||
//! cancellation `AtomicBool` and the `mpsc` receiver carrying
|
||||
//! progress events.
|
||||
//!
|
||||
//! Stays pure library — no Tauri imports here; PR-5 hooks the
|
||||
//! receiver to `AppHandle::emit`.
|
||||
|
||||
use std::sync::atomic::AtomicBool;
|
||||
use std::sync::atomic::Ordering;
|
||||
use std::sync::Arc;
|
||||
|
||||
use super::error::SyncError;
|
||||
use super::progress::{ChannelProgress, Progress, ProgressEvent};
|
||||
|
||||
pub struct SyncSupervisor {
|
||||
cancel: Arc<AtomicBool>,
|
||||
handle: Option<tokio::task::JoinHandle<Result<(), SyncError>>>,
|
||||
progress_rx: Option<tokio::sync::mpsc::UnboundedReceiver<ProgressEvent>>,
|
||||
}
|
||||
|
||||
impl SyncSupervisor {
|
||||
/// Spawn a sync workload. `task` receives the cancellation flag
|
||||
/// and a `Progress` handle backed by an internal mpsc channel.
|
||||
/// Caller drives the receiver via `progress_receiver()` and waits
|
||||
/// for completion via `join().await`.
|
||||
pub fn spawn<F, Fut>(task: F) -> Self
|
||||
where
|
||||
F: FnOnce(Arc<AtomicBool>, Arc<dyn Progress + Send + Sync>) -> Fut + Send + 'static,
|
||||
Fut: std::future::Future<Output = Result<(), SyncError>> + Send + 'static,
|
||||
{
|
||||
Self::spawn_with_interval(task, ChannelProgress::DEFAULT_INTERVAL)
|
||||
}
|
||||
|
||||
/// Variant that overrides the throttle interval — tests pass
|
||||
/// `Duration::ZERO` so every event observed in
|
||||
/// `progress_receiver()` matches the runner's emit order.
|
||||
pub fn spawn_with_interval<F, Fut>(task: F, throttle: std::time::Duration) -> Self
|
||||
where
|
||||
F: FnOnce(Arc<AtomicBool>, Arc<dyn Progress + Send + Sync>) -> Fut + Send + 'static,
|
||||
Fut: std::future::Future<Output = Result<(), SyncError>> + Send + 'static,
|
||||
{
|
||||
let cancel = Arc::new(AtomicBool::new(false));
|
||||
let (tx, rx) = tokio::sync::mpsc::unbounded_channel();
|
||||
let progress: Arc<dyn Progress + Send + Sync> =
|
||||
Arc::new(ChannelProgress::with_interval(tx, throttle));
|
||||
let cancel_clone = Arc::clone(&cancel);
|
||||
let handle = tokio::task::spawn(async move { task(cancel_clone, progress).await });
|
||||
Self {
|
||||
cancel,
|
||||
handle: Some(handle),
|
||||
progress_rx: Some(rx),
|
||||
}
|
||||
}
|
||||
|
||||
/// Trip the cancellation flag. Runners check it between batches
|
||||
/// and bail out with `SyncError::Cancelled` on the next iteration.
|
||||
pub fn cancel(&self) {
|
||||
self.cancel.store(true, Ordering::SeqCst);
|
||||
}
|
||||
|
||||
pub fn is_cancelled(&self) -> bool {
|
||||
self.cancel.load(Ordering::SeqCst)
|
||||
}
|
||||
|
||||
/// Take the progress receiver. Single consumer only — the
|
||||
/// `Option::take` semantics mean later calls return `None`.
|
||||
pub fn progress_receiver(
|
||||
&mut self,
|
||||
) -> Option<tokio::sync::mpsc::UnboundedReceiver<ProgressEvent>> {
|
||||
self.progress_rx.take()
|
||||
}
|
||||
|
||||
/// Wait for the spawned task. Returns the task's `Result` —
|
||||
/// `Ok(Ok(()))` on clean exit, `Ok(Err(SyncError))` on a
|
||||
/// runner-level failure, `Err(JoinError)` only if the task
|
||||
/// panicked, in which case we surface that as `SyncError::Storage`
|
||||
/// so callers never need to know about tokio internals.
|
||||
pub async fn join(mut self) -> Result<(), SyncError> {
|
||||
let handle = self
|
||||
.handle
|
||||
.take()
|
||||
.ok_or_else(|| SyncError::Storage("supervisor already joined".into()))?;
|
||||
match handle.await {
|
||||
Ok(inner) => inner,
|
||||
Err(join_err) => Err(SyncError::Storage(format!(
|
||||
"sync task panicked: {join_err}"
|
||||
))),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::time::Duration;
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn supervisor_runs_task_to_completion_and_emits_progress() {
|
||||
let mut sup = SyncSupervisor::spawn_with_interval(
|
||||
|_cancel, progress| async move {
|
||||
progress.emit(ProgressEvent::PhaseChanged { phase: "ingest".into() });
|
||||
progress.emit(ProgressEvent::Completed {
|
||||
kind: "initial_sync".into(),
|
||||
});
|
||||
Ok(())
|
||||
},
|
||||
Duration::ZERO,
|
||||
);
|
||||
let mut rx = sup.progress_receiver().expect("receiver still in place");
|
||||
let result = sup.join().await;
|
||||
assert!(result.is_ok());
|
||||
let mut events = Vec::new();
|
||||
while let Ok(ev) = rx.try_recv() {
|
||||
events.push(ev);
|
||||
}
|
||||
assert_eq!(events.len(), 2);
|
||||
assert!(matches!(events[0], ProgressEvent::PhaseChanged { .. }));
|
||||
assert!(matches!(events[1], ProgressEvent::Completed { .. }));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn supervisor_cancel_trips_atomic_flag_before_task_joins() {
|
||||
let started = Arc::new(tokio::sync::Notify::new());
|
||||
let started_clone = Arc::clone(&started);
|
||||
let sup = SyncSupervisor::spawn(move |cancel, _progress| {
|
||||
let started = started_clone;
|
||||
async move {
|
||||
started.notify_one();
|
||||
// Spin until cancelled or 2s timeout.
|
||||
for _ in 0..200 {
|
||||
if cancel.load(Ordering::SeqCst) {
|
||||
return Err(SyncError::Cancelled);
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(10)).await;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
});
|
||||
started.notified().await;
|
||||
sup.cancel();
|
||||
assert!(sup.is_cancelled());
|
||||
let result = sup.join().await;
|
||||
assert!(matches!(result, Err(SyncError::Cancelled)));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn supervisor_propagates_task_panic_as_sync_error() {
|
||||
let sup = SyncSupervisor::spawn(|_cancel, _progress| async move {
|
||||
panic!("simulated task panic");
|
||||
});
|
||||
let result = sup.join().await;
|
||||
assert!(matches!(result, Err(SyncError::Storage(_))));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn progress_receiver_take_returns_none_after_first_call() {
|
||||
let mut sup = SyncSupervisor::spawn(|_cancel, _progress| async move { Ok(()) });
|
||||
let first = sup.progress_receiver();
|
||||
let second = sup.progress_receiver();
|
||||
assert!(first.is_some());
|
||||
assert!(second.is_none());
|
||||
sup.join().await.unwrap();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,447 @@
|
||||
//! C4 — `TombstoneReconciler` (spec §6.7).
|
||||
//!
|
||||
//! Streams a chunk of local track ids, hits `getSong` per id, and
|
||||
//! marks `track.deleted = 1` for every `SubsonicError::NotFound`
|
||||
//! (error code 70). Designed for two callers:
|
||||
//!
|
||||
//! - **Mode A (manual integrity check):** Settings → "Verify library
|
||||
//! integrity" loops `reconcile_chunk(N)` until it returns
|
||||
//! `checked == 0`.
|
||||
//! - **Mode B (auto, threshold-triggered):** the delta scheduler
|
||||
//! tests `should_auto_reconcile` against the count drop, then loops
|
||||
//! `reconcile_chunk(budget)` once per delta tick until the gap
|
||||
//! closes.
|
||||
//!
|
||||
//! Streaming so memory stays bounded at 500k: `LIMIT N ORDER BY
|
||||
//! synced_at ASC` picks the next chunk; PR-3c keeps the loop entirely
|
||||
//! caller-driven so cancellation is checked between chunks.
|
||||
|
||||
use std::sync::atomic::Ordering;
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use psysonic_integration::subsonic::{SubsonicClient, SubsonicError};
|
||||
|
||||
use super::backoff::{jitter_salt, with_jitter, Backoff};
|
||||
use super::error::SyncError;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
const MAX_ATTEMPTS_PER_BATCH: u32 = 5;
|
||||
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq)]
|
||||
pub struct TombstoneReport {
|
||||
pub checked: u32,
|
||||
pub deleted: u32,
|
||||
}
|
||||
|
||||
pub struct TombstoneReconciler<'a> {
|
||||
store: &'a LibraryStore,
|
||||
subsonic: &'a SubsonicClient,
|
||||
server_id: String,
|
||||
cancel: Option<Arc<std::sync::atomic::AtomicBool>>,
|
||||
sleep_enabled: bool,
|
||||
}
|
||||
|
||||
impl<'a> TombstoneReconciler<'a> {
|
||||
pub fn new(
|
||||
store: &'a LibraryStore,
|
||||
subsonic: &'a SubsonicClient,
|
||||
server_id: impl Into<String>,
|
||||
) -> Self {
|
||||
Self {
|
||||
store,
|
||||
subsonic,
|
||||
server_id: server_id.into(),
|
||||
cancel: None,
|
||||
sleep_enabled: true,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn with_cancellation(mut self, flag: Arc<std::sync::atomic::AtomicBool>) -> Self {
|
||||
self.cancel = Some(flag);
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_sleep_disabled(mut self) -> Self {
|
||||
self.sleep_enabled = false;
|
||||
self
|
||||
}
|
||||
|
||||
/// Process up to `budget` not-yet-checked tracks. Returns counts
|
||||
/// for this call only — caller loops until `checked == 0` to
|
||||
/// complete a Mode A pass, or stops at any budget for Mode B
|
||||
/// sampled passes. Order: oldest `synced_at` first so the most
|
||||
/// stale rows get re-validated soonest.
|
||||
pub async fn reconcile_chunk(&self, budget: u32) -> Result<TombstoneReport, SyncError> {
|
||||
if budget == 0 {
|
||||
return Ok(TombstoneReport::default());
|
||||
}
|
||||
let ids = self.next_candidates(budget)?;
|
||||
let mut report = TombstoneReport::default();
|
||||
for id in ids {
|
||||
self.check_cancellation()?;
|
||||
report.checked = report.checked.saturating_add(1);
|
||||
let outcome = retry_with_backoff(
|
||||
self,
|
||||
|| self.subsonic.get_song(&id),
|
||||
|e: SubsonicError| -> SyncError { e.into() },
|
||||
)
|
||||
.await;
|
||||
match outcome {
|
||||
Ok(_) => {
|
||||
// Still present — stamp `synced_at` so it goes to
|
||||
// the back of the queue and we don't re-probe it
|
||||
// again on the next chunk.
|
||||
self.mark_synced(&id)?;
|
||||
}
|
||||
Err(SyncError::NotFound) => {
|
||||
self.mark_deleted(&id)?;
|
||||
report.deleted = report.deleted.saturating_add(1);
|
||||
}
|
||||
Err(other) => return Err(other),
|
||||
}
|
||||
}
|
||||
Ok(report)
|
||||
}
|
||||
|
||||
fn check_cancellation(&self) -> Result<(), SyncError> {
|
||||
if let Some(flag) = &self.cancel {
|
||||
if flag.load(Ordering::SeqCst) {
|
||||
return Err(SyncError::Cancelled);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn next_candidates(&self, budget: u32) -> Result<Vec<String>, SyncError> {
|
||||
self.store
|
||||
.with_conn("misc", |c| {
|
||||
let mut stmt = c.prepare(
|
||||
"SELECT id FROM track \
|
||||
WHERE server_id = ?1 AND deleted = 0 \
|
||||
ORDER BY synced_at ASC LIMIT ?2",
|
||||
)?;
|
||||
let rows: rusqlite::Result<Vec<String>> = stmt
|
||||
.query_map(rusqlite::params![self.server_id, budget as i64], |r| {
|
||||
r.get::<_, String>(0)
|
||||
})?
|
||||
.collect();
|
||||
rows
|
||||
})
|
||||
.map_err(SyncError::Storage)
|
||||
}
|
||||
|
||||
fn mark_deleted(&self, id: &str) -> Result<(), SyncError> {
|
||||
self.store
|
||||
.with_conn("misc", |c| {
|
||||
c.execute(
|
||||
"UPDATE track SET deleted = 1, synced_at = ?3 \
|
||||
WHERE server_id = ?1 AND id = ?2",
|
||||
rusqlite::params![self.server_id, id, now_unix_ms()],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
.map_err(SyncError::Storage)
|
||||
}
|
||||
|
||||
fn mark_synced(&self, id: &str) -> Result<(), SyncError> {
|
||||
self.store
|
||||
.with_conn("misc", |c| {
|
||||
c.execute(
|
||||
"UPDATE track SET synced_at = ?3 \
|
||||
WHERE server_id = ?1 AND id = ?2",
|
||||
rusqlite::params![self.server_id, id, now_unix_ms()],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
.map_err(SyncError::Storage)
|
||||
}
|
||||
|
||||
async fn sleep(&self, d: Duration) {
|
||||
if self.sleep_enabled && !d.is_zero() {
|
||||
tokio::time::sleep(d).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// §6.7 Mode B threshold check — returns `true` when the local /
|
||||
/// server count gap exceeds the configured percentage. `server_count
|
||||
/// == 0` is treated as "no signal" → `false` (no spurious reconcile
|
||||
/// on a fresh server response).
|
||||
pub fn should_auto_reconcile(local_count: u32, server_count: u32, threshold_pct: u32) -> bool {
|
||||
if server_count == 0 {
|
||||
return false;
|
||||
}
|
||||
let gap = local_count.saturating_sub(server_count);
|
||||
let ratio_x100 = gap.saturating_mul(100) / server_count;
|
||||
ratio_x100 > threshold_pct
|
||||
}
|
||||
|
||||
fn now_unix_ms() -> i64 {
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.map(|d| d.as_millis().min(i64::MAX as u128) as i64)
|
||||
.unwrap_or(0)
|
||||
}
|
||||
|
||||
async fn retry_with_backoff<'a, F, FFut, T, E>(
|
||||
reconciler: &TombstoneReconciler<'a>,
|
||||
mut build: F,
|
||||
map_err: impl Fn(E) -> SyncError,
|
||||
) -> Result<T, SyncError>
|
||||
where
|
||||
F: FnMut() -> FFut,
|
||||
FFut: std::future::Future<Output = Result<T, E>>,
|
||||
{
|
||||
let mut backoff = Backoff::default();
|
||||
let mut attempt = 0u32;
|
||||
loop {
|
||||
reconciler.check_cancellation()?;
|
||||
attempt += 1;
|
||||
match build().await {
|
||||
Ok(v) => return Ok(v),
|
||||
Err(e) => {
|
||||
let mapped = map_err(e);
|
||||
if !is_retryable(&mapped) || attempt >= MAX_ATTEMPTS_PER_BATCH {
|
||||
return Err(mapped);
|
||||
}
|
||||
let delay = backoff.next_delay();
|
||||
let jittered = with_jitter(delay, jitter_salt(attempt));
|
||||
reconciler.sleep(jittered).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn is_retryable(e: &SyncError) -> bool {
|
||||
matches!(e, SyncError::Transport(_) | SyncError::Navidrome(_))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::repos::{TrackRepository, TrackRow};
|
||||
use psysonic_integration::subsonic::{SubsonicClient, SubsonicCredentials};
|
||||
use serde_json::json;
|
||||
use wiremock::matchers::{method as wm_method, path as wm_path, query_param};
|
||||
use wiremock::{Mock, MockServer, ResponseTemplate};
|
||||
|
||||
fn test_subsonic(uri: &str) -> SubsonicClient {
|
||||
SubsonicClient::with_static_credentials(
|
||||
uri,
|
||||
SubsonicCredentials::with_static("user", "tok", "salt"),
|
||||
reqwest::Client::new(),
|
||||
)
|
||||
}
|
||||
|
||||
fn seed_track(store: &LibraryStore, id: &str, synced_at: i64) {
|
||||
TrackRepository::new(store)
|
||||
.upsert_batch(&[TrackRow {
|
||||
server_id: "s1".into(),
|
||||
id: id.into(),
|
||||
title: id.into(),
|
||||
title_sort: None,
|
||||
artist: None,
|
||||
artist_id: None,
|
||||
album: String::new(),
|
||||
album_id: None,
|
||||
album_artist: None,
|
||||
duration_sec: 0,
|
||||
track_number: None,
|
||||
disc_number: None,
|
||||
year: None,
|
||||
genre: None,
|
||||
suffix: None,
|
||||
bit_rate: None,
|
||||
size_bytes: None,
|
||||
cover_art_id: None,
|
||||
starred_at: None,
|
||||
user_rating: None,
|
||||
play_count: None,
|
||||
played_at: None,
|
||||
server_path: None,
|
||||
library_id: None,
|
||||
isrc: None,
|
||||
mbid_recording: None,
|
||||
bpm: None,
|
||||
replay_gain_track_db: None,
|
||||
replay_gain_album_db: None,
|
||||
content_hash: None,
|
||||
server_updated_at: None,
|
||||
server_created_at: None,
|
||||
deleted: false,
|
||||
synced_at,
|
||||
raw_json: "{}".into(),
|
||||
}])
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
// ── should_auto_reconcile threshold predicate ─────────────────────
|
||||
|
||||
#[test]
|
||||
fn threshold_fires_when_local_outpaces_server_above_pct() {
|
||||
// 110 local vs 100 server → 10% gap > 5% threshold.
|
||||
assert!(should_auto_reconcile(110, 100, 5));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn threshold_stays_silent_within_tolerance() {
|
||||
// 102 local vs 100 server → 2% gap, threshold 5%.
|
||||
assert!(!should_auto_reconcile(102, 100, 5));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn threshold_silent_when_local_is_below_or_equal_server() {
|
||||
assert!(!should_auto_reconcile(100, 100, 0));
|
||||
assert!(!should_auto_reconcile(50, 100, 5));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn threshold_silent_when_server_count_is_zero() {
|
||||
// No signal — never reconcile on a server that's still scanning.
|
||||
assert!(!should_auto_reconcile(1000, 0, 5));
|
||||
}
|
||||
|
||||
// ── reconcile_chunk marks deleted on code 70 ─────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn reconcile_chunk_marks_deleted_for_code_70() {
|
||||
let server = MockServer::start().await;
|
||||
// tr_a → still present, tr_b → 404 via code 70.
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getSong.view"))
|
||||
.and(query_param("id", "tr_a"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": {
|
||||
"status": "ok",
|
||||
"song": { "id": "tr_a", "title": "Still here" }
|
||||
}
|
||||
})))
|
||||
.mount(&server)
|
||||
.await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getSong.view"))
|
||||
.and(query_param("id", "tr_b"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": {
|
||||
"status": "failed",
|
||||
"error": { "code": 70, "message": "Song not found" }
|
||||
}
|
||||
})))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "tr_a", 1);
|
||||
seed_track(&store, "tr_b", 2);
|
||||
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = TombstoneReconciler::new(&store, &subsonic, "s1")
|
||||
.with_sleep_disabled()
|
||||
.reconcile_chunk(10)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(report.checked, 2);
|
||||
assert_eq!(report.deleted, 1);
|
||||
|
||||
// tr_b is marked deleted; tr_a stays live but its synced_at is
|
||||
// refreshed (so it doesn't get re-picked immediately).
|
||||
let (a_deleted, b_deleted): (i64, i64) = store
|
||||
.with_conn("misc", |c| {
|
||||
let a: i64 = c.query_row(
|
||||
"SELECT deleted FROM track WHERE id='tr_a'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)?;
|
||||
let b: i64 = c.query_row(
|
||||
"SELECT deleted FROM track WHERE id='tr_b'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)?;
|
||||
Ok((a, b))
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(a_deleted, 0);
|
||||
assert_eq!(b_deleted, 1);
|
||||
}
|
||||
|
||||
// ── reconcile_chunk respects budget and ordering ─────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn reconcile_chunk_processes_oldest_first_up_to_budget() {
|
||||
let server = MockServer::start().await;
|
||||
// Any id → ok envelope.
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getSong.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": {
|
||||
"status": "ok",
|
||||
"song": { "id": "any", "title": "t" }
|
||||
}
|
||||
})))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
// Seed three tracks with distinct synced_at values; oldest first.
|
||||
seed_track(&store, "tr_oldest", 100);
|
||||
seed_track(&store, "tr_middle", 200);
|
||||
seed_track(&store, "tr_newest", 300);
|
||||
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = TombstoneReconciler::new(&store, &subsonic, "s1")
|
||||
.with_sleep_disabled()
|
||||
.reconcile_chunk(2)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(report.checked, 2);
|
||||
|
||||
// After the chunk: the two checked rows have a refreshed
|
||||
// synced_at; the un-checked tr_newest still sits at 300.
|
||||
let untouched: i64 = store
|
||||
.with_conn("misc", |c| c.query_row(
|
||||
"SELECT synced_at FROM track WHERE id='tr_newest'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
))
|
||||
.unwrap();
|
||||
assert_eq!(untouched, 300, "tr_newest must not be probed within budget=2");
|
||||
}
|
||||
|
||||
// ── reconcile_chunk: empty store ───────────────────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn reconcile_chunk_returns_zero_counts_on_empty_store() {
|
||||
let server = MockServer::start().await;
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = TombstoneReconciler::new(&store, &subsonic, "s1")
|
||||
.with_sleep_disabled()
|
||||
.reconcile_chunk(50)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(report.checked, 0);
|
||||
assert_eq!(report.deleted, 0);
|
||||
}
|
||||
|
||||
// ── reconcile_chunk: cancellation ─────────────────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn reconcile_chunk_returns_cancelled_when_flag_tripped() {
|
||||
let server = MockServer::start().await;
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "tr_x", 1);
|
||||
|
||||
let flag = Arc::new(std::sync::atomic::AtomicBool::new(true));
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let err = TombstoneReconciler::new(&store, &subsonic, "s1")
|
||||
.with_cancellation(flag)
|
||||
.with_sleep_disabled()
|
||||
.reconcile_chunk(10)
|
||||
.await
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, SyncError::Cancelled));
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user