mirror of
https://github.com/kilyabin/psysonic.git
synced 2026-07-22 22:45:41 +00:00
feat(library): local library index and search (preview) (#846)
* feat(library): scaffold psysonic-library crate with v1 schema and store (#791) Adds a new workspace crate that will host the unified track store and the upcoming sync engine. PR-1a covers spec phases A1–A6: - migrations/001_initial.sql: full v1 schema — sync_state, track, album, artist, track_fts (+ ai/ad/au triggers), track_extension, track_offline, track_id_history, track_fact, track_artifact, canonical_track, canonical_identity, track_canonical_link, canonical_enrichment_link, and all §5.2 partial indexes. - store::LibraryStore: WAL + foreign_keys=ON SQLite connection rooted at app_data_dir/library.sqlite (distinct from the analysis cache, which uses app_config_dir). schema_migrations table + idempotent embedded migration runner; LIBRARY_DB_SCHEMA_VERSION = 1. - repos::TrackRepository::upsert_batch: 35-column transactional upsert with ON CONFLICT(server_id, id) all-fields rewrite; FTS rows follow via the triggers. - search::search_tracks: minimal bm25-ordered FTS5 helper scoped to a single server_id, filtering deleted rows. - filter::FilterFieldRegistry: static v1 registry (text, genre, year, starred = V1; bpm = SchemaV1UiLater; user_rating/suffix/bit_rate = Planned). Entity routing is a silent skip per §5.13.3. No Tauri commands, no frontend, no sync — those land in PR-2..PR-7. PR-1b will follow with the migration-runner edge-case tests, the initial_sync_cursor_json read/write API, and the breaking-migration hook stub (P22). * feat(library): A7 migration-runner safety net + initial-sync cursor API (PR-1b) (#792) * feat(library): wire migration-runner safety net and initial-sync cursor API PR-1b — Phase A7 infrastructure on top of PR-1a. Production behaviour is unchanged at v1 launch; everything here is plumbing that PR-3 will consume. - store::run_migrations_with: testable entry point that takes an explicit migration slice, a min-compatible-version threshold, and a breaking-bump hook. The prod `run_migrations` fixes those to MIGRATIONS, LIBRARY_DB_MIN_COMPATIBLE_VERSION, and the no-op stub. The slice is now sorted defensively before applying. - store::LIBRARY_DB_MIN_COMPATIBLE_VERSION: new public constant (currently equal to LIBRARY_DB_SCHEMA_VERSION). When a future release needs to invalidate v1 data, bumping this above the max applied version trips the hook on next open per spec §5.7 / P22. - store::MigrationOutcome (Applied | BreakingBump): crate-internal signal callers can branch on. PR-1b consumers ignore it; PR-3 / Settings will surface the "library rebuilt after update" toast when it surfaces. - store::handle_breaking_schema_bump: documented no-op stub. The drop + resync logic lands with the first real breaking bump. - repos::SyncStateRepository: ensure(server_id, scope) idempotently inserts a default row; get_initial_sync_cursor / set_initial_sync_cursor read and write sync_state.initial_sync_cursor_json via serde_json::Value. The set uses ON CONFLICT … DO UPDATE scoped to the cursor column only, so phase / poll-stats / tier survive cursor writes intact. - Tests cover: additive 002-style migration preserves prior data (spec §5.7 explicit integration test), runner sorts an unsorted source slice, breaking-bump hook fires when max applied < min_compatible, hook does not fire on a fresh DB, cursor round-trips a nested serde_json::Value, ON CONFLICT preserves sibling columns, library_scope separates rows per server. End-to-end "kill mid-500k-sync → resume same cursor" stays out of scope per the kickoff answer — it belongs to PR-3 / C2 where the InitialSyncRunner lives. * test(library): cover AC A3 — 500-row upsert_batch under perf budget * feat(library): Subsonic REST client for the sync engine (Phase B, PR-2) (#793) Phase B (B1-B9 per spec §10) — pure-Rust Subsonic client that the library-sync engine (PR-3) will drive. No Tauri commands, no events; the surface is added internally to psysonic-integration as a sibling of the existing navidrome native-REST module. - B1 — SubsonicClient + ping over /rest/{method}.view. Auth via the legacy salted-md5 token (spec v1.13+, advertised as 1.16.1). New SubsonicCredentials helper computes token = md5(password || salt) and ships a per-process unique salt nonce so back-to-back calls don't repeat. - B2 — get_scan_status → ScanStatus { scanning, count, folder_count, last_scan }. Lightweight poll for the Huge-tier path (§6.2.2). - B3 — get_album_list2(type, size, offset, musicFolderId?) + get_album(id). The two-call pattern the sync engine walks during initial ingest (§6.3). - B4 — search3(query, songCount, songOffset, musicFolderId?). Empty query → all songs paged (Navidrome quirk, spec §2.4). - B5 — get_indexes(musicFolderId?, ifModifiedSince?). Conditional fetch for file-tree fallback (S3 / §3.1). - B6 — get_song(id). Error code 70 maps to the dedicated SubsonicError::NotFound variant so the tombstone reconciler can match on the variant instead of parsing strings. - B8 — get_artists(musicFolderId?). ID3-path artist index; clients compare ArtistIndex.last_modified_ms against the local watermark to decide if a delta pass is needed (§2.2.1). - B9 — fingerprint_sample helper picks every-Nth track id for the server-fingerprint verify pass. Sampling is deterministic so reruns probe the same tracks. The verify-and-compare glue itself is library-side (PR-3 territory, deps on the store). Tests cover envelope parsing (status=ok/failed, code 70 → NotFound, missing body key), credentials (md5 vectors, salt uniqueness across 1k rapid calls, salt differs per from_password call), each endpoint end-to-end through wiremock with query-param matchers, OpenSubsonic forward-compat (unknown fields ignored on Song), and the trailing-slash base-URL normalisation. Cargo.toml — adds query + form + multipart to psysonic-integration's reqwest feature set. PR-2's client needs `query`; the other two were already used by existing navidrome::covers / remote::lastfm code and only worked via top-crate feature unification. Aligning the crate's own deps means `cargo test -p psysonic-integration` now compiles without depending on the workspace build. Out of scope: capability detection (C1 / PR-3), Navidrome native bulk path (uses existing psysonic-integration::navidrome::queries), fixtures harness expansion (G1). * feat(library): subsonic client follow-ups from PR-2 review (PR-2b) (#794) Picks up the three non-blocking items from cucadmuh's PR-793 review (handoffs/2026-05-19-pr-793-review.md) before PR-3 starts on top. - Fresh `(token, salt)` per request. `SubsonicClient` now caches the plaintext username + password and derives a new `SubsonicCredentials` inside `send()` for every endpoint call — matches the frontend's `subsonicClient.ts` `getAuthParams()` lifecycle and follows Subsonic replay-resistance guidance. Test path keeps a `with_static_credentials` constructor so wiremock matchers stay deterministic. New `build_credentials` (`pub(crate)`) routes the two modes. - `SUBSONIC_CLIENT_ID` now carries the crate version (`psysonic/<CARGO_PKG_VERSION>`) — aligns with the frontend's `psysonic/${version}` so Navidrome log lines correlate across the WebView and Rust sync paths. - `Song.mbid_recording` gains the `musicBrainzId` serde alias (plus the schema-column spelling) so the OpenSubsonic field lands on the same hot column the §5.1 schema names. P13 strong-key matching can now key off it on ingest. - `get_song_with_raw` / `get_album_with_raw` return both the typed projection and the raw `serde_json::Value` body sub-tree. PR-3 ingest will write that raw value verbatim into `track.raw_json`, so OpenSubsonic extensions (`contributors`, `replayGain`, future fields) survive without manual field mirroring. Internal `parse_envelope_body` extracts the validation + body-key lookup once; `parse_envelope` and the new `parse_envelope_with_raw` share it. Tests cover: `from_password` produces unique salt/token across two back-to-back calls (direct + over-the-wire via wiremock `received_requests`), static mode returns the same triple, `c` query param starts with `psysonic/` and equals `SUBSONIC_CLIENT_ID`, `get_song_with_raw` preserves untyped fields (`replayGain`, `contributors`) in the raw value, `get_album_with_raw` keeps per-track extensions in `raw.song[i]`, error 70 still maps to `NotFound` on the raw variant, and `Song` deserializes `musicBrainzId` and `mbid_recording` interchangeably. B9 fingerprint-verify glue and the wider raw-ingest call sites stay with PR-3 / C2 as the review's §5 / §7 checklist directs. * feat(library): capability probe + sync_state accessors (Phase C1+C7, PR-3a) (#795) First sub-PR of Phase C (sync orchestrator). Lands the foundation that PR-3b's InitialSyncRunner consumes — pure plumbing, no runners or background tasks yet. - C1 capability probe. `psysonic_library::sync::CapabilityProbe::run` drives the §6.1 probe chain: Subsonic ping (captures `ServerInfo` envelope metadata for server-type / OpenSubsonic detection), then best-effort probes for search3 / getScanStatus / getIndexes, plus an optional Navidrome native bulk probe (caller passes `NavidromeProbeCredentials`). `CapabilityFlags(u32)` matches the §6.1.1 bitfield: NavidromeNativeBulk / SubsonicSearch3Bulk / ScanStatusAvailable / OpenSubsonic / UnstableTrackIds / FileTreeBrowse. - C7 sync_state accessors. `SyncStateRepository` gains get/set capability_flags, get/set sync_phase (idle / probing / initial_sync / ready / error), and column-scoped setters for server_last_scan_iso, indexes_last_modified_ms, artists_last_modified_ms, library_tier. Every setter uses `ON CONFLICT … DO UPDATE` scoped to its own column so concurrent watermark writes don't clobber each other. - Supporting additions in `psysonic-integration`: - `subsonic::SubsonicClient::server_info()` extracts `ServerInfo` from the ping envelope (server_type, server_version, api_version, open_subsonic). Re-uses `send()` so auth lifecycle is the same. - `navidrome::probe::native_bulk_available(url, token)` does the `GET /api/song?_start=0&_end=1` Bearer-auth probe. Returns Ok(true) on 2xx, Ok(false) on 4xx (auth ok but endpoint missing), Err on 5xx. Probe-only — full nd_list_songs port is PR-3b. - `psysonic-library/Cargo.toml` gains a `psysonic-integration` dependency (sync calls into Subsonic + Navidrome probes). DAG stays acyclic: integration does not depend on library. Per cucadmuh's PR-3 kickoff answer (handoff `2026-05-19-pr3-kickoff.md`): - Crate placement: option A — sync lives in `psysonic-library/src/sync/`, no new psysonic-sync crate. - N1 gate: probe is `/api/song?_start=0&_end=1` only; `nd_list_artists_by_role` is NOT required (Q3 answer + N1 ingest port lands in PR-3b). - UnstableTrackIds: set for Navidrome via `ServerInfo.server_type`, cleared for generic Subsonic. Tests added: 23 across library/sync, library/repos/sync_state, integration/subsonic, integration/navidrome/probe. Cover bitfield contains/insert/remove + spec bit values, probe across mixed-capability servers (full Navidrome, minimal Subsonic, broken endpoints), ping-failure short-circuit, optional Navidrome creds gating N1, sync_state column-scoped upserts (capability_flags / sync_phase / watermarks / library_tier), cross-column independence (capability writes don't reset cursor), ServerInfo extraction from ping envelope, Navidrome bulk probe across 2xx/4xx/5xx. * feat(library): InitialSyncRunner + C12 backoff + C13 id remap (Phase C2/C12/C13, PR-3b) (#796) Second sub-PR of Phase C — wires the actual ingest path on top of PR-3a's capability + sync_state foundation. Runner is pure async Rust: PR-3d will spawn it inside a tokio task and emit Tauri progress events on top. - C2 InitialSyncRunner. Drives spec §6.3 IS-1 → IS-6: probe-derived IngestStrategy (enum N1/S1/S2/S3, selector picks N1 → S1 → S2 chain per kickoff Q3), per-page upsert loop, cursor flush after every successful batch, IS-4 best-effort getArtists watermark, IS-5 getScanStatus.lastScan capture, IS-6 phase=ready + cursor cleared. Resume is automatic: a non-empty initial_sync_cursor_json restarts at the persisted offset; a strategy mismatch between cursor and capability flags surfaces as SyncError::CursorIncompatible. - C12 backoff. sync::backoff::Backoff implements the §6.8 schedule (2s → 4s → … cap 120s) with ±25% jitter via deterministic salt. retry_with_backoff wraps every endpoint call: transport / Navidrome failures retry up to MAX_ATTEMPTS_PER_BATCH (5), the cursor never advances on failure, success resets the counter. Cancellation AtomicBool is checked between attempts. - C13 id remap. TrackRepository::upsert_batch_with_remap performs the §6.9 detect-and-rebind pass inside the same SQLite transaction as the upsert: a content_hash or server_path collision on a different existing id triggers UPDATE of child tables (track_offline, track_extension, track_fact, track_artifact, track_canonical_link), INSERT INTO track_id_history, DELETE old track row. Off when UnstableTrackIds is clear (generic Subsonic). New TrackIdHistoryRepository read-side helper for forward lookups (analysis cache reuse, Phase E). - IngestStrategy enum + selector (sync::strategy) — N1 → S1 → S2; N1 requires Navidrome bearer credentials at runtime (skipped when None). S3 is enumerated for future file-tree fallback but returns StrategyUnsupported in v1 per kickoff Q3. - InitialSyncCursor (sync::cursor) — JSON-serialisable { strategy, phase, library_scope, ingested_count, strategy_state }. StrategyState tagged enum: LinearOffset { offset } for N1/S1, AlbumCrawl { album_offset, current_album_id } for S2. - mapping::subsonic_song_to_track_row + navidrome_song_to_track_row centralise the JSON → TrackRow projection. Subsonic path also reads replayGain.{trackGain,albumGain} from the raw value so PR-3b doesn't drop the columns that PR-2b reserves on TrackRow. - Supporting bits in psysonic-integration: - subsonic types now derive Serialize so the runner can round-trip a typed Song back into raw JSON when feeding upsert. - navidrome::queries gains nd_list_songs_internal — pure async function (no #[tauri::command] decorator) that the N1 ingest loop calls directly. The existing Tauri command wraps it. Tests added across sync::* and repos::track_id_history. Wiremock covers S1 happy-path, mid-cursor resume from a persisted offset, strategy mismatch → CursorIncompatible, 503 transient → retry-then- succeed, AtomicBool cancellation → Cancelled, N1 paginated /api/song ingest, S2 album crawl, and §6.9 remap firing under UnstableTrackIds during an actual sync. Backoff schedule + jitter formula pinned. TrackRepository remap path covered by content_hash collision, server_path collision, hash+path-missing skip, identity-noop, and remap-off compatibility with the existing upsert_batch contract. Also fixes cucadmuh's PR-3a review minor 1: drops the dead `mount_ok` scaffolding from sync::capability tests. Out of scope per kickoff Q2: - DeltaSyncRunner + tombstones → PR-3c - Background task lifecycle, cancellation wiring, progress emit throttle, adaptive scheduler, request budget, bandwidth lane → PR-3d - Tauri command surface for "sync now" / progress events → PR-5 * feat(library): search3 raw envelope fidelity for S1 ingest (PR-3b follow-up) (#797) Picks up cucadmuh's PR-3b review minor 1: the S1 path in InitialSyncRunner was reserialising the typed `Song` for `track.raw_json`, dropping unknown OpenSubsonic extensions (`replayGain`, `contributors`, …). N1 and S2 already carry the raw sub-tree verbatim through `nd_list_songs_internal` and `get_album_with_raw`; S1 now matches via the new `SubsonicClient::search3_with_raw` mirror of the PR-2b pattern. - subsonic::SubsonicClient::search3_with_raw — returns `(SearchResult, serde_json::Value)`; uses the existing `parse_envelope_with_raw` so error 70 / `Api { code, .. }` mapping stays consistent. - sync::initial::run_s1 now calls `search3_with_raw` and feeds the per-song raw sub-tree (`raw_body.song[i]`) into `subsonic_song_to_track_row` instead of a typed reserialise. Tests cover `search3_with_raw` round-trip on a payload with `replayGain` + `contributors` (verifies the raw value preserves both) and the empty-result case where the body is `searchResult3: {}`. Plus an end-to-end S1 ingest test that asserts the persisted `track.raw_json` column contains the OpenSubsonic extensions after a full runner pass, and that `replay_gain_track_db` / `_album_db` still land on the typed columns via the mapping helper. Full review: psysonic-workdocs/internal/collaboration/handoffs/2026-05-19-pr-796-review.md * feat(library): DeltaSyncRunner + TombstoneReconciler (Phase C3/C4, PR-3c) (#798) Third sub-PR of Phase C — drives targeted delta passes on top of PR-3a/b's foundation. Pure async; PR-3d will spawn it inside the background scheduler. - C3 DeltaSyncRunner. Walks spec §6.4 DS-0 … DS-9: - DS-0/1/2/3 cheap probe via `getArtists` (small/medium tier) or `getScanStatus` (huge tier when `ScanStatusAvailable`). Server watermark match → up_to_date short-circuit, scan-in-progress → deferred_scanning report; zero further requests in either case. - DS-4 targeted ingest. Strategy from capability_flags: N1-delta when NavidromeNativeBulk is set, otherwise S2-delta. S1 has no delta semantic so it's not used here. - N1-delta: GET /api/song _sort=updated_at _order=DESC, pages until rows fall under the local `MAX(server_updated_at)` watermark; out-of-band rows in the same page are dropped. - S2-delta: getAlbumList2 type=newest then type=recent, up to a small page cap; getAlbum is fetched only for album_ids the local store doesn't already have. Known albums are skipped so a play-bump under "recent" doesn't re-ingest the whole tracklist. - DS-6 id remap reuses TrackRepository::upsert_batch_with_remap. - DS-9 stamps next watermark (artists_last_modified_ms or server_last_scan_iso) + last_delta_sync_at. - DS-5 canonical matcher (Phase H) and DS-7 starred delta are out of scope for PR-3c. - C4 TombstoneReconciler. Caller-driven streaming: each `reconcile_chunk(budget)` picks the next `budget` ids ordered by synced_at ASC, calls getSong, marks deleted=1 on code 70, and refreshes synced_at on every checked id so the queue rotates. Mode A (manual integrity) loops until checked == 0; Mode B (auto-threshold) tests `should_auto_reconcile(local, server, pct)` per delta tick and runs a small budgeted chunk. Memory bounded — no full local-id list ever held in RAM. - SyncStateRepository: new getters for artists_last_modified_ms, server_last_scan_iso, library_tier; new set_last_delta_sync_at stamp helper. All column-scoped upserts preserve neighbouring fields. Tests cover DS-2 short-circuit (watermark match), DS-3 defer (scanning=true), N1-delta watermark cutoff (3 fresh + 2 stale rows → only 3 upserted), S2-delta known-album skip (mock 404 on al_known guards the assertion), DS-9 watermark + last_delta stamping, should_auto_reconcile threshold cases (gap, tolerance, server=0, local<=server), reconcile_chunk code-70 → deleted=1, budget + ordering (oldest first, newest untouched), empty-store noop, and cancellation. PR-3d (background task, probe→flags wiring, progress emit, adaptive scheduler, request budget, bandwidth throttle) lands next on the same integration branch. * feat(library): sync supervisor + progress channel + DS-8 wiring (Phase C5/C6, PR-3d1) (#799) First half of PR-3d (cucadmuh-approved split per kickoff Q2). Pure-Rust lifecycle + progress infrastructure on top of the runners from PR-3a/b/c. Tauri events stay in the top crate (PR-5); this PR only ships the channel the top crate will subscribe to. - C5 SyncSupervisor. Spawns a sync workload inside a tokio task, owns the cancellation AtomicBool, and exposes a single-consumer mpsc receiver for ProgressEvent. join() returns the inner Result<(), SyncError>; panics surface as Storage so callers never need to know about tokio internals. - C6 progress channel. New sync::progress module: - ProgressEvent enum — lean variants (PhaseChanged / IngestPage / Remapped / Tombstoned / Completed / Error). Server / scope context lives on the channel side (one supervisor = one scope). - Progress trait + NoopProgress default + ChannelProgress forwarding through tokio mpsc. Throttle is the simple last-emit-timestamp gate; terminal events (Completed / Error) bypass it. - InitialSyncRunner + DeltaSyncRunner gain with_progress(...) builders. IS-1 / IS-6 emit PhaseChanged + Completed; delta emits PhaseChanged at strategy pick, Tombstoned at DS-8, and Completed at DS-9. Defaults to NoopProgress so existing call sites keep working. - DS-8 wired. DeltaSyncRunner::with_tombstone_budget(n) drives TombstoneReconciler::reconcile_chunk(n) after DS-4 ingest; shares the runner's cancellation flag + sleep override. The DeltaSyncReport gains tombstones_checked / tombstones_deleted so callers can act on the counts. - capability::probe_and_persist helper. Chains CapabilityProbe::run with sync_state writes: sets phase to "probing" before the probe, persists capability_flags, then drops back to "idle". PR-3d2 (the scheduler) will call this in front of every initial / delta run so the stored flags reflect the live server. Tests cover: ChannelProgress throttle (zero-interval pass-through, terminal bypass, non-terminal collapse, sender alive after receiver drop), SyncSupervisor task completion + cancel + panic-as-Storage + receiver-take-once, probe_and_persist round-trip through SyncStateRepository (flags persisted, phase ends at "idle"), DS-8 reconcile-after-ingest landing tombstones on code 70 returns. PR-3d2 follows with the adaptive scheduler (C8), request budget (C9), poll EWMA (C10), and the bandwidth / queue priority lane (C11). * feat(library): adaptive scheduler + request budget + EWMA poll + bandwidth (Phase C8/C9/C10/C11, PR-3d2) (#800) Second half of PR-3d per cucadmuh's kickoff-Q2 split. Wraps the runners + supervisor from PR-3a/b/c/d1 into a tick-driven background scheduler. Top crate (PR-5) plumbs the timer. - C8 BackgroundScheduler. Tick-based — caller drives the interval, scheduler decides whether the tick should run. is_due(now_ms) checks sync_state.next_poll_at; tick(now_ms) either skips (not due / PrefetchActive pause), or runs a DeltaSyncRunner with the right budget + tombstone trigger, then stamps the next poll_at via the adaptive formula. No tokio task ownership — tests stay deterministic, PR-5 plugs spawn behaviour to taste. - C9 RequestBudget. PassKind enum (PollTick / DeltaLight / DeltaMismatch / InitialSync) with caps per spec §6.2.5 (1 / 50 / 200 / unlimited). RequestBudget::has_room(used) gates the runner; PR-3d2 ships the data type, runner enforcement of the cap is a future tightening (DeltaSyncRunner already has its own page cap so the soft cap mostly informs Settings). - C10 PollStats EWMA. New sync::poll_stats with PollStats (artist_count, ewma_bytes, ewma_duration_ms, library_tier), observe()/set_artist_count()/reclassify() helpers, the §6.2.2 tier table (<2k / 2k-15k / >15k or ewma_bytes >2MB), and next_interval_ms following the spec formula (base * load_factor * artist_factor, load_factor clamped [1, 10]). - C11 PlaybackHint + ParallelismBudget. PlaybackHint enum (Idle / Playing / PrefetchActive) resolved to a ParallelismBudget { max_concurrent, min_request_gap_ms }. PrefetchActive pauses bulk (`max_concurrent = 0`) per §6.2.4; the scheduler honours it via tick short-circuit. - Auto-tombstone wire. Before running the DeltaSyncRunner the scheduler tests `should_auto_reconcile(local, server, pct)` against the persisted counts; on threshold trip it sets `with_tombstone_budget(200)` (the §6.2.5 DeltaMismatch cap). - SyncStateRepository gains poll_stats_json get/set, next_poll_at get/set, local_track_count get/set, and server_track_count get/set — all column-scoped upserts. Tests: ~30 new across poll_stats / budget / bandwidth / scheduler. EWMA seed + smoothing, tier-classification edges (artist + size overrides), next-interval formula bounds (idle base, slow-network load_factor clamp), RequestBudget caps per pass, ParallelismBudget resolution, scheduler is_due (no schedule / future schedule), tick short-circuit (not due, PrefetchActive pause), tick runs delta and persists next_poll_at, auto-tombstone trigger above 5 % threshold, PollStats round-trip through SQLite. Together with PR-3d1 this finishes Phase C — Tauri command surface (D1-D4) lands with PR-5. * feat(library): read-only Tauri command surface (Phase D1 part 1, PR-5a) (#801) First sub-PR of Phase D per cucadmuh's kickoff Q1 split. Lands the LibraryRuntime Tauri State plus the 8 read-only library commands from spec §7.1. No SyncSupervisor spawn, no sync lifecycle commands, no credentials store — those land in PR-5b. - New psysonic_library::runtime::LibraryRuntime — Tauri State wrapping Arc<LibraryStore>. Top crate's lib.rs setup() now calls LibraryStore::init(app), wraps the result in the runtime, and app.manage's it. Mirrors the AnalysisCache wiring above it. - New psysonic_library::dto module — camelCase wire DTOs per src-tauri/CLAUDE.md: SyncStateDto, LibraryTrackDto (flat projection over the track hot columns + raw_json sub-tree), LibraryTracksEnvelope, TrackArtifactDto, TrackFactDto, OfflinePathDto, TrackRefDto. local_tracks_max_updated_ms helper surfaces the implicit N1-delta watermark on the SyncStateDto. - New psysonic_library::payload module — pure ProgressEvent → LibrarySyncProgressPayload mapper (the payload Tauri events carry once PR-5b plugs the supervisor's mpsc receiver into AppHandle::emit). Constants for the event names too. Unit-testable without Tauri runtime. - New psysonic_library::commands module with 8 #[tauri::command] handlers: - library_get_status — joins the sync_state row + the track-watermark MAX query into one SyncStateDto. - library_search — FTS5 via the existing search_tracks helper, paginated; hydrates hits to full LibraryTrackDto. - library_get_track — single SELECT through new TrackRepository::find_one. - library_get_tracks_batch — capped at 100 refs/call per spec, preserves caller-supplied order, drops unknowns silently. - library_get_tracks_by_album — ordered by disc/track/id via new TrackRepository::find_by_album. - library_get_artifact — flexible WHERE over track_artifact (artifact_kind required, source/format optional), latest fetched_at wins. - library_get_facts — fact_kinds filter optional; returns all rows for the (server_id, track_id) pair when none specified, sorted by fact_kind + fetched_at DESC. - library_get_offline_path — returns local_path with a `missing: true` flag when the row is absent. - TrackRepository gains find_one / find_batch / find_by_album with a shared row-to-TrackRow mapper. SQL constants pinned next to the existing UPSERT_SQL so a schema change touches one file. - src-tauri/src/lib.rs: LibraryStore::init in setup(), the eight command handlers added to invoke_handler!. Tests cover: DTO field-name camelCase (IPC contract guard), LibraryTrackDto round-trip through TrackRow, raw_json fallback to Value::Null on bad input, local_tracks_max_updated_ms ignores deleted rows, TrackRepository::find_one / find_batch / find_by_album ordering + unknown-ref drop, ProgressEvent mapper across all six variants + serialization keys camelCase. Library tests at 166; workspace stays green. Out of scope per kickoff Q1: - Mutating commands (library_sync_*, library_patch_*, library_put_*, library_purge_*, library_delete_*) → PR-5b - SyncSupervisor spawn + background scheduler tick loop + progress emit → PR-5b - library_sync_bind_session / clear_session credentials → PR-5b - TS wrappers + Settings UI + server-remove modal → PR-5c - library_advanced_search / library_search_cross_server SQL builders → PR-5d * feat(library): sync lifecycle + mutate + purge Tauri surface (Phase D1 part 2, PR-5b) (#802) Second sub-PR of Phase D per cucadmuh's kickoff Q1 split. Adds the mutating side of §7.1 plus the SyncSession credentials store, the PlaybackHint setter, the orchestrator that runs InitialSyncRunner / DeltaSyncRunner under a Tauri AppHandle and emits library:sync-progress and library:sync-idle events, and the top-crate scheduler tick task that sweeps every bound session through BackgroundScheduler::tick. - LibraryRuntime extended per kickoff Q2: sync_sessions HashMap, playback_hint cell, current_job (cancel handle + identity), and scheduler_cancel flag the tick task watches. Kickoff sketch said Mutex<Option<SyncSupervisor>> — supervisor's join() consumes self, so holding it in the mutex would block library_sync_cancel behind the orchestrator's join; CurrentJob carries the Arc<AtomicBool> cancel + metadata instead, orchestrator task owns supervisor / receiver / join. - New commands (spec §7.1): - library_sync_bind_session — caches Subsonic creds in memory, tries navidrome_token once for bearer cache, runs probe_and_persist so capability_flags reflect the live server. - library_sync_clear_session — drops cached credentials. - library_set_playback_hint — JS pushes idle / playing / prefetch_active from existing audio listeners. - library_sync_start — dispatches InitialSyncRunner (mode='full') or DeltaSyncRunner (mode='delta', with auto-tombstone budget when local/server count gap exceeds threshold). Spawns runner + orchestrator task that drains the progress mpsc into library:sync-progress emits and emits library:sync-idle when the runner exits. - library_sync_cancel — trips the current job's cancel flag. - library_patch_track — sparse JSON patch (starredAt, userRating, playCount, playedAt) per §6.5. - library_put_artifact / library_put_fact — upserts with ON CONFLICT scoped to the PK so lyrics / BPM writes survive re-fetches. - library_purge_server — transactional DELETE across the v1 schema tables for this server_id. include_offline (default false) controls track_offline + bytes_freed. - library_delete_server_data — alias that always purges offline too (logout flow). - src-tauri/src/lib.rs setup() spawns a 30 s MissedTickBehavior::Skip task that snapshots bound sessions and drives BackgroundScheduler::tick(now_ms) for each. Honours runtime.scheduler_cancel + the current PlaybackHint. Background ticks stay silent (NoopProgress) — Tauri emit for the scheduler path lands when Settings (PR-5c) surfaces it. - psysonic-integration::navidrome re-exports navidrome_token so the bind_session command can drive the bearer cache without making the client module pub. Tests cover: LibraryRuntime session round-trip (set/get/clear scopes per server), playback_hint default + setter, snapshot returns clones so callers can mutate freely. Existing library tests stay green (171 → 171; new code paths under the Tauri command surface — devtools integration smoke is PR-5c's job). Out of scope per kickoff Q1: - src/library/ TS wrappers + Settings UI subsection + server-remove modal → PR-5c - library_advanced_search / library_search_cross_server SQL builders → PR-5d - Background-tick Tauri emit (NoopProgress today) → PR-5c - analysis_cache cross-purge in library_purge_server → PR-6 * feat(library): typed invoke wrappers + verify_integrity command (Phase D2 + part of D1, PR-5c) (#803) Frontend-facing slice of Phase D. Ships the typed src/api/library.ts wrapper layer that any Settings / browse code will import from, plus the manual-integrity backend command PR-5b's review §5 note 2 called out as missing. Scope cut from cucadmuh's PR-5 kickoff Q1 split: that proposal had PR-5c = D2 + D3 + D4 (wrappers + Settings subsection + server-remove modal). The Settings UI + server-remove + audio playback-hint wiring + authStore extensions + i18n strings turn into a thick frontend patch in their own right; landing them in one PR with the wrappers would mix Tauri-surface review with Settings UX review. The split: - PR-5c (this PR) — D2 wrappers + library_sync_verify_integrity. - PR-5c-ui (follow-up) — D3 Library Settings subsection, D4 server-remove modal contract, playback hint feed, authStore / i18n. Per kickoff exit clause ("Do not split 5c unless review size forces it"). Reviewable as a clean Tauri-surface vs UX boundary. - Backend: `library_sync_verify_integrity { serverId, libraryScope? }` command — same dispatch shape as `library_sync_start { mode:'delta' }` but always forces the full `DELTA_MISMATCH_CAP` tombstone budget regardless of the local/server count gap. Spec §6.7 Mode A user- initiated full reconcile bypasses the threshold check that governs background ticks. `library_sync_start` itself is refactored to delegate to a private `library_sync_start_inner(force_full_tombstone)` so both entry points share the runner-spawn + orchestrator + emit code. - Frontend `src/api/library.ts`: full typed wrapper layer over the 19 `library_*` Tauri commands. DTO mirrors carry the camelCase wire shape (`SyncStateDto`, `LibraryTrackDto`, `TrackArtifactDto`, `TrackFactDto`, `OfflinePathDto`, `PurgeReportDto`, `SyncJobDto`, `TrackRefDto`, `ArtifactInputDto`, `FactInputDto`). Plus the `LibrarySyncProgressPayload` / `LibrarySyncIdlePayload` interfaces and `subscribeLibrarySyncProgress` / `subscribeLibrarySyncIdle` helpers that wrap `@tauri-apps/api/event` listen. PlaybackHint literal type lives here too (`'idle' | 'playing' | 'prefetch_active'`) so the audio listeners in PR-5c-ui can import a single source of truth. - `src-tauri/src/lib.rs` adds the new verify_integrity handler to the `invoke_handler!` aggregate. Tests: library tests stay at 171 — verify_integrity is exercised through the existing `sync_start_inner` paths; the wrapper layer is trivial passthrough that TypeScript types already check. Vitest coverage for the typed wrappers belongs with PR-5c-ui where there are real consumers (LibraryTab) to drive integration tests. PR-5c-ui (next) lands: - Library Settings subsection (§7.3 minus advanced toggles) - ServerRemoveModal extension (keep vs delete local index per §5.6) - authStore: libraryIndexEnabledByServer + auto-reconcile toggle - src/store/audioListenerSetup audio:playing / ended / setDeferHotCachePrefetch → library_set_playback_hint - i18n keys for the new strings * feat(library): Settings library index UI + playback hint + purge-on-remove (Phase D3/D4, PR-5c-ui) (#804) * feat(library): Settings library index UI + playback hint + purge-on-remove (Phase D3/D4, PR-5c-ui) Frontend half of Phase D, on top of PR-5c's typed wrappers. Wires the Settings → Library subsection (§7.3), the audio playback-hint feed (§6.2.4), and the server-remove keep-vs-delete choice (§5.6). - New libraryIndexStore (Zustand, persisted) — per-server enable flag + auto-reconcile toggle. Kept out of authStore so the index feature evolves independently and the persisted blob stays small. - New LibraryIndexSection in Settings → Library: - Per-server "Enable local library index" toggle → binds / clears the Rust sync session with the active server's credentials. Off by default (P6). - Read-only status (Idle / Checking / Initial sync / Ready (n) / Error) polled from library_get_status every 3 s, overlaid with live library:sync-progress events. - Sync now / Verify integrity / Cancel buttons. Verify runs one §6.7 pass (budget 200) per click; the status line shows the checked/removed counts so large libraries can be continued with another click (auto-resume loop is a follow-up). - Auto-reconcile toggle. - Subscribes to library:sync-progress + library:sync-idle for the active server; errors surface as a toast. - Audio playback hint: handleAudioPlaying → 'playing', handleAudioEnded → 'idle' via notifyLibraryPlaybackHint, which gates on the per-server index toggle + dedupes repeated hints so the IPC boundary isn't spammed on every progress tick. - ServersTab delete flow: when a server with an enabled index is removed, a second confirm offers keep-vs-delete of the local library cache (OK = library_delete_server_data, Cancel = retain for offline). Always clears the sync session. - i18n: en + de keys for the new strings; other locales fall back to en via i18next (later sweep). Per PR-803 review §5: verify-integrity resume UX is one-pass-per-click with a visible counter; sync_start idempotency (replaces in-flight) is surfaced via the Cancel button appearing while busy. Out of scope: - VirtualSongList / playerStore local-mode consumers → PR-7 (F1/F3/F5) - library_advanced_search / cross-server UI → PR-5d + PR-7 F2 - Auto-resume loop for very large verify-integrity runs → follow-up - Search-all-servers + threshold input (advanced §7.3) → later * fix(library): normalize server base URL before bind probe The bind toggle threw "subsonic transport: builder error | relative URL without a base" — `server.url` is stored bare (e.g. `nas.example.com`) and reqwest needs a scheme. Two-sided fix: - Frontend: LibraryIndexSection passes `authStore.getBaseUrl()` (adds http:// + strips trailing slash) instead of the raw `server.url`, matching the existing `subsonic.ts` convention. - Backend: `library_sync_bind_session` normalizes the incoming `base_url` defensively so the stored session + every downstream caller (sync_start, scheduler tick, navidrome_token) gets a scheme-qualified URL regardless of what the WebView sends. Tests: normalize_base_url covers bare host, trailing slash, existing http/https scheme, and whitespace. * fix(library): re-bind sync session on startup + server switch "Library sync failed: no bound session" — the per-server index toggle persists in localStorage but the Rust sync session (credentials + bearer) lives in process memory and is gone after an app restart, so the toggle showed "on" while no session existed. Per PR-5 kickoff Q5 ("on server connect if index already on"). - New `ensureActiveServerSessionBound()` helper: re-binds the active server's session when its index toggle is enabled. Best-effort — silent on failure (Settings surfaces the real error on explicit toggle). - MainApp re-binds on every `activeServerId` change (covers app startup + server switch — `setActiveServer` drives the effect). - LibraryIndexSection re-binds on mount before the first status poll, so Sync now / Verify integrity work immediately even when the toggle was already on from a previous run. * fix(library): trigger initial full sync on first enable (PR-804 review §5.1) cucadmuh's PR-804 review flagged this as release-blocking: the toggle only bound the session and «Sync now» / the background tick ran delta-only, so a fresh enable left the index empty — delta can't populate a never-synced library. - On first enable, after bind, fetch status and dispatch `library_sync_start { mode: 'full' }` when `lastFullSyncAt` is null (matches spec §6.2 "initial sync always background"). - «Sync now» now picks mode adaptively: `full` until a full sync has completed, `delta` afterward — so the button works both for the initial population and incremental updates. Other PR-804 review notes (auto-reconcile toggle → backend wiring, prefetch_active hint, clear-old-session-on-switch) stay as documented non-blocking follow-ups. * feat(library): advanced search + cross-server SQL builders (Phase D-search, PR-5d) (#806) * feat(library): advanced search + cross-server SQL builders + commands (Phase D-search, PR-5d) - FilterFieldRegistry SQL resolution: SqlFragment, compare_fragment, validate_for_entity (§5.13.5) - Advanced Search builder: per-entity track/album/artist queries; genre (case-insensitive), year, starred, bpm filters; bpm dual-storage resolution (§5.13.4); libraryScope; sort allowlist; full-match totals - Cross-server FTS union (§5.5B / §5.9 A') with canonical-id dedup - library_advanced_search + library_search_cross_server commands, registered in the shell * feat(library): typed advanced search / cross-server invoke wrappers (PR-5d) Mirror request/response DTOs and add libraryAdvancedSearch / librarySearchCrossServer in src/api/library.ts. UI parity (AdvancedSearch.tsx) stays PR-7. * fix(library): self-heal stale/unreadable initial-sync cursor instead of bricking (#807) The initial-sync cursor records the ingest strategy it was created under. When a re-probe later selects a different strategy (e.g. the Navidrome native bearer is briefly unavailable, downgrading N1->S2), the cursor guard returned a hard error — and since nothing clears the cursor, every later full sync failed with no recovery path. Reset the stale (or unreadable) cursor and start fresh under the selected strategy instead of erroring. Re-ingest is idempotent (upsert); the tombstone pass reconciles leftovers. * fix(library): emit per-batch progress during initial sync (#808) The initial-sync runner only emitted PhaseChanged (start) and Completed (end), so the Settings status sat at "initial_sync" with no count for the entire ingest — looking stuck on large libraries even while rows landed. Emit IngestPage per batch from the N1/S1/S2 loops with the running ingested total; the existing <=2 Hz throttle paces it. The frontend already renders the count from these events. * feat(library): Advanced Search reads the local index when ready (Phase F2, PR-7a) (#811) When the active server's index is fully synced, Advanced Search serves query / genre / year / result-type from library_advanced_search (instant + offline) and pages songs locally. On not-ready or any failure it falls back to the existing network path unchanged (spec 5.13.6). Results map from each entity's stored Subsonic rawJson, with the flat hot columns as a fallback. * feat(library): canonical matcher — link tracks by ISRC/MBID on ingest (Phase H1/H2, PR-4a) (#812) Adds the strong-key cross-server matcher (spec §5.5A): on every track upsert, link (server_id, track_id) to a canonical id derived from its ISRC (preferred) or MBID recording. Deterministic id (`{kind}:{value}`) keeps it O(1) and idempotent — no lookup-then-create race, no fuzzy loop on the bulk path. Tracks without a strong key stay standalone (fuzzy/search-time matching is H3). * feat(library): cross-server fuzzy fallback in search (Phase H3, PR-4b) (#813) library_search_cross_server now returns a `fuzzy` list alongside the exact FTS `hits` (spec §5.9): per-server `title LIKE %query%` for matches the exact pass missed (diacritics, partial words), capped per server, excluding exact hits and deduped by canonical id against them. Shared `like_contains` moved to the `search` module. * feat(library): FactRepository with TTL + provenance rules (Phase E4, PR-6a) (#814) Typed CRUD over track_fact behind library_get_facts / library_put_fact (spec §5.12): get lazily deletes the track's expired facts then returns the survivors (no background GC, P34); a `user` bpm fact also writes the hot track.bpm column so the override wins and survives a resync (R6-3.4). The commands now delegate here instead of inlining the SQL. * feat(library): ArtifactRepository with TTL + 512KB cap (Phase E4, PR-6b) (#815) * fix(integration): decode OpenSubsonic isrc string-array on Song (#818) OpenSubsonic types `isrc` as `string[]`; Navidrome 0.61.2 ships it as `isrc: []` or `["USRC…"]`. The typed `Song.isrc: Option<String>` could not decode either form, which broke the S1 (`search3`) and S2 (`getAlbum`) ingest paths on real Navidrome libraries — initial sync could not complete past the first array-valued track. Add a tolerant `de_string_or_seq` deserializer: plain string → `Some`, non-empty array → first usable value (string element, or an object element's `name` for the `[{ "name": … }]` shape), `[]`/null → `None`. The full multi-value set still survives verbatim in `track.raw_json` (ADR-7). Applied to `Song.isrc`. Per maintainer policy R7-15 (workdocs question 2026-05-20-large-library-ingest-client-only, checklist item 1): treat Navidrome as a black box, harden the client decode. Tests cover `isrc: []` → None, populated array, and the legacy single-string form. * feat(library): large-library ingest strategy — S1 over N1 (R7-15) (#819) Per maintainer policy R7-15 (large-library ingest, client-only): very large Navidrome catalogs must not start initial sync on N1 — its native `/api/song` returns HTTP 500 beyond a deep offset and can never finish. S1 (`search3`) does not hit that wall. - Add `IngestStrategy::select_initial_strategy(flags, server_track_count, n1_bulk_unreliable)`. Large libraries (count > LARGE_LIBRARY_THRESHOLD, default 40_000) or servers flagged `n1_bulk_unreliable` route to S1 — or S2 when search3 bulk is absent. Normal-size libraries keep the cheapest N1 → S1 → S2 chain unchanged. - Persist the learned per-server `n1_bulk_unreliable` flag on `sync_state` (additive migration 002, DEFAULT 0). The mid-run N1→S1 fallback that sets it lands in a follow-up. - Capture `getScanStatus.count` in the capability probe and persist it as the `server_track_count` watermark, so the threshold applies from the first sync rather than only after N1 hits the wall once. A count-less probe never clobbers a watermark from a prior run. - The initial-sync runner now selects via the new policy. Tests: selector table (all branches incl. threshold boundary and the search3-absent fallback), repo flag roundtrip, probe count capture + watermark-preservation, migration head-version bookkeeping. * feat(library): freeze ingest strategy on resume (R7-15 Q3) (#820) A persisted initial-sync cursor that has already made progress must resume under its own strategy and ignore what a fresh capability probe would now pick. Previously any strategy mismatch reset the cursor to a fresh one — so a flapping Navidrome bearer (N1 flag toggling between probes) restarted ingest from offset 0 on every launch, which is why large initial syncs never completed across restarts. `load_or_init_cursor` now: - resumes the cursor's strategy when it has progress (`ingested_count > 0` or `phase != Ingest`), regardless of the re-selected strategy; - adopts the freshly-selected strategy only when there is no resumable progress (offset 0), where re-selecting costs nothing; - still resets a corrupt/unreadable cursor rather than hard-erroring. One guarded exception: a cursor still on N1 after the server was learned `n1_bulk_unreliable` is known-broken and re-selects onto the non-N1 path instead of resuming a wall-bound N1 loop (the mid-run N1→S1 fallback that preserves progress lands next). Tests: resume-with-progress freezes strategy and keeps the count; no-progress cursor adopts the re-selected strategy; known-broken N1 cursor re-selects; unreadable cursor still resets. * feat(library): one-way N1→S1 fallback on deep-offset 500 (R7-15 Q5) (#821) When the N1 ingest loop hits a persistent HTTP 500 at or beyond the deep-offset safety line (`N1_DEEP_OFFSET_SAFE`, 50_000) it now treats it as Navidrome's server-side deep-offset wall rather than a transient error: it learns `n1_bulk_unreliable` for the server and finishes the sync on S1. - `run_n1` catches the wall after retry exhaustion (`n1_hit_deep_offset_wall`: HTTP 500 AND offset >= the safety line) and hands off to `fall_back_n1_to_s1`. A 500 below the line stays a propagated error — no silent downgrade. - The fallback flags the server, then restarts S1 from offset 0. N1 (`id ASC`) and S1 (`search3` default order) don't share an offset space, so resuming from the N1 offset would skip songs; re-ingest is idempotent (PK upsert), duplicate work over the rows N1 already wrote is acceptable for v1. The cursor is rewritten in place, never zeroed. - One-way only: S1 never flips back to N1 mid-run. Combined with the persisted flag and the resume freeze, a future sync selects S1 directly. - `N1_DEEP_OFFSET_SAFE` is overridable on the runner so the fallback is testable without 50k rows of fixture data. Tests: deep-offset 500 falls back to S1, ingests the full set without duplicating N1's rows, and persists the flag; a shallow 500 propagates and does not flag the server. * feat(library): cache + retry Navidrome bearer, keep N1 flag on transient loss (R7-15 Q3) (#822) A flaky `/auth/login` previously stripped N1 for a whole bind: the bearer was fetched once, best-effort, and a single miss dropped to Subsonic-only. Per R7-15 Q3 a transient `navidrome_token` failure must not drop the `NavidromeNativeBulk` capability. - `bind_session` fetches the bearer with `navidrome_token_with_retry` (3 attempts, short backoff); if it still fails, it keeps the bearer cached from a prior bind instead of overwriting it with `None`. The token / credentials are never logged. - `probe_and_persist` preserves a previously-learned `NavidromeNativeBulk` flag when it probes without a token — the server still supports `/api/song`; only the bearer is missing this bind. The capability is a stable server property, so a token-less probe must not clear it. - `library_sync_start_inner` masks `NavidromeNativeBulk` from *this run's* strategy selection when the session has no token, so the run proceeds Subsonic-only (S1/S2) instead of selecting N1 with no creds. The persisted capability stays intact for a later bind that recovers the token. The in-flight cursor is already protected by the resume freeze. Tests: token retry yields the token on success and `None` after exhausting attempts; the probe keeps a learned N1 flag across a token-less re-probe. * feat(library): mid-run S1→S2 fallback on persistent S1 failure (R7-15 Q8) (#823) The N1→S1 fallback (#821) had no analogue when S1 itself fails on a server. Per R7-15 Q8, a persistent S1 failure (C12 retries already exhausted) must fall back to the universal S2 album crawl — no new artist-walk strategy. - `run_s1` catches a persistent fetch failure from the `search3` retry loop (`is_fetch_failure`: transport / HTTP / decode / Subsonic API / not-found) and hands off to `fall_back_s1_to_s2`. Cancellation and storage errors propagate untouched. - The fallback restarts S2 from scratch. S1 (`search3` order) and S2 (album-list order) don't share an offset space, so resuming from the S1 offset would skip songs; re-ingest is idempotent (PK upsert). The cursor is rewritten in place, never zeroed — the resume freeze then keeps the run on S2 across restarts. This completes the ingest fallback chain N1→S1→S2 from the §6.3 strategy order; the start-time "no search3 → S2" selection was already covered (#819). Tests: a persistent S1 500 falls back to S2 and the album crawl ingests the track. * fix(library): resume interrupted initial sync on startup (#824) * fix(library): resume interrupted initial sync on startup An initial sync killed mid-run (app restart) sat at `idle` until the user clicked «Sync now» — the background scheduler is delta-only and the auto-full-sync only fired on the index toggle, not on the startup re-bind. `resumeInitialSyncIfIncomplete` runs after the active server's session is re-bound (startup + server switch): if no full sync has completed yet (`!lastFullSyncAt`) it dispatches `library_sync_start { mode: 'full' }`, which resumes from the persisted cursor instead of restarting from zero. Once a full sync has landed it is a no-op, so delta stays the scheduler's job. Best-effort — errors stay silent (Settings surfaces them on explicit action). Tests: starts a full sync when none has completed, no-ops once a full sync has landed, stays silent when the status lookup fails. * fix(library): silence cancelled-sync toast, de-dupe startup resume Two rough edges from the startup resume: - A cancelled sync surfaced as «Library sync failed: sync cancelled». The orchestrator emitted the runner's `Cancelled` result as an error on the sync-idle event. Cancellation is expected — the user cancelled, or a newer `library_sync_start` superseded the job (server switch / startup resume) — and is documented as silent. `sync_outcome_to_result` now maps `SyncError::Cancelled` to a clean idle, only real errors toast. - `resumeInitialSyncIfIncomplete` is now de-duped per server. React StrictMode fires the startup effect twice, so a second `library_sync_start` cancelled the first (`set_current_job` is cancel-and-replace) — harmless with the fix above, but the dedupe avoids the wasted job + probe entirely. Tests: `sync_outcome_to_result` keeps `Cancelled` silent and forwards real errors; concurrent resume calls start a single full sync. * fix(library): run DB read commands off the main thread (async) (#825) The 10 library read commands were synchronous (`pub fn`). Per the Tauri v2 docs, commands without `async` run on the main thread — so a read that blocks freezes the UI. During an initial sync the runner holds the single `Mutex<Connection>` for a whole batch write (500 rows × per-row remap on Navidrome + upsert + FTS, one transaction), and the Settings library section polls `library_get_status` on an interval. Each batch write blocked that polled read on the main thread → the window greyed out until the batch finished, with the freeze growing as the DB grew. Make the DB-touching read commands `async` so they run off the main thread: `library_get_status`, `library_search`, `library_get_track`, `library_get_tracks_batch`, `library_get_tracks_by_album`, `library_get_artifact`, `library_get_facts`, `library_get_offline_path`, `library_advanced_search`, `library_search_cross_server`. Reads still serialize behind the writer (the single connection is intentional — the schema mirrors `analysis_cache`, spec §5.1), but the wait no longer blocks the UI. State-only commands stay sync. Invoke names / payloads are unchanged, so the frontend is unaffected. Spec §15 R7-15 follow-up — surfaced in live QA on a 170k library. * feat(library): scope analysis cache by server_id (E1, schema only) (#826) Add a versioned migration to audio-analysis.sqlite so waveform/loudness rows are keyed per server. This is the schema-only step (PR-6c-1): every existing row migrates to server_id='' and behaviour is unchanged. The server_id write/read wiring, legacy fallback and lazy re-tag follow in 6c-2. - migrations 001 (baseline = the pre-versioning schema) + 002 (rebuild the three tables with server_id; PK (server_id, track_id, md5_16kb), loudness + target_lufs) - versioned runner mirroring the library store; each migration commits its schema change and version marker in one transaction, so a failure or crash rolls the whole migration back and retries cleanly - VACUUM INTO snapshot before the table rewrite as a safety net beyond the transaction (disk-full at COMMIT, FS corruption) - TrackKey gains server_id; all callers pass "" for now * feat(library): analysis cache server_id wiring (E1, 6c-2) (#827) * feat(library): scope analysis cache writes/reads/deletes by server_id (E1 wiring) Build on the 6c-1 schema migration: thread the playback server scope (playbackServerId ?? activeServerId) through the analysis cache so a server switch can no longer surface another server's waveform/loudness for the same bare track_id. - Write: seed_from_bytes_* and the CPU-seed / HTTP-backfill queues carry a server_id; every audio write path (in-memory, ranged, legacy stream, local file, spill, preload), the syncfs offline/hot caches, and the backfill command write under the playback server (empty = legacy ''). - Read: get_latest_*_for_track and the exact-key lookup try the server scope first, then fall back to the legacy '' rows; a legacy hit is re-tagged onto the server scope (INSERT OR IGNORE, never clobbers a precise row). No bulk backfill — existing caches re-tag lazily on play, so they are not re-analysed wholesale. - The backend gain-resolution path (loudness normalization, replay-gain updates, device resume) is scoped via a pinned current_playback_server_id on the audio engine, so normalization keeps working for server-scoped rows. - Delete: delete_*_for_track_id scope to (server + legacy ''); reseed on one server no longer wipes another server's analysis. delete_all_waveforms stays global (Settings -> Storage). Tauri boundary: analysis_get_waveform(_for_track), analysis_get_loudness_for_track, analysis_delete_waveform/loudness_for_track and analysis_enqueue_seed_from_url gain an optional serverId; audio_play and audio_preload gain an optional serverId. All additive (absent = legacy ''). * feat(library): pass playback serverId to analysis IPC (E1 wiring) Send getPlaybackServerId() (queueServerId ?? activeServerId) with every analysis-cache call so reads/writes/deletes scope to the right server: - audio_play / audio_preload (playTrack, resume, queue-undo restore, gapless byte-preload) - analysis_get_waveform_for_track / analysis_get_loudness_for_track (waveform + loudness refresh) - analysis_delete_waveform/loudness_for_track + analysis_enqueue_seed_from_url (reseed + loudness backfill) Absent serverId stays backward-compatible (legacy '' scope). * feat(library): content_hash from playback (E2, 6d) (#828) * feat(library): record playback content_hash into the track store (E2) Bridge the playback-derived md5_16kb into library `track.content_hash` (R7-16 Q4) so id-remap can rebind a track when the server reassigns ids (§6.9). - New `ContentHashSink` port in psysonic-core (closure handle, mirrors PlaybackQueryHandle): keeps psysonic-analysis decoupled from psysonic-library. - `seed_from_bytes_into_cache` returns the computed md5; `seed_from_bytes_execute` fires the sink after a successful seed (Upserted or cache-hit) when a real server is known. The shell crate registers the sink to patch the library. - `patch_content_hash` + `library_patch_track`'s new optional `contentHash` field write it; both no-op when the library has no row for (server_id, id), i.e. the index is off for that server. - Sync upsert no longer clobbers it: `content_hash = COALESCE(NULLIF( excluded.content_hash,''), track.content_hash)` — a sync (which passes NULL) preserves the playback hash, a non-empty incoming hash still wins. No schema migration — the `content_hash` column already exists. Tauri boundary: `library_patch_track` gains optional `contentHash` (additive). * feat(library): expose contentHash on libraryPatchTrack wrapper (E2) Add optional `contentHash` to the `libraryPatchTrack` patch type so the TS contract matches the extended Rust command. Normally written by the Rust analysis bridge; exposed for completeness. * feat(library): enrichment summary on library_get_track (E3, 6e) (#829) * feat(library): enrichment summary on library_get_track (E3) Add an optional `enrichment { waveformReady, loudnessReady, lyricsCached }` to the single-track `library_get_track` read (R7-16 Q5). Read-only, per-server, never blocks on the network; list/batch projections leave it unset. - New `AnalysisReadinessQuery` port in psysonic-core (closure handle, mirrors ContentHashSink) keeps psysonic-library decoupled from psysonic-analysis. The shell crate registers it to probe the analysis cache by exact (server_id, track_id, content_hash) key with legacy '' fallback — read-only, no re-tag. waveform/loudness readiness is gated on a known content_hash (E2). - `lyricsCached` from a new pure-read `ArtifactRepository::lyrics_cached` (valid, non-expired, non-not_found lyrics row). - `library_purge_server`'s `includeAnalysis` documented as a deliberate v1 no-op (R7-16 Q7): analysis is never deleted on purge / server remove. Tauri boundary: `LibraryTrackDto` gains optional `enrichment` (additive). * feat(library): mirror enrichment on LibraryTrackDto wrapper (E3) Add `TrackEnrichmentDto` + optional `enrichment` to the TS `LibraryTrackDto` so the contract matches the extended `library_get_track` response. * feat(library): VirtualSongList browses the local index when ready (F1) (#830) The all-songs browse now serves pages from the local library index when it is ready for the active server, falling back to the unchanged network path otherwise. - `runLocalSongBrowse` (reuses the F2 local-read adapters): empty-query browse-all via `library_advanced_search`, whose default track order (`t.title COLLATE NOCASE ASC`) matches the network `ndListSongs('title','ASC')` path, so paging stays coherent across a local↔network boundary. - Gated per page on `libraryIsReady` + `source === 'local'`; any miss / failure returns null → VirtualSongList uses the existing browse path unchanged. - Search (non-empty query) stays on the network path for now; rich search is already covered by Advanced Search (F2). * feat(library): patch-on-use for star/rating/scrobble (PR-7 F3) (#831) * feat(library): library_patch_track clears nullable fields on explicit null (F3) Extract the patch logic into a testable `apply_track_patch`. Nullable integer fields (`starredAt` / `userRating` / `playCount` / `playedAt`) now distinguish an absent key (leave untouched) from an explicit `null` (clear the column), so `unstar` ({ starredAt: null }) actually un-stars the local row. `.map` keeps the present/absent distinction; `as_i64()` yields the value or `None` → bound as SQL NULL. F3 is the first caller that sends null, so no existing behaviour changes. * feat(library): patch-on-use wiring for star / rating / scrobble (F3) After a successful star/unstar, setRating, or play scrobble, mirror the change into the local library index via `library_patch_track` so its reads (browse F1, advanced search F2) reflect the action immediately — no stale list after a rate, no full resync. - `patchLibraryTrackOnUse` helper: fire-and-forget, gated on the index being enabled for the server; the Rust command additionally no-ops when no row matches (album/artist id, or index off). - Wired at the central API chokepoints: `star`/`unstar` (song only) → `starredAt`, `setRating` → `userRating`, `scrobbleSong` → `playedAt`. - `play_count` is left to the next sync (the patch sets absolute values; a correct increment needs the current base). F4 (deprecating the player-store override maps) is intentionally separate — removing them would break instant star feedback when the index is off. * feat(library): full-queue restore from the index on startup (PR-7 F5) (#832) Persist the whole queue as a lightweight ref list and rehydrate it from the local index on startup, so the entire queue survives a restart instead of only the windowed slice (R7-17 / §8.6). - Persist adds `queueRefs` (full ordered ids) + `queueRefsIndex` alongside the existing windowed `queue`. Ids are tiny; the windowed objects stay as the no-index fallback. - `hydrateQueueFromIndex` (startup, after session bind): when the library index is ready for the queue's server, hydrate the full queue via `library_get_tracks_batch` (batched ≤100), map `songToTrack ∘ trackToSong`, re-locate the current track so `queueIndex` stays aligned, then clear the refs. - Index not ready / missing rows / current track not found → keep the windowed fallback (queue never empty when the index is off, the P6 default). Old persisted shape without refs loads unchanged. - `trackToSong` exported from the F2 local-read adapters (one mapper). Kept the windowed-objects persist (did not drop the cap per R7-17 note): the index-off default needs the embedded fallback or the queue would restore empty. * feat(library): pending-sync for song star/rating (PR-7 F4) (#833) * feat(library): central pending-sync helper for song star/rating (PR-7 F4) `queueSongStar` / `queueSongRating` (spec §6.5 / R7-18): set the player-store override optimistically, retry the Subsonic API with exponential backoff (flush on `online` / window focus), and on success clear the override + patch the in-memory Track so the UI stays correct without it. The F3 index patch-on-use runs inside the API layer, unchanged. - No rollback on the first network error (the override survives until the retry succeeds or the app restarts; overrides are session-only, not persisted). - Latest-toggle-wins coalescing + an identity guard so a fast re-toggle while a request is in flight can't retire the newer task. - v1: songs only. * feat(library): route song star/rating through the pending-sync helper (PR-7 F4) Replace the scattered optimistic-set + API-call + rollback logic with the single `queueSongStar` / `queueSongRating` helper across cucadmuh's named v1 surfaces: PlayerBar, FullscreenPlayer, MobilePlayerView, both context menus (song + queue row), both shortcut paths, the song-rating hook + player-bar stars, skip→1★, and AlbumDetail (song star + rating). The 30+ override read sites are unchanged — they already read `override ?? track`, and the override now clears on success. Standalone page toggles (Favorites, RandomMix, NowPlaying star) and the separate mini-player webview keep their existing path — no regression (a non-migrated override simply lingers as before) — and move to a follow-up. * feat(library): route remaining song star/rating sites through pending-sync (F4 follow-up) (#834) Migrate the three standalone song write sites left out of #833 onto the central queueSongStar / queueSongRating helper: - Favorites: handleRate + removeSong (un-star) - RandomMix: toggleSongStar (drops local try/catch rollback per no-rollback policy) - useNowPlayingStarLove: toggleStar (keeps local view state, helper owns override + retried sync) MiniContextMenu stays on its direct path (separate webview, no shared store). No behaviour change for album/artist rating paths. * feat(library): route playlist song star/rating through pending-sync (F4 follow-up) (#835) The playlist-detail star/rating hook was the last shared-store song write site still calling the Subsonic API directly. Route handleRate + handleToggleStar through queueSongRating / queueSongStar, matching the Favorites and RandomMix follow-ups; keep the local ratings/starredSongs view state, drop the inline override. MiniContextMenu remains on its direct path (separate webview). * feat(library): BPM range filter UI in Advanced Search (PR-7 F6) (#836) * feat(library-sync): parallel initial ingest (S2 + N1/S1 prefetch) Wire C11 ParallelismBudget (max 4 when idle) into InitialSyncRunner: parallel getAlbum for S2, up to 4 in-flight pages for N1/S1, and persist S2 cursor once per album-list page instead of per album. * fix(library-sync): defer scheduler during initial sync and improve ingest diagnostics Background delta/tombstone ticks every 30s were competing with IS-3 bulk ingest for the write mutex (20–60s lock waits on large libraries). Skip scheduler while sync_phase is initial_sync/probing or bulk ingest is active. Serialize ingest batch metrics as camelCase for DevTools, add bulk-ingest FTS/index suspension, combined cursor persist, write-op tracing, live local search, and library dev logging helpers. * fix(library-search): scoped FTS, cancel stale live search, skip 1-char queries Use column-scoped FTS for artists/albums/songs, min two graphemes for local FTS, capped match counts in Advanced Search, and title browse index (m004). Live Search aborts superseded network requests, passes requestEpoch to drop stale Rust FTS, and avoids search3 fallback for too-short queries. * fix(library-search): prefix FTS, fast subquery joins, hide BPM in Advanced Search Live and Advanced Search now use FTS5 prefix tokens ("metal"*) and limit bm25 ranking inside rowid subqueries so large libraries stay in the ms range. Advanced Search BPM filter is removed from the UI until enrichment ships. * feat(library-search): race local index vs search3, show first result Live Search and Advanced Search text queries run library and network backends in parallel; the faster source wins. Adds searchRace helper and search_race dev logging. * feat(library-index): multi-server UI, serial sync queue, scoped local search Add master library index toggle with per-server rows, offline retry, and a frontend sync queue so initial ingest runs one server at a time. Scope Live Search and Advanced Search to the sidebar music library filter via library_id and raw_json fallbacks; coerce numeric libraryId on ingest. Promote idle sync state to ready when a full sync stamp exists and block cross-server initial sync starts in Rust. * chore(library-store): compliance — clippy, i18n, CHANGELOG Fix clippy/tsc blockers (request structs, IngestPageCtx, type aliases), add library index strings to all 9 locales, and document the preview feature in CHANGELOG [1.47.0] with Psychotoxical + cucadmuh attribution. * docs(credits): library index preview contributions Credit Psychotoxical for the local library store foundation and cucadmuh for multi-server UI, scoped search, and i18n. Drop removed scan-trigger wording from PR #780 entry. * docs(release): link library index preview to PR #846 * docs(changelog): sort [1.47.0] entries by ascending PR number * docs(changelog): mark library index as Added in [1.47.0] * docs(changelog): restructure [1.47.0] into Added/Changed/Fixed Match 1.46.0 layout: new features in Added (incl. library index), enhancements in Changed, bug fixes in Fixed — PR ascending within each block. * fix(library): address PR #846 review — delta guard + FTS order Skip background scheduler delta when LibraryRuntime already has a foreground sync job for the same server. Preserve bm25 rowid ordering in live search track/artist/album fetches. * fix(library): address remaining PR #846 review items S2 resume persists current_album_id per album; same-server resync awaits the previous runner. N1 delta watermark uses strict less-than; Navidrome HTTP 500 detection is structured. Adds genre/year indexes, backoff jitter salt, LiveSearch failure toast, and user-facing search badge copy. * fix(library): close resync notify race and tighten FTS trigger test Use notify_one() so an early runner completion cannot lose the drain signal before same-server full resync awaits. FTS test now compares normalized trigger bodies from migration vs suspend/restore roundtrip. --------- Co-authored-by: Frank Stellmacher <171614930+Psychotoxical@users.noreply.github.com>
This commit is contained in:
cucadmuh
GitHub
@@ -0,0 +1,21 @@
|
||||
[package]
|
||||
name = "psysonic-library"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
rust-version.workspace = true
|
||||
publish = false
|
||||
|
||||
[dependencies]
|
||||
psysonic-core = { path = "../psysonic-core" }
|
||||
psysonic-integration = { path = "../psysonic-integration" }
|
||||
|
||||
tauri = { version = "2" }
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
serde_json = "1"
|
||||
rusqlite = { version = "0.39", features = ["bundled"] }
|
||||
reqwest = { version = "0.13", default-features = false, features = ["json", "rustls", "gzip", "brotli"] }
|
||||
tokio = { version = "1", features = ["rt", "rt-multi-thread", "sync", "macros", "time"] }
|
||||
|
||||
[dev-dependencies]
|
||||
tokio = { version = "1", features = ["rt", "macros", "rt-multi-thread", "test-util"] }
|
||||
wiremock = { workspace = true }
|
||||
@@ -0,0 +1,247 @@
|
||||
-- psysonic-library v1 schema — see implementation-spec.ru.md §5.1–§5.3
|
||||
-- Tables are ordered so FK targets exist before their referrers.
|
||||
|
||||
-- Migration-runner bookkeeping (§5.7). Defined here so the schema file is
|
||||
-- self-describing — `LibraryStore::run_migrations` also creates this table
|
||||
-- defensively before applying migrations, which keeps both paths idempotent.
|
||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
version INTEGER PRIMARY KEY,
|
||||
applied_at INTEGER NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE canonical_track (
|
||||
id TEXT PRIMARY KEY,
|
||||
created_at INTEGER NOT NULL,
|
||||
updated_at INTEGER NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE canonical_identity (
|
||||
canonical_id TEXT NOT NULL,
|
||||
kind TEXT NOT NULL,
|
||||
value TEXT NOT NULL,
|
||||
confidence REAL NOT NULL DEFAULT 1.0,
|
||||
PRIMARY KEY (kind, value),
|
||||
FOREIGN KEY (canonical_id) REFERENCES canonical_track(id)
|
||||
);
|
||||
|
||||
CREATE TABLE sync_state (
|
||||
server_id TEXT NOT NULL,
|
||||
library_scope TEXT NOT NULL DEFAULT '',
|
||||
normalized_base_url TEXT NOT NULL DEFAULT '',
|
||||
server_fingerprint_ok INTEGER,
|
||||
fingerprint_checked_at INTEGER,
|
||||
capability_flags INTEGER NOT NULL DEFAULT 0,
|
||||
last_full_sync_at INTEGER,
|
||||
last_delta_sync_at INTEGER,
|
||||
server_last_scan_iso TEXT,
|
||||
indexes_last_modified_ms INTEGER,
|
||||
artists_last_modified_ms INTEGER,
|
||||
server_track_count INTEGER,
|
||||
local_track_count INTEGER,
|
||||
artist_count INTEGER,
|
||||
library_tier TEXT NOT NULL DEFAULT 'unknown',
|
||||
poll_stats_json TEXT NOT NULL DEFAULT '{}',
|
||||
next_poll_at INTEGER,
|
||||
initial_sync_cursor_json TEXT NOT NULL DEFAULT '{}',
|
||||
sync_phase TEXT NOT NULL DEFAULT 'idle',
|
||||
last_error TEXT,
|
||||
PRIMARY KEY (server_id, library_scope)
|
||||
);
|
||||
|
||||
CREATE TABLE artist (
|
||||
server_id TEXT NOT NULL,
|
||||
id TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
album_count INTEGER,
|
||||
synced_at INTEGER NOT NULL,
|
||||
raw_json TEXT,
|
||||
PRIMARY KEY (server_id, id)
|
||||
);
|
||||
|
||||
CREATE TABLE album (
|
||||
server_id TEXT NOT NULL,
|
||||
id TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
artist TEXT,
|
||||
artist_id TEXT,
|
||||
song_count INTEGER,
|
||||
duration_sec INTEGER,
|
||||
year INTEGER,
|
||||
genre TEXT,
|
||||
cover_art_id TEXT,
|
||||
starred_at INTEGER,
|
||||
synced_at INTEGER NOT NULL,
|
||||
raw_json TEXT,
|
||||
PRIMARY KEY (server_id, id)
|
||||
);
|
||||
|
||||
CREATE TABLE track (
|
||||
server_id TEXT NOT NULL,
|
||||
id TEXT NOT NULL,
|
||||
title TEXT NOT NULL,
|
||||
title_sort TEXT,
|
||||
artist TEXT,
|
||||
artist_id TEXT,
|
||||
album TEXT NOT NULL DEFAULT '',
|
||||
album_id TEXT,
|
||||
album_artist TEXT,
|
||||
duration_sec INTEGER NOT NULL DEFAULT 0,
|
||||
track_number INTEGER,
|
||||
disc_number INTEGER,
|
||||
year INTEGER,
|
||||
genre TEXT,
|
||||
suffix TEXT,
|
||||
bit_rate INTEGER,
|
||||
size_bytes INTEGER,
|
||||
cover_art_id TEXT,
|
||||
starred_at INTEGER,
|
||||
user_rating INTEGER,
|
||||
play_count INTEGER,
|
||||
played_at INTEGER,
|
||||
server_path TEXT,
|
||||
library_id TEXT,
|
||||
isrc TEXT,
|
||||
mbid_recording TEXT,
|
||||
bpm INTEGER,
|
||||
replay_gain_track_db REAL,
|
||||
replay_gain_album_db REAL,
|
||||
content_hash TEXT,
|
||||
server_updated_at INTEGER,
|
||||
server_created_at INTEGER,
|
||||
deleted INTEGER NOT NULL DEFAULT 0,
|
||||
synced_at INTEGER NOT NULL,
|
||||
raw_json TEXT NOT NULL,
|
||||
PRIMARY KEY (server_id, id)
|
||||
);
|
||||
|
||||
CREATE VIRTUAL TABLE track_fts USING fts5(
|
||||
title, artist, album, album_artist, genre,
|
||||
content='track', content_rowid='rowid',
|
||||
tokenize='unicode61 remove_diacritics 2'
|
||||
);
|
||||
|
||||
CREATE TRIGGER track_ai AFTER INSERT ON track BEGIN
|
||||
INSERT INTO track_fts(rowid, title, artist, album, album_artist, genre)
|
||||
VALUES (new.rowid, new.title, new.artist, new.album, new.album_artist, new.genre);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER track_ad AFTER DELETE ON track BEGIN
|
||||
INSERT INTO track_fts(track_fts, rowid, title, artist, album, album_artist, genre)
|
||||
VALUES ('delete', old.rowid, old.title, old.artist, old.album, old.album_artist, old.genre);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER track_au AFTER UPDATE ON track BEGIN
|
||||
INSERT INTO track_fts(track_fts, rowid, title, artist, album, album_artist, genre)
|
||||
VALUES ('delete', old.rowid, old.title, old.artist, old.album, old.album_artist, old.genre);
|
||||
INSERT INTO track_fts(rowid, title, artist, album, album_artist, genre)
|
||||
VALUES (new.rowid, new.title, new.artist, new.album, new.album_artist, new.genre);
|
||||
END;
|
||||
|
||||
CREATE TABLE track_extension (
|
||||
server_id TEXT NOT NULL,
|
||||
track_id TEXT NOT NULL,
|
||||
kind TEXT NOT NULL,
|
||||
version INTEGER NOT NULL DEFAULT 1,
|
||||
payload BLOB NOT NULL,
|
||||
updated_at INTEGER NOT NULL,
|
||||
PRIMARY KEY (server_id, track_id, kind),
|
||||
FOREIGN KEY (server_id, track_id) REFERENCES track(server_id, id)
|
||||
);
|
||||
|
||||
-- NO FK to track: survives library purge when user keeps the cached file (§5.14).
|
||||
CREATE TABLE track_offline (
|
||||
server_id TEXT NOT NULL,
|
||||
track_id TEXT NOT NULL,
|
||||
local_path TEXT NOT NULL,
|
||||
file_size_bytes INTEGER,
|
||||
suffix TEXT,
|
||||
content_hash TEXT NOT NULL DEFAULT '',
|
||||
server_path TEXT,
|
||||
cached_at INTEGER NOT NULL,
|
||||
last_verified_at INTEGER,
|
||||
PRIMARY KEY (server_id, track_id)
|
||||
);
|
||||
|
||||
CREATE TABLE track_id_history (
|
||||
server_id TEXT NOT NULL,
|
||||
old_id TEXT NOT NULL,
|
||||
new_id TEXT NOT NULL,
|
||||
content_hash TEXT,
|
||||
server_path TEXT,
|
||||
remapped_at INTEGER NOT NULL,
|
||||
PRIMARY KEY (server_id, old_id)
|
||||
);
|
||||
|
||||
CREATE TABLE track_fact (
|
||||
server_id TEXT NOT NULL,
|
||||
track_id TEXT NOT NULL,
|
||||
fact_kind TEXT NOT NULL,
|
||||
value_real REAL,
|
||||
value_int INTEGER,
|
||||
value_text TEXT,
|
||||
unit TEXT,
|
||||
source_kind TEXT NOT NULL,
|
||||
source_id TEXT NOT NULL,
|
||||
source_detail TEXT,
|
||||
confidence REAL NOT NULL DEFAULT 1.0,
|
||||
content_hash TEXT,
|
||||
fetched_at INTEGER NOT NULL,
|
||||
expires_at INTEGER,
|
||||
PRIMARY KEY (server_id, track_id, fact_kind, source_kind, source_id),
|
||||
FOREIGN KEY (server_id, track_id) REFERENCES track(server_id, id)
|
||||
);
|
||||
|
||||
CREATE TABLE track_artifact (
|
||||
server_id TEXT NOT NULL,
|
||||
track_id TEXT NOT NULL,
|
||||
artifact_kind TEXT NOT NULL,
|
||||
format TEXT NOT NULL,
|
||||
language TEXT,
|
||||
source_kind TEXT NOT NULL,
|
||||
source_id TEXT NOT NULL,
|
||||
content_text TEXT,
|
||||
content_blob BLOB,
|
||||
content_bytes INTEGER NOT NULL DEFAULT 0,
|
||||
not_found INTEGER NOT NULL DEFAULT 0,
|
||||
content_hash TEXT,
|
||||
fetched_at INTEGER NOT NULL,
|
||||
expires_at INTEGER,
|
||||
PRIMARY KEY (server_id, track_id, artifact_kind, source_kind, source_id, format),
|
||||
FOREIGN KEY (server_id, track_id) REFERENCES track(server_id, id)
|
||||
);
|
||||
|
||||
CREATE TABLE track_canonical_link (
|
||||
server_id TEXT NOT NULL,
|
||||
track_id TEXT NOT NULL,
|
||||
canonical_id TEXT NOT NULL,
|
||||
match_method TEXT NOT NULL,
|
||||
confidence REAL NOT NULL,
|
||||
linked_at INTEGER NOT NULL,
|
||||
PRIMARY KEY (server_id, track_id),
|
||||
FOREIGN KEY (server_id, track_id) REFERENCES track(server_id, id),
|
||||
FOREIGN KEY (canonical_id) REFERENCES canonical_track(id)
|
||||
);
|
||||
|
||||
CREATE TABLE canonical_enrichment_link (
|
||||
canonical_id TEXT NOT NULL,
|
||||
enrichment_kind TEXT NOT NULL,
|
||||
owner_server_id TEXT NOT NULL,
|
||||
owner_track_id TEXT NOT NULL,
|
||||
share_policy TEXT NOT NULL DEFAULT 'isrc_match',
|
||||
linked_at INTEGER NOT NULL,
|
||||
PRIMARY KEY (canonical_id, enrichment_kind, owner_server_id, owner_track_id),
|
||||
FOREIGN KEY (canonical_id) REFERENCES canonical_track(id)
|
||||
);
|
||||
|
||||
CREATE INDEX idx_track_album ON track(server_id, album_id) WHERE deleted = 0;
|
||||
CREATE INDEX idx_track_artist ON track(server_id, artist_id) WHERE deleted = 0;
|
||||
CREATE INDEX idx_track_updated ON track(server_id, server_updated_at DESC) WHERE deleted = 0;
|
||||
CREATE INDEX idx_track_starred ON track(server_id, starred_at) WHERE deleted = 0 AND starred_at IS NOT NULL;
|
||||
CREATE INDEX idx_track_library ON track(server_id, library_id) WHERE deleted = 0;
|
||||
CREATE INDEX idx_track_bpm ON track(server_id, bpm) WHERE deleted = 0 AND bpm IS NOT NULL;
|
||||
CREATE INDEX idx_track_isrc ON track(isrc) WHERE deleted = 0 AND isrc IS NOT NULL;
|
||||
CREATE INDEX idx_track_fact_lookup ON track_fact(server_id, track_id, fact_kind);
|
||||
CREATE INDEX idx_track_artifact_lookup ON track_artifact(server_id, track_id, artifact_kind);
|
||||
CREATE INDEX idx_track_offline_hash ON track_offline(server_id, content_hash);
|
||||
CREATE INDEX idx_track_id_history_new ON track_id_history(server_id, new_id);
|
||||
CREATE INDEX idx_canonical_identity_lookup ON canonical_identity(kind, value);
|
||||
@@ -0,0 +1,6 @@
|
||||
-- psysonic-library schema v2 — large-library ingest policy (R7-15).
|
||||
-- Per-server learned flag: when N1 (`/api/song`) returns HTTP 500 beyond a
|
||||
-- deep offset on a large catalog, the strategy selector stops choosing N1 for
|
||||
-- that server on future initial syncs (spec §6.3 / R7-15 Q1/Q5). Additive
|
||||
-- column, DEFAULT 0 → existing rows keep N1 eligible until they hit the wall.
|
||||
ALTER TABLE sync_state ADD COLUMN n1_bulk_unreliable INTEGER NOT NULL DEFAULT 0;
|
||||
@@ -0,0 +1,10 @@
|
||||
-- Remap detection (§6.9) and unstable-id servers: without these indexes
|
||||
-- each upsert in a 500-row batch can scan the whole track table.
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_track_remap_path
|
||||
ON track(server_id, server_path)
|
||||
WHERE deleted = 0 AND server_path IS NOT NULL AND server_path != '';
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_track_remap_hash
|
||||
ON track(server_id, content_hash)
|
||||
WHERE deleted = 0 AND content_hash IS NOT NULL AND content_hash != '';
|
||||
@@ -0,0 +1,4 @@
|
||||
-- Browse / sort-by-title without sorting the full server slice on every page.
|
||||
CREATE INDEX IF NOT EXISTS idx_track_title
|
||||
ON track(server_id, title COLLATE NOCASE)
|
||||
WHERE deleted = 0;
|
||||
@@ -0,0 +1,8 @@
|
||||
-- Advanced search filters on genre and year (partial indexes — only non-null rows).
|
||||
CREATE INDEX IF NOT EXISTS idx_track_genre
|
||||
ON track(server_id, genre COLLATE NOCASE)
|
||||
WHERE deleted = 0 AND genre IS NOT NULL;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_track_year
|
||||
ON track(server_id, year)
|
||||
WHERE deleted = 0 AND year IS NOT NULL;
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,84 @@
|
||||
//! IS-3 bulk ingest tuning — drop hot-path indexes, restore at end.
|
||||
//!
|
||||
//! Secondary `track` indexes (album/artist/remap/…) are useless during the
|
||||
//! initial upsert-only pass but cost several index inserts per row. Dropping
|
||||
//! them for IS-3 keeps batch writes flat into the tens-of-ms range on large
|
||||
//! libraries; they are recreated once before FTS rebuild.
|
||||
|
||||
use rusqlite::Connection;
|
||||
|
||||
const DROP_TRACK_SECONDARY_INDEXES: &str = r#"
|
||||
DROP INDEX IF EXISTS idx_track_album;
|
||||
DROP INDEX IF EXISTS idx_track_artist;
|
||||
DROP INDEX IF EXISTS idx_track_updated;
|
||||
DROP INDEX IF EXISTS idx_track_starred;
|
||||
DROP INDEX IF EXISTS idx_track_library;
|
||||
DROP INDEX IF EXISTS idx_track_bpm;
|
||||
DROP INDEX IF EXISTS idx_track_isrc;
|
||||
DROP INDEX IF EXISTS idx_track_remap_path;
|
||||
DROP INDEX IF EXISTS idx_track_remap_hash;
|
||||
DROP INDEX IF EXISTS idx_track_title;
|
||||
"#;
|
||||
|
||||
const RESTORE_TRACK_SECONDARY_INDEXES: &str = r#"
|
||||
CREATE INDEX IF NOT EXISTS idx_track_album ON track(server_id, album_id) WHERE deleted = 0;
|
||||
CREATE INDEX IF NOT EXISTS idx_track_artist ON track(server_id, artist_id) WHERE deleted = 0;
|
||||
CREATE INDEX IF NOT EXISTS idx_track_updated ON track(server_id, server_updated_at DESC) WHERE deleted = 0;
|
||||
CREATE INDEX IF NOT EXISTS idx_track_starred ON track(server_id, starred_at) WHERE deleted = 0 AND starred_at IS NOT NULL;
|
||||
CREATE INDEX IF NOT EXISTS idx_track_library ON track(server_id, library_id) WHERE deleted = 0;
|
||||
CREATE INDEX IF NOT EXISTS idx_track_bpm ON track(server_id, bpm) WHERE deleted = 0 AND bpm IS NOT NULL;
|
||||
CREATE INDEX IF NOT EXISTS idx_track_isrc ON track(isrc) WHERE deleted = 0 AND isrc IS NOT NULL;
|
||||
CREATE INDEX IF NOT EXISTS idx_track_remap_path
|
||||
ON track(server_id, server_path)
|
||||
WHERE deleted = 0 AND server_path IS NOT NULL AND server_path != '';
|
||||
CREATE INDEX IF NOT EXISTS idx_track_remap_hash
|
||||
ON track(server_id, content_hash)
|
||||
WHERE deleted = 0 AND content_hash IS NOT NULL AND content_hash != '';
|
||||
CREATE INDEX IF NOT EXISTS idx_track_title
|
||||
ON track(server_id, title COLLATE NOCASE)
|
||||
WHERE deleted = 0;
|
||||
"#;
|
||||
|
||||
/// Drop secondary indexes on `track` so bulk upserts only touch the PK.
|
||||
pub fn suspend_track_secondary_indexes(conn: &Connection) -> rusqlite::Result<()> {
|
||||
conn.execute_batch(DROP_TRACK_SECONDARY_INDEXES)
|
||||
}
|
||||
|
||||
/// Recreate secondary indexes after bulk ingest (may take tens of seconds on
|
||||
/// very large libraries — runs once at the end of IS-3, not per batch).
|
||||
pub fn restore_track_secondary_indexes(conn: &Connection) -> rusqlite::Result<()> {
|
||||
conn.execute_batch(RESTORE_TRACK_SECONDARY_INDEXES)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
#[test]
|
||||
fn suspend_and_restore_track_indexes_roundtrip() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
store
|
||||
.with_conn_mut("misc", |conn| {
|
||||
suspend_track_secondary_indexes(conn)?;
|
||||
conn.execute(
|
||||
"INSERT INTO track (server_id, id, title, album, album_id, artist_id, \
|
||||
duration_sec, deleted, synced_at, raw_json) \
|
||||
VALUES ('s1', 't1', 'T', 'Al', 'al1', 'ar1', 1, 0, 1, '{}')",
|
||||
[],
|
||||
)?;
|
||||
restore_track_secondary_indexes(conn)
|
||||
})
|
||||
.unwrap();
|
||||
let n: i64 = store
|
||||
.with_read_conn(|c| {
|
||||
c.query_row(
|
||||
"SELECT COUNT(*) FROM track WHERE server_id = 's1' AND album_id = 'al1'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(n, 1);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,264 @@
|
||||
//! H1 — `CanonicalMatcher`: cross-server track identity by **strong keys
|
||||
//! only** (ISRC, then MBID recording), per spec §5.5A / P13.
|
||||
//!
|
||||
//! Runs inline on ingest and is O(1) per track — a deterministic canonical
|
||||
//! id (`{kind}:{value}`) plus `INSERT … ON CONFLICT DO NOTHING`, so there's
|
||||
//! no lookup-then-create race and no fuzzy loop on the 500k upsert path
|
||||
//! (the fuzzy / album-similarity layer is search-time only — §5.5B / H3).
|
||||
//!
|
||||
//! v1 does **not** merge across kinds (a recording seen with an ISRC on one
|
||||
//! server and only an MBID on another yields two canonical rows). Spec §5.5A
|
||||
//! accepts this; a future merge layer would remap ids without a schema break.
|
||||
|
||||
use rusqlite::{params, OptionalExtension, Transaction};
|
||||
|
||||
pub const KIND_ISRC: &str = "isrc";
|
||||
pub const KIND_MBID: &str = "mbid_recording";
|
||||
|
||||
/// Resolve (create-if-absent) the canonical id for one strong identity.
|
||||
/// Idempotent: the deterministic id means re-running over an existing
|
||||
/// identity is a pair of no-op upserts.
|
||||
pub fn resolve_canonical_id(
|
||||
tx: &Transaction<'_>,
|
||||
kind: &str,
|
||||
value: &str,
|
||||
now: i64,
|
||||
) -> rusqlite::Result<String> {
|
||||
let canonical_id = format!("{kind}:{value}");
|
||||
tx.execute(
|
||||
"INSERT INTO canonical_track (id, created_at, updated_at) VALUES (?1, ?2, ?2) \
|
||||
ON CONFLICT(id) DO NOTHING",
|
||||
params![canonical_id, now],
|
||||
)?;
|
||||
tx.execute(
|
||||
"INSERT INTO canonical_identity (canonical_id, kind, value, confidence) \
|
||||
VALUES (?1, ?2, ?3, 1.0) \
|
||||
ON CONFLICT(kind, value) DO NOTHING",
|
||||
params![canonical_id, kind, value],
|
||||
)?;
|
||||
Ok(canonical_id)
|
||||
}
|
||||
|
||||
/// Link `(server_id, track_id)` to its canonical id for whichever strong key
|
||||
/// it carries — ISRC preferred, then MBID recording (§5.5A). No-op (returns
|
||||
/// `Ok(None)`) when the track has neither. Idempotent on re-ingest.
|
||||
pub fn link_track(
|
||||
tx: &Transaction<'_>,
|
||||
server_id: &str,
|
||||
track_id: &str,
|
||||
isrc: Option<&str>,
|
||||
mbid_recording: Option<&str>,
|
||||
now: i64,
|
||||
) -> rusqlite::Result<Option<String>> {
|
||||
let (kind, value) = match (
|
||||
isrc.filter(|s| !s.is_empty()),
|
||||
mbid_recording.filter(|s| !s.is_empty()),
|
||||
) {
|
||||
(Some(isrc), _) => (KIND_ISRC, isrc),
|
||||
(None, Some(mbid)) => (KIND_MBID, mbid),
|
||||
(None, None) => return Ok(None),
|
||||
};
|
||||
|
||||
let canonical_id = resolve_canonical_id(tx, kind, value, now)?;
|
||||
tx.execute(
|
||||
"INSERT INTO track_canonical_link \
|
||||
(server_id, track_id, canonical_id, match_method, confidence, linked_at) \
|
||||
VALUES (?1, ?2, ?3, ?4, 1.0, ?5) \
|
||||
ON CONFLICT(server_id, track_id) DO UPDATE SET \
|
||||
canonical_id = excluded.canonical_id, \
|
||||
match_method = excluded.match_method, \
|
||||
confidence = excluded.confidence, \
|
||||
linked_at = excluded.linked_at",
|
||||
params![server_id, track_id, canonical_id, kind, now],
|
||||
)?;
|
||||
Ok(Some(canonical_id))
|
||||
}
|
||||
|
||||
/// Link every track on `server_id` that carries a strong identity key.
|
||||
/// Used once after IS-3 bulk ingest instead of per-row inline linking.
|
||||
pub fn link_all_tracks_for_server(
|
||||
store: &crate::store::LibraryStore,
|
||||
server_id: &str,
|
||||
now: i64,
|
||||
) -> Result<u32, String> {
|
||||
store.with_conn_mut("misc", |conn| {
|
||||
let tx = conn.transaction()?;
|
||||
let mut stmt = tx.prepare(
|
||||
"SELECT id, isrc, mbid_recording FROM track \
|
||||
WHERE server_id = ?1 AND deleted = 0 \
|
||||
AND (\
|
||||
(isrc IS NOT NULL AND isrc != '') \
|
||||
OR (mbid_recording IS NOT NULL AND mbid_recording != '')\
|
||||
)",
|
||||
)?;
|
||||
let rows: Vec<(String, Option<String>, Option<String>)> = stmt
|
||||
.query_map(params![server_id], |r| {
|
||||
Ok((r.get(0)?, r.get(1)?, r.get(2)?))
|
||||
})?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||
drop(stmt);
|
||||
let mut linked = 0u32;
|
||||
for (track_id, isrc, mbid) in rows {
|
||||
if link_track(
|
||||
&tx,
|
||||
server_id,
|
||||
&track_id,
|
||||
isrc.as_deref(),
|
||||
mbid.as_deref(),
|
||||
now,
|
||||
)?
|
||||
.is_some()
|
||||
{
|
||||
linked = linked.saturating_add(1);
|
||||
}
|
||||
}
|
||||
tx.commit()?;
|
||||
Ok(linked)
|
||||
})
|
||||
}
|
||||
|
||||
/// Read a track's canonical id, if linked. Convenience for tests / callers.
|
||||
pub fn canonical_id_for(
|
||||
tx: &Transaction<'_>,
|
||||
server_id: &str,
|
||||
track_id: &str,
|
||||
) -> rusqlite::Result<Option<String>> {
|
||||
tx.query_row(
|
||||
"SELECT canonical_id FROM track_canonical_link WHERE server_id = ?1 AND track_id = ?2",
|
||||
params![server_id, track_id],
|
||||
|r| r.get(0),
|
||||
)
|
||||
.optional()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
fn with_tx<R>(store: &LibraryStore, f: impl FnOnce(&Transaction<'_>) -> R) -> R {
|
||||
store
|
||||
.with_conn_mut("misc", |conn| {
|
||||
let tx = conn.transaction()?;
|
||||
let out = f(&tx);
|
||||
tx.commit()?;
|
||||
Ok(out)
|
||||
})
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
/// `track_canonical_link` has an FK to `track` — seed a minimal row so
|
||||
/// the link insert is valid (foreign_keys are ON in-memory).
|
||||
fn seed_track(tx: &Transaction<'_>, server: &str, id: &str) {
|
||||
tx.execute(
|
||||
"INSERT INTO track (server_id, id, title, synced_at, raw_json) \
|
||||
VALUES (?1, ?2, 'T', 1, '{}')",
|
||||
params![server, id],
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resolve_is_deterministic_and_idempotent() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let (a, b) = with_tx(&store, |tx| {
|
||||
let a = resolve_canonical_id(tx, KIND_ISRC, "USRC123", 1).unwrap();
|
||||
let b = resolve_canonical_id(tx, KIND_ISRC, "USRC123", 2).unwrap();
|
||||
(a, b)
|
||||
});
|
||||
assert_eq!(a, "isrc:USRC123");
|
||||
assert_eq!(a, b, "same identity → same canonical id");
|
||||
|
||||
let (tracks, identities): (i64, i64) = store
|
||||
.with_conn("misc", |c| {
|
||||
Ok((
|
||||
c.query_row("SELECT COUNT(*) FROM canonical_track", [], |r| r.get(0))?,
|
||||
c.query_row("SELECT COUNT(*) FROM canonical_identity", [], |r| r.get(0))?,
|
||||
))
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(tracks, 1, "idempotent — no duplicate canonical_track");
|
||||
assert_eq!(identities, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn link_prefers_isrc_over_mbid() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let cid = with_tx(&store, |tx| {
|
||||
seed_track(tx, "s1", "t1");
|
||||
link_track(tx, "s1", "t1", Some("USRC1"), Some("mbid-1"), 1).unwrap()
|
||||
});
|
||||
assert_eq!(cid.as_deref(), Some("isrc:USRC1"));
|
||||
let stored =
|
||||
with_tx(&store, |tx| canonical_id_for(tx, "s1", "t1").unwrap());
|
||||
assert_eq!(stored.as_deref(), Some("isrc:USRC1"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn link_uses_mbid_when_no_isrc() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let cid = with_tx(&store, |tx| {
|
||||
seed_track(tx, "s1", "t1");
|
||||
link_track(tx, "s1", "t1", None, Some("mbid-9"), 1).unwrap()
|
||||
});
|
||||
assert_eq!(cid.as_deref(), Some("mbid_recording:mbid-9"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn link_is_noop_without_strong_keys() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let cid = with_tx(&store, |tx| {
|
||||
// empty strings count as absent
|
||||
link_track(tx, "s1", "t1", Some(""), None, 1).unwrap()
|
||||
});
|
||||
assert!(cid.is_none());
|
||||
let count: i64 = store
|
||||
.with_conn("misc", |c| c.query_row("SELECT COUNT(*) FROM track_canonical_link", [], |r| r.get(0)))
|
||||
.unwrap();
|
||||
assert_eq!(count, 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn same_isrc_across_servers_shares_one_canonical_id() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let (a, b) = with_tx(&store, |tx| {
|
||||
seed_track(tx, "s1", "t1");
|
||||
seed_track(tx, "s2", "t9");
|
||||
let a = link_track(tx, "s1", "t1", Some("USRC7"), None, 1).unwrap();
|
||||
let b = link_track(tx, "s2", "t9", Some("USRC7"), None, 1).unwrap();
|
||||
(a, b)
|
||||
});
|
||||
assert_eq!(a, b);
|
||||
let canon_count: i64 = store
|
||||
.with_conn("misc", |c| c.query_row("SELECT COUNT(*) FROM canonical_track", [], |r| r.get(0)))
|
||||
.unwrap();
|
||||
assert_eq!(canon_count, 1, "one canonical row shared across two servers");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn relink_updates_existing_row_in_place() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
with_tx(&store, |tx| {
|
||||
seed_track(tx, "s1", "t1");
|
||||
link_track(tx, "s1", "t1", None, Some("mbid-old"), 1).unwrap();
|
||||
// Later ingest now carries an ISRC — re-link overwrites.
|
||||
link_track(tx, "s1", "t1", Some("USRCnew"), Some("mbid-old"), 2).unwrap();
|
||||
});
|
||||
let (cid, method): (String, String) = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT canonical_id, match_method FROM track_canonical_link \
|
||||
WHERE server_id = 's1' AND track_id = 't1'",
|
||||
[],
|
||||
|r| Ok((r.get(0)?, r.get(1)?)),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(cid, "isrc:USRCnew");
|
||||
assert_eq!(method, "isrc");
|
||||
let link_count: i64 = store
|
||||
.with_conn("misc", |c| c.query_row("SELECT COUNT(*) FROM track_canonical_link", [], |r| r.get(0)))
|
||||
.unwrap();
|
||||
assert_eq!(link_count, 1, "re-link updates in place, no duplicate");
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,376 @@
|
||||
//! Cross-server search (spec §5.5B / §5.9). Primary FTS union (`hits`,
|
||||
//! bm25-ordered, deduped by canonical id) plus the H3 fuzzy fallback
|
||||
//! (`fuzzy`): per-server `title LIKE` matches the exact FTS pass missed.
|
||||
//! UI wiring stays PR-7.
|
||||
|
||||
use std::collections::HashSet;
|
||||
|
||||
use rusqlite::types::Value as SqlValue;
|
||||
|
||||
use crate::dto::{LibraryCrossServerSearchResponse, LibraryTrackDto};
|
||||
use crate::repos;
|
||||
use crate::search::{aliased_track_columns, fts_query, like_contains, PAGE_LIMIT_MAX};
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
/// §5.9 caps the fuzzy fallback at "top 20 per server" so a `LIKE %…%` scan
|
||||
/// (no index) stays bounded.
|
||||
const FUZZY_PER_SERVER_CAP: usize = 20;
|
||||
|
||||
/// `library_search_cross_server` (§5.5B / §5.9 A′). Primary FTS union over
|
||||
/// the requested servers (or all `ready` servers), bm25-ordered, deduped by
|
||||
/// canonical id where a `track_canonical_link` row exists.
|
||||
pub fn run_cross_server_search(
|
||||
store: &LibraryStore,
|
||||
query: &str,
|
||||
limit: u32,
|
||||
servers: Option<&[String]>,
|
||||
) -> Result<LibraryCrossServerSearchResponse, String> {
|
||||
let limit = limit.clamp(1, PAGE_LIMIT_MAX);
|
||||
let Some(fts) = fts_query(query) else {
|
||||
return Ok(LibraryCrossServerSearchResponse::default());
|
||||
};
|
||||
|
||||
// Explicit `servers` is an override (caller's choice); otherwise default
|
||||
// to every server whose index is `ready` (§5.9).
|
||||
let targets: Vec<String> = match servers {
|
||||
Some(list) if !list.is_empty() => list.to_vec(),
|
||||
_ => ready_servers(store)?,
|
||||
};
|
||||
if targets.is_empty() {
|
||||
return Ok(LibraryCrossServerSearchResponse::default());
|
||||
}
|
||||
|
||||
let placeholders = vec!["?"; targets.len()].join(", ");
|
||||
let cols = aliased_track_columns("t");
|
||||
let sql = format!(
|
||||
"SELECT {cols}, l.canonical_id \
|
||||
FROM track_fts f \
|
||||
JOIN track t ON t.rowid = f.rowid \
|
||||
LEFT JOIN track_canonical_link l ON l.server_id = t.server_id AND l.track_id = t.id \
|
||||
WHERE track_fts MATCH ? AND t.deleted = 0 AND t.server_id IN ({placeholders}) \
|
||||
ORDER BY bm25(track_fts) LIMIT ?"
|
||||
);
|
||||
|
||||
let mut params: Vec<SqlValue> = Vec::with_capacity(targets.len() + 2);
|
||||
params.push(SqlValue::Text(fts));
|
||||
for s in &targets {
|
||||
params.push(SqlValue::Text(s.clone()));
|
||||
}
|
||||
params.push(SqlValue::Integer(limit as i64));
|
||||
|
||||
let canonical_idx = repos::track_columns().split(',').count();
|
||||
let rows: Vec<(LibraryTrackDto, Option<String>)> = store.with_read_conn(|conn| {
|
||||
let mut stmt = conn.prepare(&sql)?;
|
||||
// Bind the collected `Result` before unwrapping so the `MappedRows`
|
||||
// borrow of `stmt` ends inside the block (rusqlite borrow quirk).
|
||||
let collected: rusqlite::Result<Vec<(LibraryTrackDto, Option<String>)>> = stmt
|
||||
.query_map(rusqlite::params_from_iter(params.iter()), |r| {
|
||||
let track = repos::row_to_track_row(r).map(|row| LibraryTrackDto::from_row(&row))?;
|
||||
let canonical: Option<String> = r.get(canonical_idx)?;
|
||||
Ok((track, canonical))
|
||||
})?
|
||||
.collect();
|
||||
collected
|
||||
})?;
|
||||
|
||||
// Dedup the exact hits by canonical id (§5.5B step 2). Rows with no
|
||||
// canonical link are always kept — the link table is sparse for tracks
|
||||
// lacking ISRC/MBID.
|
||||
let mut seen: HashSet<String> = HashSet::new();
|
||||
let mut hits: Vec<LibraryTrackDto> = Vec::with_capacity(rows.len());
|
||||
for (track, canonical) in rows {
|
||||
if let Some(cid) = canonical {
|
||||
if !seen.insert(cid) {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
hits.push(track);
|
||||
}
|
||||
|
||||
// H3 fuzzy fallback (§5.9): catch what the exact FTS pass missed.
|
||||
let hit_keys: HashSet<(String, String)> = hits
|
||||
.iter()
|
||||
.map(|t| (t.server_id.clone(), t.id.clone()))
|
||||
.collect();
|
||||
let fuzzy = fuzzy_matches(store, &targets, query.trim(), &mut seen, &hit_keys, limit as usize)?;
|
||||
|
||||
Ok(LibraryCrossServerSearchResponse {
|
||||
hits,
|
||||
fuzzy,
|
||||
servers_searched: targets,
|
||||
})
|
||||
}
|
||||
|
||||
/// §5.9 fuzzy fallback: per target server, `title LIKE %query%` for matches
|
||||
/// the exact FTS pass missed (diacritics, partial words). Skips rows already
|
||||
/// in `hit_keys` and dedupes by canonical id against `seen` (which holds the
|
||||
/// exact hits' canonical ids). Capped per server (`FUZZY_PER_SERVER_CAP`) and
|
||||
/// overall (`overall_cap`).
|
||||
fn fuzzy_matches(
|
||||
store: &LibraryStore,
|
||||
targets: &[String],
|
||||
query: &str,
|
||||
seen: &mut HashSet<String>,
|
||||
hit_keys: &HashSet<(String, String)>,
|
||||
overall_cap: usize,
|
||||
) -> Result<Vec<LibraryTrackDto>, String> {
|
||||
let like = like_contains(query);
|
||||
let cols = aliased_track_columns("t");
|
||||
let canonical_idx = repos::track_columns().split(',').count();
|
||||
let sql = format!(
|
||||
"SELECT {cols}, l.canonical_id \
|
||||
FROM track t \
|
||||
LEFT JOIN track_canonical_link l ON l.server_id = t.server_id AND l.track_id = t.id \
|
||||
WHERE t.server_id = ? AND t.deleted = 0 AND t.title LIKE ? ESCAPE '\\' \
|
||||
ORDER BY t.title COLLATE NOCASE ASC LIMIT ?"
|
||||
);
|
||||
|
||||
let mut out: Vec<LibraryTrackDto> = Vec::new();
|
||||
for server in targets {
|
||||
if out.len() >= overall_cap {
|
||||
break;
|
||||
}
|
||||
let bound = [
|
||||
SqlValue::Text(server.clone()),
|
||||
SqlValue::Text(like.clone()),
|
||||
SqlValue::Integer(FUZZY_PER_SERVER_CAP as i64),
|
||||
];
|
||||
let rows: Vec<(LibraryTrackDto, Option<String>)> = store.with_read_conn(|conn| {
|
||||
let mut stmt = conn.prepare(&sql)?;
|
||||
let collected: rusqlite::Result<Vec<(LibraryTrackDto, Option<String>)>> = stmt
|
||||
.query_map(rusqlite::params_from_iter(bound.iter()), |r| {
|
||||
let track =
|
||||
repos::row_to_track_row(r).map(|row| LibraryTrackDto::from_row(&row))?;
|
||||
let canonical: Option<String> = r.get(canonical_idx)?;
|
||||
Ok((track, canonical))
|
||||
})?
|
||||
.collect();
|
||||
collected
|
||||
})?;
|
||||
|
||||
for (track, canonical) in rows {
|
||||
if out.len() >= overall_cap {
|
||||
break;
|
||||
}
|
||||
if hit_keys.contains(&(track.server_id.clone(), track.id.clone())) {
|
||||
continue; // already an exact hit
|
||||
}
|
||||
if let Some(cid) = canonical {
|
||||
if !seen.insert(cid) {
|
||||
continue; // canonical already represented (hit or earlier fuzzy)
|
||||
}
|
||||
}
|
||||
out.push(track);
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn ready_servers(store: &LibraryStore) -> Result<Vec<String>, String> {
|
||||
store.with_read_conn(|conn| {
|
||||
let mut stmt =
|
||||
conn.prepare("SELECT DISTINCT server_id FROM sync_state WHERE sync_phase = 'ready'")?;
|
||||
let collected: rusqlite::Result<Vec<String>> =
|
||||
stmt.query_map([], |r| r.get(0))?.collect();
|
||||
collected
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::repos::{TrackRepository, TrackRow};
|
||||
|
||||
fn track(server: &str, id: &str, title: &str, artist: &str, album: &str) -> TrackRow {
|
||||
TrackRow {
|
||||
server_id: server.into(),
|
||||
id: id.into(),
|
||||
title: title.into(),
|
||||
title_sort: None,
|
||||
artist: Some(artist.into()),
|
||||
artist_id: Some(format!("ar_{artist}")),
|
||||
album: album.into(),
|
||||
album_id: Some(format!("al_{album}")),
|
||||
album_artist: Some(artist.into()),
|
||||
duration_sec: 200,
|
||||
track_number: Some(1),
|
||||
disc_number: Some(1),
|
||||
year: None,
|
||||
genre: None,
|
||||
suffix: None,
|
||||
bit_rate: None,
|
||||
size_bytes: None,
|
||||
cover_art_id: None,
|
||||
starred_at: None,
|
||||
user_rating: None,
|
||||
play_count: None,
|
||||
played_at: None,
|
||||
server_path: None,
|
||||
library_id: None,
|
||||
isrc: None,
|
||||
mbid_recording: None,
|
||||
bpm: None,
|
||||
replay_gain_track_db: None,
|
||||
replay_gain_album_db: None,
|
||||
content_hash: None,
|
||||
server_updated_at: None,
|
||||
server_created_at: None,
|
||||
deleted: false,
|
||||
synced_at: 1,
|
||||
raw_json: "{}".into(),
|
||||
}
|
||||
}
|
||||
|
||||
fn set_phase(store: &LibraryStore, server: &str, phase: &str) {
|
||||
store
|
||||
.with_conn("misc", |c| {
|
||||
c.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, sync_phase) \
|
||||
VALUES (?1, '', ?2) \
|
||||
ON CONFLICT(server_id, library_scope) DO UPDATE SET sync_phase = excluded.sync_phase",
|
||||
rusqlite::params![server, phase],
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn union_searches_ready_servers_only() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
TrackRepository::new(&store)
|
||||
.upsert_batch(&[
|
||||
track("s1", "t1", "Aurora", "Anna", "Alb"),
|
||||
track("s2", "t2", "Aurora", "Beth", "Alb"),
|
||||
track("s3", "t3", "Aurora", "Cara", "Alb"),
|
||||
])
|
||||
.unwrap();
|
||||
set_phase(&store, "s1", "ready");
|
||||
set_phase(&store, "s2", "ready");
|
||||
set_phase(&store, "s3", "idle"); // not ready → excluded
|
||||
let resp = run_cross_server_search(&store, "aurora", 50, None).unwrap();
|
||||
let servers: HashSet<&str> = resp.hits.iter().map(|t| t.server_id.as_str()).collect();
|
||||
assert_eq!(servers, HashSet::from(["s1", "s2"]));
|
||||
assert_eq!(resp.servers_searched.len(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn explicit_servers_override_ready_gate() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
TrackRepository::new(&store)
|
||||
.upsert_batch(&[track("s9", "t1", "Aurora", "Anna", "Alb")])
|
||||
.unwrap();
|
||||
// s9 is not marked ready, but an explicit servers list overrides.
|
||||
let resp = run_cross_server_search(&store, "aurora", 50, Some(&["s9".to_string()])).unwrap();
|
||||
assert_eq!(resp.hits.len(), 1);
|
||||
assert_eq!(resp.servers_searched, vec!["s9".to_string()]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn dedups_by_canonical_id() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
TrackRepository::new(&store)
|
||||
.upsert_batch(&[
|
||||
track("s1", "t1", "Aurora", "Anna", "Alb"),
|
||||
track("s2", "t2", "Aurora", "Anna", "Alb"),
|
||||
])
|
||||
.unwrap();
|
||||
// Both tracks link to the same canonical id → one survives.
|
||||
store
|
||||
.with_conn("misc", |c| {
|
||||
c.execute(
|
||||
"INSERT INTO canonical_track (id, created_at, updated_at) VALUES ('can1', 1, 1)",
|
||||
[],
|
||||
)?;
|
||||
for (s, t) in [("s1", "t1"), ("s2", "t2")] {
|
||||
c.execute(
|
||||
"INSERT INTO track_canonical_link \
|
||||
(server_id, track_id, canonical_id, match_method, confidence, linked_at) \
|
||||
VALUES (?1, ?2, 'can1', 'isrc', 1.0, 1)",
|
||||
rusqlite::params![s, t],
|
||||
)?;
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
.unwrap();
|
||||
set_phase(&store, "s1", "ready");
|
||||
set_phase(&store, "s2", "ready");
|
||||
let resp = run_cross_server_search(&store, "aurora", 50, None).unwrap();
|
||||
assert_eq!(resp.hits.len(), 1, "duplicate canonical id collapses to one hit");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unlinked_rows_are_never_deduped() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
TrackRepository::new(&store)
|
||||
.upsert_batch(&[
|
||||
track("s1", "t1", "Aurora", "Anna", "Alb"),
|
||||
track("s2", "t2", "Aurora", "Beth", "Alb"),
|
||||
])
|
||||
.unwrap();
|
||||
set_phase(&store, "s1", "ready");
|
||||
set_phase(&store, "s2", "ready");
|
||||
// No canonical links → both kept even though titles match.
|
||||
let resp = run_cross_server_search(&store, "aurora", 50, None).unwrap();
|
||||
assert_eq!(resp.hits.len(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_query_returns_empty() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
TrackRepository::new(&store)
|
||||
.upsert_batch(&[track("s1", "t1", "Aurora", "Anna", "Alb")])
|
||||
.unwrap();
|
||||
set_phase(&store, "s1", "ready");
|
||||
let resp = run_cross_server_search(&store, " ", 50, None).unwrap();
|
||||
assert!(resp.hits.is_empty());
|
||||
assert!(resp.servers_searched.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_ready_servers_returns_empty() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
TrackRepository::new(&store)
|
||||
.upsert_batch(&[track("s1", "t1", "Aurora", "Anna", "Alb")])
|
||||
.unwrap();
|
||||
// No sync_state row marked ready, and no explicit servers given.
|
||||
let resp = run_cross_server_search(&store, "aurora", 50, None).unwrap();
|
||||
assert!(resp.hits.is_empty());
|
||||
assert!(resp.servers_searched.is_empty());
|
||||
}
|
||||
|
||||
// ── H3: fuzzy fallback (§5.9) ──────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn fuzzy_catches_titles_the_exact_pass_missed() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
TrackRepository::new(&store)
|
||||
.upsert_batch(&[
|
||||
// exact FTS term hit
|
||||
track("s1", "t1", "Aurora", "Anna", "Alb"),
|
||||
// FTS tokenizes to "auroras"/"borealis" → misses term "aurora";
|
||||
// LIKE %aurora% still catches it.
|
||||
track("s2", "t2", "Auroras Borealis", "Beth", "Alb"),
|
||||
])
|
||||
.unwrap();
|
||||
set_phase(&store, "s1", "ready");
|
||||
set_phase(&store, "s2", "ready");
|
||||
let resp = run_cross_server_search(&store, "aurora", 50, None).unwrap();
|
||||
assert_eq!(resp.hits.len(), 1, "exact FTS hit");
|
||||
assert_eq!(resp.hits[0].id, "t1");
|
||||
assert_eq!(resp.fuzzy.len(), 1, "fuzzy catches the FTS miss");
|
||||
assert_eq!(resp.fuzzy[0].id, "t2");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fuzzy_excludes_exact_hits() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
TrackRepository::new(&store)
|
||||
.upsert_batch(&[track("s1", "t1", "Aurora", "Anna", "Alb")])
|
||||
.unwrap();
|
||||
set_phase(&store, "s1", "ready");
|
||||
// "Aurora" is both an FTS hit and a LIKE match — must appear only once.
|
||||
let resp = run_cross_server_search(&store, "aurora", 50, None).unwrap();
|
||||
assert_eq!(resp.hits.len(), 1);
|
||||
assert!(resp.fuzzy.is_empty(), "exact hits are not repeated in fuzzy");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,690 @@
|
||||
//! Public DTOs the Tauri command surface returns. camelCase wire shape
|
||||
//! per `src-tauri/CLAUDE.md`. PR-5a only defines what the read-only
|
||||
//! commands need; PR-5b adds sync-progress / cancel-ack shapes.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::filter::{EntityKind, FilterOp};
|
||||
use crate::repos::TrackRow;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
/// `library_get_status` payload — mirrors the `sync_state` row plus a
|
||||
/// few derived counters from `track`.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct SyncStateDto {
|
||||
pub server_id: String,
|
||||
pub library_scope: String,
|
||||
#[serde(default)]
|
||||
pub sync_phase: String,
|
||||
#[serde(default)]
|
||||
pub capability_flags: u32,
|
||||
#[serde(default)]
|
||||
pub library_tier: String,
|
||||
pub last_full_sync_at: Option<i64>,
|
||||
pub last_delta_sync_at: Option<i64>,
|
||||
pub next_poll_at: Option<i64>,
|
||||
pub server_last_scan_iso: Option<String>,
|
||||
pub indexes_last_modified_ms: Option<i64>,
|
||||
pub artists_last_modified_ms: Option<i64>,
|
||||
pub local_track_count: Option<i64>,
|
||||
pub server_track_count: Option<i64>,
|
||||
pub last_error: Option<String>,
|
||||
/// `MAX(server_updated_at)` over local non-deleted tracks — the
|
||||
/// implicit "tracks watermark" the N1-delta uses.
|
||||
pub local_tracks_max_updated_ms: Option<i64>,
|
||||
/// Cheap `EXISTS` over `track` — avoids a full `COUNT(*)` on every status read.
|
||||
#[serde(default)]
|
||||
pub has_local_tracks: bool,
|
||||
/// Active/resumed initial-sync ingest strategy (`n1` / `s1` / `s2`), if any.
|
||||
#[serde(default)]
|
||||
pub ingest_strategy: Option<String>,
|
||||
/// Cursor phase during initial sync (`ingest`, `artist_pass`, …).
|
||||
#[serde(default)]
|
||||
pub ingest_phase: Option<String>,
|
||||
/// Tracks ingested so far per persisted cursor (informational during IS-3).
|
||||
#[serde(default)]
|
||||
pub cursor_ingested_count: Option<u32>,
|
||||
/// Server flagged after N1 deep-offset failure — prefers S1/S2 on next run.
|
||||
#[serde(default)]
|
||||
pub n1_bulk_unreliable: Option<bool>,
|
||||
}
|
||||
|
||||
/// E3 readiness summary attached to a single-track `library_get_track` read.
|
||||
/// Per-server, read-only, best-effort — never blocks on the network. Omitted
|
||||
/// from list/batch reads (would be one analysis probe per row).
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Default)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct TrackEnrichmentDto {
|
||||
pub waveform_ready: bool,
|
||||
pub loudness_ready: bool,
|
||||
pub lyrics_cached: bool,
|
||||
}
|
||||
|
||||
/// `library_get_track` / `library_search` row shape — flat projection
|
||||
/// over `track`'s hot columns plus the raw JSON sub-tree. Frontend
|
||||
/// re-assembles its own `LibraryTrack` shape from this.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct LibraryTrackDto {
|
||||
// Ref
|
||||
pub server_id: String,
|
||||
pub id: String,
|
||||
pub content_hash: Option<String>,
|
||||
|
||||
// Hot columns
|
||||
pub title: String,
|
||||
pub title_sort: Option<String>,
|
||||
pub artist: Option<String>,
|
||||
pub artist_id: Option<String>,
|
||||
pub album: String,
|
||||
pub album_id: Option<String>,
|
||||
pub album_artist: Option<String>,
|
||||
pub duration_sec: i64,
|
||||
pub track_number: Option<i64>,
|
||||
pub disc_number: Option<i64>,
|
||||
pub year: Option<i64>,
|
||||
pub genre: Option<String>,
|
||||
pub suffix: Option<String>,
|
||||
pub bit_rate: Option<i64>,
|
||||
pub size_bytes: Option<i64>,
|
||||
pub cover_art_id: Option<String>,
|
||||
pub starred_at: Option<i64>,
|
||||
pub user_rating: Option<i64>,
|
||||
pub play_count: Option<i64>,
|
||||
pub played_at: Option<i64>,
|
||||
pub server_path: Option<String>,
|
||||
pub library_id: Option<String>,
|
||||
pub isrc: Option<String>,
|
||||
pub mbid_recording: Option<String>,
|
||||
pub bpm: Option<i64>,
|
||||
pub replay_gain_track_db: Option<f64>,
|
||||
pub replay_gain_album_db: Option<f64>,
|
||||
|
||||
pub server_updated_at: Option<i64>,
|
||||
pub server_created_at: Option<i64>,
|
||||
pub synced_at: i64,
|
||||
|
||||
/// E3 readiness summary. Only populated by `library_get_track`; `None`
|
||||
/// (omitted on the wire) for list/batch projections via `from_row`.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub enrichment: Option<TrackEnrichmentDto>,
|
||||
|
||||
/// Original Subsonic / Navidrome song JSON the sync engine stored.
|
||||
/// Frontend parses this lazily when it needs OpenSubsonic extras
|
||||
/// (contributors, replayGain detail, …).
|
||||
pub raw_json: Value,
|
||||
}
|
||||
|
||||
impl LibraryTrackDto {
|
||||
pub fn from_row(row: &TrackRow) -> Self {
|
||||
let raw_json: Value = serde_json::from_str(&row.raw_json).unwrap_or(Value::Null);
|
||||
Self {
|
||||
server_id: row.server_id.clone(),
|
||||
id: row.id.clone(),
|
||||
content_hash: row.content_hash.clone(),
|
||||
title: row.title.clone(),
|
||||
title_sort: row.title_sort.clone(),
|
||||
artist: row.artist.clone(),
|
||||
artist_id: row.artist_id.clone(),
|
||||
album: row.album.clone(),
|
||||
album_id: row.album_id.clone(),
|
||||
album_artist: row.album_artist.clone(),
|
||||
duration_sec: row.duration_sec,
|
||||
track_number: row.track_number,
|
||||
disc_number: row.disc_number,
|
||||
year: row.year,
|
||||
genre: row.genre.clone(),
|
||||
suffix: row.suffix.clone(),
|
||||
bit_rate: row.bit_rate,
|
||||
size_bytes: row.size_bytes,
|
||||
cover_art_id: row.cover_art_id.clone(),
|
||||
starred_at: row.starred_at,
|
||||
user_rating: row.user_rating,
|
||||
play_count: row.play_count,
|
||||
played_at: row.played_at,
|
||||
server_path: row.server_path.clone(),
|
||||
library_id: row.library_id.clone(),
|
||||
isrc: row.isrc.clone(),
|
||||
mbid_recording: row.mbid_recording.clone(),
|
||||
bpm: row.bpm,
|
||||
replay_gain_track_db: row.replay_gain_track_db,
|
||||
replay_gain_album_db: row.replay_gain_album_db,
|
||||
server_updated_at: row.server_updated_at,
|
||||
server_created_at: row.server_created_at,
|
||||
synced_at: row.synced_at,
|
||||
enrichment: None,
|
||||
raw_json,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// `library_get_tracks_batch` / `library_search` envelope.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct LibraryTracksEnvelope {
|
||||
pub tracks: Vec<LibraryTrackDto>,
|
||||
pub total: u32,
|
||||
}
|
||||
|
||||
/// `library_get_artifact` payload — one row of `track_artifact`.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct TrackArtifactDto {
|
||||
pub server_id: String,
|
||||
pub track_id: String,
|
||||
pub artifact_kind: String,
|
||||
pub format: String,
|
||||
pub source_kind: String,
|
||||
pub source_id: String,
|
||||
pub language: Option<String>,
|
||||
pub content_text: Option<String>,
|
||||
pub content_bytes: i64,
|
||||
pub not_found: bool,
|
||||
pub content_hash: Option<String>,
|
||||
pub fetched_at: i64,
|
||||
pub expires_at: Option<i64>,
|
||||
}
|
||||
|
||||
/// `library_get_facts` row.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct TrackFactDto {
|
||||
pub server_id: String,
|
||||
pub track_id: String,
|
||||
pub fact_kind: String,
|
||||
pub value_real: Option<f64>,
|
||||
pub value_int: Option<i64>,
|
||||
pub value_text: Option<String>,
|
||||
pub unit: Option<String>,
|
||||
pub source_kind: String,
|
||||
pub source_id: String,
|
||||
pub confidence: f64,
|
||||
pub content_hash: Option<String>,
|
||||
pub fetched_at: i64,
|
||||
pub expires_at: Option<i64>,
|
||||
}
|
||||
|
||||
/// `library_get_offline_path` outcome — either a path string or a
|
||||
/// `missing` flag so the frontend can show a hint without polling.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct OfflinePathDto {
|
||||
pub server_id: String,
|
||||
pub track_id: String,
|
||||
pub local_path: Option<String>,
|
||||
pub missing: bool,
|
||||
}
|
||||
|
||||
/// Compact track reference used as input by `library_get_tracks_batch`.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, Hash)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct TrackRefDto {
|
||||
pub server_id: String,
|
||||
pub track_id: String,
|
||||
#[serde(default)]
|
||||
pub content_hash: Option<String>,
|
||||
}
|
||||
|
||||
/// Input to `library_put_artifact`. Same shape as `TrackArtifactDto`
|
||||
/// minus the server-supplied `server_id` / `track_id` (provided as
|
||||
/// command args) and `fetched_at` (stamped server-side from `now`).
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct ArtifactInputDto {
|
||||
pub artifact_kind: String,
|
||||
pub format: String,
|
||||
pub source_kind: String,
|
||||
pub source_id: String,
|
||||
#[serde(default)]
|
||||
pub language: Option<String>,
|
||||
#[serde(default)]
|
||||
pub content_text: Option<String>,
|
||||
#[serde(default)]
|
||||
pub content_blob: Option<Vec<u8>>,
|
||||
#[serde(default)]
|
||||
pub content_bytes: i64,
|
||||
#[serde(default)]
|
||||
pub not_found: bool,
|
||||
#[serde(default)]
|
||||
pub content_hash: Option<String>,
|
||||
#[serde(default)]
|
||||
pub expires_at: Option<i64>,
|
||||
}
|
||||
|
||||
/// Input to `library_put_fact`. Shape matches `TrackFactDto` minus the
|
||||
/// indices.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct FactInputDto {
|
||||
pub fact_kind: String,
|
||||
#[serde(default)]
|
||||
pub value_real: Option<f64>,
|
||||
#[serde(default)]
|
||||
pub value_int: Option<i64>,
|
||||
#[serde(default)]
|
||||
pub value_text: Option<String>,
|
||||
#[serde(default)]
|
||||
pub unit: Option<String>,
|
||||
pub source_kind: String,
|
||||
pub source_id: String,
|
||||
#[serde(default = "default_confidence")]
|
||||
pub confidence: f64,
|
||||
#[serde(default)]
|
||||
pub content_hash: Option<String>,
|
||||
#[serde(default)]
|
||||
pub expires_at: Option<i64>,
|
||||
}
|
||||
|
||||
fn default_confidence() -> f64 {
|
||||
1.0
|
||||
}
|
||||
|
||||
/// `library_purge_server` outcome.
|
||||
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct PurgeReportDto {
|
||||
pub tracks_deleted: u32,
|
||||
pub albums_deleted: u32,
|
||||
pub artists_deleted: u32,
|
||||
pub offline_rows_deleted: u32,
|
||||
/// Total bytes freed across the purged scopes (best-effort).
|
||||
pub bytes_freed: i64,
|
||||
}
|
||||
|
||||
/// `library_sync_start` ack.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct SyncJobDto {
|
||||
pub job_id: String,
|
||||
pub server_id: String,
|
||||
/// `"initial_sync"` or `"delta_sync"`.
|
||||
pub kind: String,
|
||||
}
|
||||
|
||||
// ──────────────────────────────────────────────────────────────────────
|
||||
// PR-5d — Advanced Search (§5.13) + cross-server search (§5.5B)
|
||||
// ──────────────────────────────────────────────────────────────────────
|
||||
|
||||
/// `library_advanced_search` row shape for an album. Flat projection over
|
||||
/// the `album` hot columns plus the raw JSON sub-tree (mirrors
|
||||
/// `LibraryTrackDto`'s lazy-parse contract).
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct LibraryAlbumDto {
|
||||
pub server_id: String,
|
||||
pub id: String,
|
||||
pub name: String,
|
||||
pub artist: Option<String>,
|
||||
pub artist_id: Option<String>,
|
||||
pub song_count: Option<i64>,
|
||||
pub duration_sec: Option<i64>,
|
||||
pub year: Option<i64>,
|
||||
pub genre: Option<String>,
|
||||
pub cover_art_id: Option<String>,
|
||||
pub starred_at: Option<i64>,
|
||||
pub synced_at: i64,
|
||||
pub raw_json: Value,
|
||||
}
|
||||
|
||||
/// `library_advanced_search` row shape for an artist.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct LibraryArtistDto {
|
||||
pub server_id: String,
|
||||
pub id: String,
|
||||
pub name: String,
|
||||
pub album_count: Option<i64>,
|
||||
pub synced_at: i64,
|
||||
pub raw_json: Value,
|
||||
}
|
||||
|
||||
/// One filter predicate. `field` is a `FilterFieldRegistry` id (§5.13.3),
|
||||
/// `op` the comparison, `value` / `valueTo` the operands. `between` uses
|
||||
/// both bounds (inclusive); scalar ops use `value` only; `isTrue` ignores
|
||||
/// the value side.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct LibraryFilterClause {
|
||||
pub field: String,
|
||||
pub op: FilterOp,
|
||||
#[serde(default)]
|
||||
pub value: Option<Value>,
|
||||
#[serde(default)]
|
||||
pub value_to: Option<Value>,
|
||||
}
|
||||
|
||||
/// One sort key. `field` is a registry id; `dir` is `asc` / `desc`.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct LibrarySortClause {
|
||||
pub field: String,
|
||||
pub dir: SortDir,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[serde(rename_all = "lowercase")]
|
||||
pub enum SortDir {
|
||||
Asc,
|
||||
Desc,
|
||||
}
|
||||
|
||||
/// `library_advanced_search` request (§5.13.2). `query` is shorthand for an
|
||||
/// `fts` clause on the text fields; `entityTypes` controls which of the
|
||||
/// three queries run; `filters` are combined with AND.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct LibraryAdvancedSearchRequest {
|
||||
pub server_id: String,
|
||||
#[serde(default)]
|
||||
pub library_scope: Option<String>,
|
||||
#[serde(default)]
|
||||
pub query: Option<String>,
|
||||
pub entity_types: Vec<EntityKind>,
|
||||
#[serde(default)]
|
||||
pub filters: Vec<LibraryFilterClause>,
|
||||
#[serde(default)]
|
||||
pub starred_only: Option<bool>,
|
||||
#[serde(default)]
|
||||
pub sort: Vec<LibrarySortClause>,
|
||||
pub limit: u32,
|
||||
#[serde(default)]
|
||||
pub offset: u32,
|
||||
/// When true, skip per-entity COUNT queries (Live Search / small pages).
|
||||
#[serde(default)]
|
||||
pub skip_totals: bool,
|
||||
}
|
||||
|
||||
/// Per-entity result counts (full match count, not page size).
|
||||
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct LibrarySearchTotals {
|
||||
pub artists: u32,
|
||||
pub albums: u32,
|
||||
pub tracks: u32,
|
||||
}
|
||||
|
||||
/// `library_advanced_search` response (§5.13.2).
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct LibraryAdvancedSearchResponse {
|
||||
pub artists: Vec<LibraryArtistDto>,
|
||||
pub albums: Vec<LibraryAlbumDto>,
|
||||
pub tracks: Vec<LibraryTrackDto>,
|
||||
pub totals: LibrarySearchTotals,
|
||||
/// Distinct registry field ids that were actually applied — UI chips /
|
||||
/// debug. Includes `starred` when `starredOnly` is set.
|
||||
pub applied_filters: Vec<String>,
|
||||
/// Always `"local"` from this command (it queries the local index); the
|
||||
/// frontend's fallback decides local vs network (§5.13.6).
|
||||
pub source: String,
|
||||
}
|
||||
|
||||
/// `library_live_search` response — lean FTS dropdown (§5.9 / P24).
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct LibraryLiveSearchRequest {
|
||||
pub server_id: String,
|
||||
pub query: String,
|
||||
#[serde(default)]
|
||||
pub library_scope: Option<String>,
|
||||
#[serde(default)]
|
||||
pub artist_limit: Option<u32>,
|
||||
#[serde(default)]
|
||||
pub album_limit: Option<u32>,
|
||||
#[serde(default)]
|
||||
pub song_limit: Option<u32>,
|
||||
#[serde(default)]
|
||||
pub request_epoch: Option<u64>,
|
||||
}
|
||||
|
||||
/// `library_live_search` response — lean FTS dropdown (§5.9 / P24).
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct LibraryLiveSearchResponse {
|
||||
pub artists: Vec<LibraryArtistDto>,
|
||||
pub albums: Vec<LibraryAlbumDto>,
|
||||
pub tracks: Vec<LibraryTrackDto>,
|
||||
pub source: String,
|
||||
}
|
||||
|
||||
/// `library_search_cross_server` response (§5.5B / §5.9).
|
||||
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct LibraryCrossServerSearchResponse {
|
||||
/// Primary FTS-union hits, deduped by canonical id where a link exists.
|
||||
pub hits: Vec<LibraryTrackDto>,
|
||||
/// Fuzzy fallback (§5.9 / H3): per-server `title LIKE` matches that the
|
||||
/// exact FTS pass missed (diacritics, partial words). Excludes anything
|
||||
/// already in `hits` and dedupes by canonical id against them.
|
||||
pub fuzzy: Vec<LibraryTrackDto>,
|
||||
/// The server ids that were actually searched (resolved from the
|
||||
/// request's `servers` or all `ready` servers).
|
||||
pub servers_searched: Vec<String>,
|
||||
}
|
||||
|
||||
/// Read `MAX(server_updated_at)` for non-deleted tracks on this server
|
||||
/// — used by `SyncStateDto` so callers can show "tracks watermark" in
|
||||
/// Settings without a separate column.
|
||||
pub fn local_tracks_max_updated_ms(
|
||||
store: &LibraryStore,
|
||||
server_id: &str,
|
||||
) -> Result<Option<i64>, String> {
|
||||
store
|
||||
.with_read_conn(|c| {
|
||||
c.query_row(
|
||||
"SELECT MAX(server_updated_at) FROM track \
|
||||
WHERE server_id = ?1 AND deleted = 0",
|
||||
rusqlite::params![server_id],
|
||||
|row| row.get::<_, Option<i64>>(0),
|
||||
)
|
||||
})
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
/// Cheap `EXISTS` — true when at least one non-deleted track is indexed.
|
||||
pub fn track_index_nonempty(store: &LibraryStore, server_id: &str) -> Result<bool, String> {
|
||||
store
|
||||
.with_read_conn(|c| {
|
||||
c.query_row(
|
||||
"SELECT EXISTS(SELECT 1 FROM track WHERE server_id = ?1 AND deleted = 0 LIMIT 1)",
|
||||
rusqlite::params![server_id],
|
||||
|row| row.get(0),
|
||||
)
|
||||
})
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
/// Live non-deleted track count for a server (used when the sync_state
|
||||
/// snapshot is missing or stale).
|
||||
pub fn count_local_tracks(store: &LibraryStore, server_id: &str) -> Result<i64, String> {
|
||||
store
|
||||
.with_read_conn(|c| {
|
||||
c.query_row(
|
||||
"SELECT COUNT(*) FROM track WHERE server_id = ?1 AND deleted = 0",
|
||||
rusqlite::params![server_id],
|
||||
|row| row.get(0),
|
||||
)
|
||||
})
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::repos::TrackRepository;
|
||||
|
||||
fn sample_row() -> TrackRow {
|
||||
TrackRow {
|
||||
server_id: "s1".into(),
|
||||
id: "tr_1".into(),
|
||||
title: "Hello".into(),
|
||||
title_sort: None,
|
||||
artist: Some("World".into()),
|
||||
artist_id: Some("ar_1".into()),
|
||||
album: "An Album".into(),
|
||||
album_id: Some("al_1".into()),
|
||||
album_artist: Some("World".into()),
|
||||
duration_sec: 240,
|
||||
track_number: Some(3),
|
||||
disc_number: Some(1),
|
||||
year: Some(2024),
|
||||
genre: Some("Ambient".into()),
|
||||
suffix: Some("flac".into()),
|
||||
bit_rate: Some(1000),
|
||||
size_bytes: Some(32_000_000),
|
||||
cover_art_id: Some("cv_1".into()),
|
||||
starred_at: None,
|
||||
user_rating: None,
|
||||
play_count: Some(0),
|
||||
played_at: None,
|
||||
server_path: Some("/path/x.flac".into()),
|
||||
library_id: Some("1".into()),
|
||||
isrc: Some("USRC17607839".into()),
|
||||
mbid_recording: Some("mb-1".into()),
|
||||
bpm: Some(120),
|
||||
replay_gain_track_db: Some(-1.2),
|
||||
replay_gain_album_db: Some(-0.8),
|
||||
content_hash: Some("deadbeef".into()),
|
||||
server_updated_at: Some(1_700_000_000),
|
||||
server_created_at: Some(1_699_000_000),
|
||||
deleted: false,
|
||||
synced_at: 1_700_000_500,
|
||||
raw_json: r#"{"replayGain":{"trackGain":-1.2}}"#.into(),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn library_track_dto_serializes_field_names_camel_case() {
|
||||
let dto = LibraryTrackDto::from_row(&sample_row());
|
||||
let json = serde_json::to_value(&dto).unwrap();
|
||||
// Spot-check critical wire keys — IPC contract.
|
||||
for key in [
|
||||
"serverId",
|
||||
"contentHash",
|
||||
"albumArtist",
|
||||
"durationSec",
|
||||
"trackNumber",
|
||||
"discNumber",
|
||||
"coverArtId",
|
||||
"userRating",
|
||||
"playCount",
|
||||
"playedAt",
|
||||
"serverPath",
|
||||
"libraryId",
|
||||
"mbidRecording",
|
||||
"replayGainTrackDb",
|
||||
"replayGainAlbumDb",
|
||||
"serverUpdatedAt",
|
||||
"syncedAt",
|
||||
"rawJson",
|
||||
] {
|
||||
assert!(
|
||||
json.get(key).is_some(),
|
||||
"expected camelCase key `{key}` in serialized DTO, got {json}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn library_track_dto_parses_raw_json_into_value() {
|
||||
let dto = LibraryTrackDto::from_row(&sample_row());
|
||||
let rg = dto
|
||||
.raw_json
|
||||
.get("replayGain")
|
||||
.and_then(|v| v.get("trackGain"))
|
||||
.and_then(|v| v.as_f64())
|
||||
.unwrap();
|
||||
assert!((rg - -1.2).abs() < 0.001);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn library_track_dto_falls_back_to_null_on_invalid_raw_json() {
|
||||
let mut row = sample_row();
|
||||
row.raw_json = "{not valid json}".into();
|
||||
let dto = LibraryTrackDto::from_row(&row);
|
||||
assert!(dto.raw_json.is_null(), "invalid JSON must surface as Value::Null");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn local_tracks_max_updated_returns_max_over_non_deleted_rows() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
let mut r1 = sample_row();
|
||||
r1.server_updated_at = Some(1_000);
|
||||
let mut r2 = sample_row();
|
||||
r2.id = "tr_2".into();
|
||||
r2.server_updated_at = Some(3_000);
|
||||
let mut r3 = sample_row();
|
||||
r3.id = "tr_3".into();
|
||||
r3.server_updated_at = Some(5_000);
|
||||
r3.deleted = true;
|
||||
repo.upsert_batch(&[r1, r2, r3]).unwrap();
|
||||
|
||||
assert_eq!(
|
||||
local_tracks_max_updated_ms(&store, "s1").unwrap(),
|
||||
Some(3_000),
|
||||
"deleted rows must be excluded"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn count_local_tracks_matches_non_deleted_rows() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
let mut deleted = sample_row();
|
||||
deleted.id = "tr_del".into();
|
||||
deleted.deleted = true;
|
||||
repo.upsert_batch(&[sample_row(), deleted]).unwrap();
|
||||
assert_eq!(count_local_tracks(&store, "s1").unwrap(), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn track_ref_dto_roundtrips_through_json() {
|
||||
let r = TrackRefDto {
|
||||
server_id: "s1".into(),
|
||||
track_id: "tr_1".into(),
|
||||
content_hash: Some("h".into()),
|
||||
};
|
||||
let json = serde_json::to_value(&r).unwrap();
|
||||
assert_eq!(json.get("serverId").and_then(|v| v.as_str()), Some("s1"));
|
||||
let back: TrackRefDto = serde_json::from_value(json).unwrap();
|
||||
assert_eq!(back, r);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sync_state_dto_omits_null_optionals_cleanly() {
|
||||
let dto = SyncStateDto {
|
||||
server_id: "s1".into(),
|
||||
library_scope: "".into(),
|
||||
sync_phase: "idle".into(),
|
||||
capability_flags: 0,
|
||||
library_tier: "unknown".into(),
|
||||
last_full_sync_at: None,
|
||||
last_delta_sync_at: None,
|
||||
next_poll_at: None,
|
||||
server_last_scan_iso: None,
|
||||
indexes_last_modified_ms: None,
|
||||
artists_last_modified_ms: None,
|
||||
local_track_count: None,
|
||||
server_track_count: None,
|
||||
last_error: None,
|
||||
local_tracks_max_updated_ms: None,
|
||||
has_local_tracks: false,
|
||||
ingest_strategy: None,
|
||||
ingest_phase: None,
|
||||
cursor_ingested_count: None,
|
||||
n1_bulk_unreliable: None,
|
||||
};
|
||||
let json = serde_json::to_value(dto).unwrap();
|
||||
assert_eq!(
|
||||
json.get("syncPhase").and_then(|v| v.as_str()),
|
||||
Some("idle")
|
||||
);
|
||||
// `null` survives as JSON null, not omitted — explicit shape
|
||||
// for the WebView so it can distinguish "missing" from
|
||||
// "unset".
|
||||
assert!(json.get("lastFullSyncAt").unwrap().is_null());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,415 @@
|
||||
//! `FilterFieldRegistry` — Rust source of truth for Advanced Search filter
|
||||
//! fields (spec §5.13.3 / P38). The full SQL builder (`AdvancedSearchQuery`,
|
||||
//! §5.13.5) and the Tauri command surface (§5.13.6) come later; PR-1a
|
||||
//! ships the registry shape + the v1 fields + the entity-routing rule.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum EntityKind {
|
||||
Track,
|
||||
Album,
|
||||
Artist,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum FilterOp {
|
||||
/// FTS5 MATCH (`track_fts`). Only valid on the `text` field in v1.
|
||||
Fts,
|
||||
Eq,
|
||||
/// Membership test against a list of values. (Not yet in v1; reserved.)
|
||||
In,
|
||||
Gte,
|
||||
Lte,
|
||||
Between,
|
||||
/// Boolean field — value side is ignored, presence = `IS NOT NULL`.
|
||||
IsTrue,
|
||||
}
|
||||
|
||||
impl FilterOp {
|
||||
/// Wire spelling — matches the TypeScript `FilterOperator` union and the
|
||||
/// `FilterOp::from_wire` parser. Used in error messages too.
|
||||
pub fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
FilterOp::Fts => "fts",
|
||||
FilterOp::Eq => "eq",
|
||||
FilterOp::In => "in",
|
||||
FilterOp::Gte => "gte",
|
||||
FilterOp::Lte => "lte",
|
||||
FilterOp::Between => "between",
|
||||
FilterOp::IsTrue => "is_true",
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse the wire operator. Spec §5.13.2 lists a few operators
|
||||
/// (`neq` / `contains` / `is_false`) the v1 builder doesn't implement
|
||||
/// yet — those return `None` so the caller can raise a clear error
|
||||
/// rather than silently dropping the clause.
|
||||
pub fn from_wire(wire: &str) -> Option<FilterOp> {
|
||||
match wire {
|
||||
"fts" => Some(FilterOp::Fts),
|
||||
"eq" => Some(FilterOp::Eq),
|
||||
"in" => Some(FilterOp::In),
|
||||
"gte" => Some(FilterOp::Gte),
|
||||
"lte" => Some(FilterOp::Lte),
|
||||
"between" => Some(FilterOp::Between),
|
||||
"is_true" => Some(FilterOp::IsTrue),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
|
||||
pub enum FilterStatus {
|
||||
/// v1: built into the SQL builder and exercised by parity tests.
|
||||
V1,
|
||||
/// Schema + SQL builder present (the request and registry accept it),
|
||||
/// but hidden from the v1 UI — see §5.13.4 (`bpm` dual-storage).
|
||||
SchemaV1UiLater,
|
||||
/// Reserved column / planned-but-not-built.
|
||||
Planned,
|
||||
/// Out of scope for v1 entirely.
|
||||
Future,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct FilterField {
|
||||
pub id: &'static str,
|
||||
pub entities: &'static [EntityKind],
|
||||
pub ops: &'static [FilterOp],
|
||||
pub status: FilterStatus,
|
||||
}
|
||||
|
||||
/// Static v1 registry. Adding a row here is the only thing required to expose
|
||||
/// a new filter field (plus, when the storage isn't yet a hot column / index,
|
||||
/// a separate `00X_*.sql` migration — see §5.7). No new invoke is needed.
|
||||
pub const FILTER_FIELD_REGISTRY: &[FilterField] = &[
|
||||
FilterField {
|
||||
id: "text",
|
||||
entities: &[EntityKind::Track, EntityKind::Album, EntityKind::Artist],
|
||||
ops: &[FilterOp::Fts],
|
||||
status: FilterStatus::V1,
|
||||
},
|
||||
FilterField {
|
||||
id: "genre",
|
||||
entities: &[EntityKind::Track, EntityKind::Album],
|
||||
ops: &[FilterOp::Eq],
|
||||
status: FilterStatus::V1,
|
||||
},
|
||||
FilterField {
|
||||
id: "year",
|
||||
entities: &[EntityKind::Track, EntityKind::Album],
|
||||
ops: &[FilterOp::Gte, FilterOp::Lte, FilterOp::Between],
|
||||
status: FilterStatus::V1,
|
||||
},
|
||||
FilterField {
|
||||
id: "starred",
|
||||
entities: &[EntityKind::Track, EntityKind::Album, EntityKind::Artist],
|
||||
ops: &[FilterOp::IsTrue],
|
||||
status: FilterStatus::V1,
|
||||
},
|
||||
FilterField {
|
||||
id: "user_rating",
|
||||
entities: &[EntityKind::Track],
|
||||
ops: &[FilterOp::Gte, FilterOp::Eq],
|
||||
status: FilterStatus::Planned,
|
||||
},
|
||||
FilterField {
|
||||
id: "suffix",
|
||||
entities: &[EntityKind::Track],
|
||||
ops: &[FilterOp::Eq, FilterOp::In],
|
||||
status: FilterStatus::Planned,
|
||||
},
|
||||
FilterField {
|
||||
id: "bit_rate",
|
||||
entities: &[EntityKind::Track],
|
||||
ops: &[FilterOp::Gte, FilterOp::Lte, FilterOp::Between],
|
||||
status: FilterStatus::Planned,
|
||||
},
|
||||
FilterField {
|
||||
id: "bpm",
|
||||
entities: &[EntityKind::Track],
|
||||
ops: &[FilterOp::Gte, FilterOp::Lte, FilterOp::Between],
|
||||
status: FilterStatus::SchemaV1UiLater,
|
||||
},
|
||||
];
|
||||
|
||||
pub fn lookup(id: &str) -> Option<&'static FilterField> {
|
||||
FILTER_FIELD_REGISTRY.iter().find(|f| f.id == id)
|
||||
}
|
||||
|
||||
/// `true` when this filter field is applicable for a request that targets
|
||||
/// the given entity. Per §5.13.3 the routing rule is a *skip*, not an error:
|
||||
/// if the request asks for `entityTypes = [album, artist]` and a clause names
|
||||
/// a track-only field, the clause is silently dropped.
|
||||
pub fn applies_to(field: &FilterField, entity: EntityKind) -> bool {
|
||||
field.entities.contains(&entity)
|
||||
}
|
||||
|
||||
// ── SQL fragment resolution (§5.13.5) ─────────────────────────────────
|
||||
|
||||
/// A resolved WHERE-clause snippet plus the values it binds. The builder
|
||||
/// appends fragments in order and binds their params left-to-right against
|
||||
/// anonymous `?` placeholders (`params_from_iter`), so a fragment must keep
|
||||
/// its `sql` placeholders and `params` in the same order.
|
||||
///
|
||||
/// `sql` only ever contains builder-supplied column expressions and literal
|
||||
/// operators — never user input (spec §5.13.5: parameterised only).
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct SqlFragment {
|
||||
pub sql: String,
|
||||
pub params: Vec<rusqlite::types::Value>,
|
||||
}
|
||||
|
||||
/// Why a `LibraryFilterClause` could not be turned into SQL. Surfaced to the
|
||||
/// command as a human-readable string; `UnknownField` carries the known-field
|
||||
/// list for dev builds (§5.13.5).
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum FilterError {
|
||||
/// `field` is not in the registry at all (typo / wrong client).
|
||||
UnknownField(String),
|
||||
/// `field` is registered but has no v1 SQL builder yet (`user_rating`,
|
||||
/// `suffix`, `bit_rate`, …). Distinct from `UnknownField` so the caller
|
||||
/// can tell a typo from a planned-but-unbuilt field.
|
||||
NotQueryable(String),
|
||||
/// `op` is not declared for `field` in the registry.
|
||||
UnsupportedOp { field: String, op: &'static str },
|
||||
/// The value side was missing or the wrong type for the operator.
|
||||
BadValue { field: String, detail: String },
|
||||
}
|
||||
|
||||
impl std::fmt::Display for FilterError {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
FilterError::UnknownField(field) => {
|
||||
let known = FILTER_FIELD_REGISTRY
|
||||
.iter()
|
||||
.map(|x| x.id)
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ");
|
||||
write!(f, "unknown filter field `{field}` (known: {known})")
|
||||
}
|
||||
FilterError::NotQueryable(field) => {
|
||||
write!(f, "filter field `{field}` is registered but not queryable in v1")
|
||||
}
|
||||
FilterError::UnsupportedOp { field, op } => {
|
||||
write!(f, "operator `{op}` is not supported for filter field `{field}`")
|
||||
}
|
||||
FilterError::BadValue { field, detail } => {
|
||||
write!(f, "bad value for filter field `{field}`: {detail}")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Validate that `field` exists, applies to `entity`, and declares `op`.
|
||||
///
|
||||
/// Returns `Ok(true)` when the clause is applicable, `Ok(false)` when the
|
||||
/// field is known but doesn't route to this entity (§5.13.3: skip, don't
|
||||
/// error), and `Err` when the field is unknown or the op is undeclared.
|
||||
pub fn validate_for_entity(
|
||||
field_id: &str,
|
||||
op: FilterOp,
|
||||
entity: EntityKind,
|
||||
) -> Result<bool, FilterError> {
|
||||
let field = lookup(field_id).ok_or_else(|| FilterError::UnknownField(field_id.to_string()))?;
|
||||
if !field.ops.contains(&op) {
|
||||
return Err(FilterError::UnsupportedOp {
|
||||
field: field_id.to_string(),
|
||||
op: op.as_str(),
|
||||
});
|
||||
}
|
||||
Ok(applies_to(field, entity))
|
||||
}
|
||||
|
||||
/// Build a comparison fragment for a builder-supplied column expression.
|
||||
/// `col` is trusted SQL (never user input); only the bound values come from
|
||||
/// the request. `eq`/`gte`/`lte`/`between` parameterise their operands;
|
||||
/// `is_true` ignores the value and tests `IS NOT NULL`. `between` is
|
||||
/// inclusive on both ends (matches the year UI — §5.13.5).
|
||||
pub fn compare_fragment(
|
||||
field: &str,
|
||||
col: &str,
|
||||
op: FilterOp,
|
||||
value: Option<rusqlite::types::Value>,
|
||||
value_to: Option<rusqlite::types::Value>,
|
||||
) -> Result<SqlFragment, FilterError> {
|
||||
let need_value = |v: Option<rusqlite::types::Value>| {
|
||||
v.ok_or_else(|| FilterError::BadValue {
|
||||
field: field.to_string(),
|
||||
detail: format!("operator `{}` requires a value", op.as_str()),
|
||||
})
|
||||
};
|
||||
match op {
|
||||
FilterOp::Eq => Ok(SqlFragment {
|
||||
sql: format!("{col} = ?"),
|
||||
params: vec![need_value(value)?],
|
||||
}),
|
||||
FilterOp::Gte => Ok(SqlFragment {
|
||||
sql: format!("{col} >= ?"),
|
||||
params: vec![need_value(value)?],
|
||||
}),
|
||||
FilterOp::Lte => Ok(SqlFragment {
|
||||
sql: format!("{col} <= ?"),
|
||||
params: vec![need_value(value)?],
|
||||
}),
|
||||
FilterOp::Between => {
|
||||
let lo = need_value(value)?;
|
||||
let hi = value_to.ok_or_else(|| FilterError::BadValue {
|
||||
field: field.to_string(),
|
||||
detail: "operator `between` requires `valueTo`".to_string(),
|
||||
})?;
|
||||
Ok(SqlFragment {
|
||||
sql: format!("{col} BETWEEN ? AND ?"),
|
||||
params: vec![lo, hi],
|
||||
})
|
||||
}
|
||||
FilterOp::IsTrue => Ok(SqlFragment {
|
||||
sql: format!("{col} IS NOT NULL"),
|
||||
params: vec![],
|
||||
}),
|
||||
FilterOp::Fts | FilterOp::In => Err(FilterError::UnsupportedOp {
|
||||
field: field.to_string(),
|
||||
op: op.as_str(),
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn registry_contains_all_v1_fields() {
|
||||
for id in ["text", "genre", "year", "starred"] {
|
||||
let f = lookup(id).unwrap_or_else(|| panic!("missing v1 field `{id}`"));
|
||||
assert_eq!(f.status, FilterStatus::V1, "`{id}` must be V1");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bpm_is_schema_v1_but_ui_later() {
|
||||
// §5.13.3: bpm has a hot column + index from day one, but is hidden
|
||||
// from the v1 UI until §5.13.4 dual-storage resolution lands.
|
||||
assert_eq!(lookup("bpm").unwrap().status, FilterStatus::SchemaV1UiLater);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn text_routes_to_all_three_entities() {
|
||||
let f = lookup("text").unwrap();
|
||||
assert!(applies_to(f, EntityKind::Track));
|
||||
assert!(applies_to(f, EntityKind::Album));
|
||||
assert!(applies_to(f, EntityKind::Artist));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn track_only_field_is_skipped_for_album_entity() {
|
||||
let f = lookup("user_rating").unwrap();
|
||||
assert!(applies_to(f, EntityKind::Track));
|
||||
assert!(!applies_to(f, EntityKind::Album));
|
||||
assert!(!applies_to(f, EntityKind::Artist));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unknown_field_lookup_returns_none() {
|
||||
assert!(lookup("nope").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn registry_has_no_duplicate_ids() {
|
||||
let mut ids: Vec<&str> = FILTER_FIELD_REGISTRY.iter().map(|f| f.id).collect();
|
||||
ids.sort();
|
||||
let len_before = ids.len();
|
||||
ids.dedup();
|
||||
assert_eq!(ids.len(), len_before, "duplicate filter field id detected");
|
||||
}
|
||||
|
||||
// ── SQL fragment resolution ───────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn op_wire_roundtrips() {
|
||||
for op in [
|
||||
FilterOp::Fts,
|
||||
FilterOp::Eq,
|
||||
FilterOp::In,
|
||||
FilterOp::Gte,
|
||||
FilterOp::Lte,
|
||||
FilterOp::Between,
|
||||
FilterOp::IsTrue,
|
||||
] {
|
||||
assert_eq!(FilterOp::from_wire(op.as_str()), Some(op));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn op_from_wire_rejects_unbuilt_operators() {
|
||||
// Spec §5.13.2 lists these but the v1 builder doesn't implement them.
|
||||
for wire in ["neq", "contains", "is_false", "nope"] {
|
||||
assert!(FilterOp::from_wire(wire).is_none(), "`{wire}` must not parse");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validate_unknown_field_errors() {
|
||||
let err = validate_for_entity("nope", FilterOp::Eq, EntityKind::Track).unwrap_err();
|
||||
assert!(matches!(err, FilterError::UnknownField(f) if f == "nope"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validate_undeclared_op_errors() {
|
||||
// `genre` only declares `eq` in v1.
|
||||
let err = validate_for_entity("genre", FilterOp::Gte, EntityKind::Track).unwrap_err();
|
||||
assert!(matches!(err, FilterError::UnsupportedOp { .. }));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validate_known_field_off_entity_is_skip_not_error() {
|
||||
// `bpm` is track-only — for an album query it routes to "skip".
|
||||
assert_eq!(
|
||||
validate_for_entity("bpm", FilterOp::Between, EntityKind::Album),
|
||||
Ok(false)
|
||||
);
|
||||
assert_eq!(
|
||||
validate_for_entity("bpm", FilterOp::Between, EntityKind::Track),
|
||||
Ok(true)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn compare_fragment_between_is_inclusive_both_ends() {
|
||||
use rusqlite::types::Value;
|
||||
let frag = compare_fragment(
|
||||
"year",
|
||||
"t.year",
|
||||
FilterOp::Between,
|
||||
Some(Value::Integer(2000)),
|
||||
Some(Value::Integer(2010)),
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(frag.sql, "t.year BETWEEN ? AND ?");
|
||||
assert_eq!(frag.params, vec![Value::Integer(2000), Value::Integer(2010)]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn compare_fragment_between_without_value_to_errors() {
|
||||
use rusqlite::types::Value;
|
||||
let err = compare_fragment(
|
||||
"year",
|
||||
"t.year",
|
||||
FilterOp::Between,
|
||||
Some(Value::Integer(2000)),
|
||||
None,
|
||||
)
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, FilterError::BadValue { .. }));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn compare_fragment_is_true_ignores_value() {
|
||||
let frag = compare_fragment("starred", "t.starred_at", FilterOp::IsTrue, None, None).unwrap();
|
||||
assert_eq!(frag.sql, "t.starred_at IS NOT NULL");
|
||||
assert!(frag.params.is_empty());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
//! `psysonic-library` — unified track store and (future) sync engine.
|
||||
//!
|
||||
//! v1 scope (this crate, across PR-1..PR-7):
|
||||
//! - `store` — SQLite connection, WAL config, versioned migration runner
|
||||
//! - `repos` — typed repositories over the v1 schema (track, album, artist, …)
|
||||
//! - `search` — FTS5 query helpers
|
||||
//! - `filter` — `FilterFieldRegistry` (Rust source of truth for Advanced Search)
|
||||
//! - `sync` — capability probe + orchestrator (PR-3*)
|
||||
|
||||
pub(crate) mod bulk_ingest;
|
||||
pub mod advanced_search;
|
||||
pub mod canonical;
|
||||
pub mod commands;
|
||||
pub mod cross_server;
|
||||
pub mod dto;
|
||||
pub mod filter;
|
||||
pub mod live_search;
|
||||
pub mod payload;
|
||||
pub mod repos;
|
||||
pub mod runtime;
|
||||
pub mod search;
|
||||
pub mod store;
|
||||
pub mod sync;
|
||||
pub(crate) mod track_fts;
|
||||
|
||||
pub use payload::LibrarySyncProgressPayload;
|
||||
pub use runtime::LibraryRuntime;
|
||||
|
||||
pub use store::{LibraryStore, LIBRARY_DB_SCHEMA_VERSION};
|
||||
|
||||
// Re-export logging facade so submodules can write `crate::app_eprintln!()`.
|
||||
pub use psysonic_core::{app_deprintln, app_eprintln, logging};
|
||||
@@ -0,0 +1,660 @@
|
||||
//! Live Search dropdown (spec §5.9 / P24) — column-scoped FTS with LIMIT inside
|
||||
//! the FTS subquery (bm25 on ≤N rowids), then a cheap join to `track`.
|
||||
//! Avoids the SQLite pitfall where `JOIN track … ORDER BY bm25` on an OR
|
||||
//! MATCH scans/ranks the whole hit set on 100k+ libraries (10–20s queries).
|
||||
|
||||
use std::collections::{HashMap, HashSet};
|
||||
|
||||
use rusqlite::params;
|
||||
|
||||
use crate::dto::{LibraryAlbumDto, LibraryArtistDto, LibraryLiveSearchResponse, LibraryTrackDto};
|
||||
use crate::search::{fts_column_prefix_query, fts_query_meets_min_len, library_scope_equals_sql};
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
const SONG_FTS_COLUMNS: [&str; 4] = ["title", "artist", "album", "album_artist"];
|
||||
const ALBUM_FTS_COLUMNS: [&str; 2] = ["album", "album_artist"];
|
||||
|
||||
struct LiveHit {
|
||||
track: LibraryTrackDto,
|
||||
}
|
||||
|
||||
/// `library_live_search` — read connection, scoped FTS rowid picks + join.
|
||||
pub fn run_live_search(
|
||||
store: &LibraryStore,
|
||||
server_id: &str,
|
||||
query: &str,
|
||||
library_scope: Option<&str>,
|
||||
artist_limit: u32,
|
||||
album_limit: u32,
|
||||
song_limit: u32,
|
||||
) -> Result<LibraryLiveSearchResponse, String> {
|
||||
if !fts_query_meets_min_len(query) {
|
||||
return Ok(LibraryLiveSearchResponse {
|
||||
artists: Vec::new(),
|
||||
albums: Vec::new(),
|
||||
tracks: Vec::new(),
|
||||
source: "local".to_string(),
|
||||
});
|
||||
}
|
||||
|
||||
store.with_read_conn(|conn| {
|
||||
let scope = trimmed_scope(library_scope);
|
||||
let songs = query_songs(conn, query, server_id, scope.as_deref(), song_limit)?;
|
||||
let artists = query_artists(conn, query, server_id, scope.as_deref(), artist_limit)?;
|
||||
let albums = query_albums(conn, query, server_id, scope.as_deref(), album_limit)?;
|
||||
Ok(LibraryLiveSearchResponse {
|
||||
artists,
|
||||
albums,
|
||||
tracks: songs,
|
||||
source: "local".to_string(),
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
/// Top FTS rowids for one or more column-scoped MATCH strings (deduped, ordered).
|
||||
fn collect_fts_rowids(
|
||||
conn: &rusqlite::Connection,
|
||||
match_queries: &[String],
|
||||
per_query_limit: i64,
|
||||
total_limit: usize,
|
||||
) -> rusqlite::Result<Vec<i64>> {
|
||||
let sql =
|
||||
"SELECT rowid FROM track_fts WHERE track_fts MATCH ?1 ORDER BY bm25(track_fts) LIMIT ?2";
|
||||
let mut stmt = conn.prepare(sql)?;
|
||||
let mut seen = HashSet::new();
|
||||
let mut rowids = Vec::new();
|
||||
for mq in match_queries {
|
||||
let rows = stmt.query_map(params![mq, per_query_limit], |r| r.get(0))?;
|
||||
for rowid in rows {
|
||||
let rowid = rowid?;
|
||||
if seen.insert(rowid) {
|
||||
rowids.push(rowid);
|
||||
if rowids.len() >= total_limit {
|
||||
return Ok(rowids);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(rowids)
|
||||
}
|
||||
|
||||
fn column_matches(query: &str, columns: &[&str]) -> Result<Vec<String>, String> {
|
||||
columns
|
||||
.iter()
|
||||
.map(|col| {
|
||||
fts_column_prefix_query(col, query).ok_or_else(|| "empty query".to_string())
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn trimmed_scope(scope: Option<&str>) -> Option<String> {
|
||||
scope
|
||||
.map(str::trim)
|
||||
.filter(|s| !s.is_empty())
|
||||
.map(str::to_string)
|
||||
}
|
||||
|
||||
fn append_library_scope(
|
||||
sql: &mut String,
|
||||
params: &mut Vec<rusqlite::types::Value>,
|
||||
library_scope: Option<&str>,
|
||||
) {
|
||||
if let Some(scope) = library_scope.filter(|s| !s.trim().is_empty()) {
|
||||
sql.push_str(" AND ");
|
||||
sql.push_str(&library_scope_equals_sql("t"));
|
||||
params.push(rusqlite::types::Value::Text(scope.to_string()));
|
||||
}
|
||||
}
|
||||
|
||||
fn query_songs(
|
||||
conn: &rusqlite::Connection,
|
||||
query: &str,
|
||||
server_id: &str,
|
||||
library_scope: Option<&str>,
|
||||
limit: u32,
|
||||
) -> rusqlite::Result<Vec<LibraryTrackDto>> {
|
||||
let matches = column_matches(query, &SONG_FTS_COLUMNS).map_err(|e| {
|
||||
rusqlite::Error::ToSqlConversionFailure(Box::new(std::io::Error::new(
|
||||
std::io::ErrorKind::InvalidInput,
|
||||
e,
|
||||
)))
|
||||
})?;
|
||||
let per_col = i64::from(limit.max(4));
|
||||
let rowids = collect_fts_rowids(conn, &matches, per_col, limit as usize)?;
|
||||
if rowids.is_empty() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
fetch_tracks_by_rowids(conn, &rowids, server_id, library_scope)
|
||||
}
|
||||
|
||||
fn query_artists(
|
||||
conn: &rusqlite::Connection,
|
||||
query: &str,
|
||||
server_id: &str,
|
||||
library_scope: Option<&str>,
|
||||
limit: u32,
|
||||
) -> rusqlite::Result<Vec<LibraryArtistDto>> {
|
||||
let Some(artist_fts) = fts_column_prefix_query("artist", query) else {
|
||||
return Ok(Vec::new());
|
||||
};
|
||||
let fetch = limit.saturating_mul(3).clamp(limit, 24);
|
||||
let rowids = collect_fts_rowids(conn, &[artist_fts], i64::from(fetch), fetch as usize)?;
|
||||
if rowids.is_empty() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
let placeholders = rowid_placeholders(rowids.len());
|
||||
let sql = format!(
|
||||
"SELECT t.server_id, t.artist_id, t.artist, t.synced_at, t.rowid \
|
||||
FROM track t \
|
||||
WHERE t.rowid IN ({placeholders}) \
|
||||
AND t.server_id = ? \
|
||||
AND t.deleted = 0 \
|
||||
AND t.artist_id IS NOT NULL AND t.artist_id != ''"
|
||||
);
|
||||
let mut params: Vec<rusqlite::types::Value> = rowids
|
||||
.iter()
|
||||
.copied()
|
||||
.map(rusqlite::types::Value::Integer)
|
||||
.collect();
|
||||
params.push(rusqlite::types::Value::Text(server_id.to_string()));
|
||||
let mut sql = sql;
|
||||
append_library_scope(&mut sql, &mut params, library_scope);
|
||||
let rank: HashMap<i64, usize> = rowids
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(i, &rid)| (rid, i))
|
||||
.collect();
|
||||
let mut ranked: Vec<(usize, LibraryArtistDto)> = Vec::new();
|
||||
let mut stmt = conn.prepare(&sql)?;
|
||||
for row in stmt.query_map(rusqlite::params_from_iter(params.iter()), |r| {
|
||||
Ok((
|
||||
r.get::<_, i64>(4)?,
|
||||
r.get::<_, String>(0)?,
|
||||
r.get::<_, String>(1)?,
|
||||
r.get::<_, Option<String>>(2)?,
|
||||
r.get::<_, i64>(3)?,
|
||||
))
|
||||
})? {
|
||||
let (rowid, server_id, artist_id, artist, synced_at) = row?;
|
||||
let fts_rank = rank.get(&rowid).copied().unwrap_or(usize::MAX);
|
||||
ranked.push((
|
||||
fts_rank,
|
||||
LibraryArtistDto {
|
||||
server_id,
|
||||
id: artist_id,
|
||||
name: artist.unwrap_or_default(),
|
||||
album_count: None,
|
||||
synced_at,
|
||||
raw_json: serde_json::Value::Null,
|
||||
},
|
||||
));
|
||||
}
|
||||
ranked.sort_by_key(|(r, _)| *r);
|
||||
let mut out = Vec::new();
|
||||
let mut seen = HashSet::new();
|
||||
for (_, dto) in ranked {
|
||||
if !seen.insert(dto.id.clone()) {
|
||||
continue;
|
||||
}
|
||||
out.push(dto);
|
||||
if out.len() >= limit as usize {
|
||||
break;
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn query_albums(
|
||||
conn: &rusqlite::Connection,
|
||||
query: &str,
|
||||
server_id: &str,
|
||||
library_scope: Option<&str>,
|
||||
limit: u32,
|
||||
) -> rusqlite::Result<Vec<LibraryAlbumDto>> {
|
||||
let matches = column_matches(query, &ALBUM_FTS_COLUMNS).map_err(|e| {
|
||||
rusqlite::Error::ToSqlConversionFailure(Box::new(std::io::Error::new(
|
||||
std::io::ErrorKind::InvalidInput,
|
||||
e,
|
||||
)))
|
||||
})?;
|
||||
let fetch = limit.saturating_mul(3).clamp(limit, 24);
|
||||
let rowids = collect_fts_rowids(conn, &matches, i64::from(fetch), fetch as usize)?;
|
||||
if rowids.is_empty() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
let placeholders = rowid_placeholders(rowids.len());
|
||||
let sql = format!(
|
||||
"SELECT t.server_id, t.album_id, t.album, t.artist, t.artist_id, t.year, \
|
||||
t.genre, t.cover_art_id, t.starred_at, t.synced_at, t.rowid \
|
||||
FROM track t \
|
||||
WHERE t.rowid IN ({placeholders}) \
|
||||
AND t.server_id = ? \
|
||||
AND t.deleted = 0 \
|
||||
AND t.album_id IS NOT NULL AND t.album_id != ''"
|
||||
);
|
||||
let mut params: Vec<rusqlite::types::Value> = rowids
|
||||
.iter()
|
||||
.copied()
|
||||
.map(rusqlite::types::Value::Integer)
|
||||
.collect();
|
||||
params.push(rusqlite::types::Value::Text(server_id.to_string()));
|
||||
let mut sql = sql;
|
||||
append_library_scope(&mut sql, &mut params, library_scope);
|
||||
let rank: HashMap<i64, usize> = rowids
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(i, &rid)| (rid, i))
|
||||
.collect();
|
||||
let mut ranked: Vec<(usize, LibraryAlbumDto)> = Vec::new();
|
||||
let mut stmt = conn.prepare(&sql)?;
|
||||
for row in stmt.query_map(rusqlite::params_from_iter(params.iter()), |r| {
|
||||
Ok((
|
||||
r.get::<_, i64>(10)?,
|
||||
r.get::<_, String>(0)?,
|
||||
r.get::<_, String>(1)?,
|
||||
r.get::<_, String>(2)?,
|
||||
r.get::<_, Option<String>>(3)?,
|
||||
r.get::<_, Option<String>>(4)?,
|
||||
r.get::<_, Option<i64>>(5)?,
|
||||
r.get::<_, Option<String>>(6)?,
|
||||
r.get::<_, Option<String>>(7)?,
|
||||
r.get::<_, Option<i64>>(8)?,
|
||||
r.get::<_, i64>(9)?,
|
||||
))
|
||||
})? {
|
||||
let (
|
||||
rowid,
|
||||
server_id,
|
||||
album_id,
|
||||
album,
|
||||
artist,
|
||||
artist_id,
|
||||
year,
|
||||
genre,
|
||||
cover_art_id,
|
||||
starred_at,
|
||||
synced_at,
|
||||
) = row?;
|
||||
let fts_rank = rank.get(&rowid).copied().unwrap_or(usize::MAX);
|
||||
ranked.push((
|
||||
fts_rank,
|
||||
LibraryAlbumDto {
|
||||
server_id,
|
||||
id: album_id,
|
||||
name: album,
|
||||
artist,
|
||||
artist_id,
|
||||
song_count: None,
|
||||
duration_sec: None,
|
||||
year,
|
||||
genre,
|
||||
cover_art_id,
|
||||
starred_at,
|
||||
synced_at,
|
||||
raw_json: serde_json::Value::Null,
|
||||
},
|
||||
));
|
||||
}
|
||||
ranked.sort_by_key(|(r, _)| *r);
|
||||
let mut out = Vec::new();
|
||||
let mut seen = HashSet::new();
|
||||
for (_, dto) in ranked {
|
||||
if !seen.insert(dto.id.clone()) {
|
||||
continue;
|
||||
}
|
||||
out.push(dto);
|
||||
if out.len() >= limit as usize {
|
||||
break;
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn rowid_placeholders(n: usize) -> String {
|
||||
(0..n).map(|_| "?").collect::<Vec<_>>().join(", ")
|
||||
}
|
||||
|
||||
fn fetch_tracks_by_rowids(
|
||||
conn: &rusqlite::Connection,
|
||||
rowids: &[i64],
|
||||
server_id: &str,
|
||||
library_scope: Option<&str>,
|
||||
) -> rusqlite::Result<Vec<LibraryTrackDto>> {
|
||||
let placeholders = rowid_placeholders(rowids.len());
|
||||
let sql = format!(
|
||||
"SELECT \
|
||||
t.rowid, \
|
||||
t.server_id, t.id, t.title, t.artist, t.artist_id, t.album, t.album_id, \
|
||||
t.album_artist, t.duration_sec, t.track_number, t.disc_number, t.year, \
|
||||
t.genre, t.suffix, t.bit_rate, t.size_bytes, t.cover_art_id, \
|
||||
t.starred_at, t.user_rating, t.play_count, t.bpm, t.synced_at \
|
||||
FROM track t \
|
||||
WHERE t.rowid IN ({placeholders}) \
|
||||
AND t.server_id = ? \
|
||||
AND t.deleted = 0"
|
||||
);
|
||||
let mut params: Vec<rusqlite::types::Value> = rowids
|
||||
.iter()
|
||||
.copied()
|
||||
.map(rusqlite::types::Value::Integer)
|
||||
.collect();
|
||||
params.push(rusqlite::types::Value::Text(server_id.to_string()));
|
||||
let mut sql = sql;
|
||||
append_library_scope(&mut sql, &mut params, library_scope);
|
||||
let mut stmt = conn.prepare(&sql)?;
|
||||
let mut by_rowid: HashMap<i64, LibraryTrackDto> = HashMap::new();
|
||||
for row in stmt.query_map(rusqlite::params_from_iter(params.iter()), |r| {
|
||||
let rowid: i64 = r.get(0)?;
|
||||
let hit = map_live_hit_row(r, 1)?;
|
||||
Ok((rowid, hit.track))
|
||||
})? {
|
||||
let (rowid, track) = row?;
|
||||
by_rowid.insert(rowid, track);
|
||||
}
|
||||
Ok(rowids
|
||||
.iter()
|
||||
.filter_map(|rid| by_rowid.get(rid).cloned())
|
||||
.collect())
|
||||
}
|
||||
|
||||
fn map_live_hit_row(row: &rusqlite::Row<'_>, offset: usize) -> rusqlite::Result<LiveHit> {
|
||||
Ok(LiveHit {
|
||||
track: LibraryTrackDto {
|
||||
server_id: row.get(offset)?,
|
||||
id: row.get(offset + 1)?,
|
||||
content_hash: None,
|
||||
title: row.get(offset + 2)?,
|
||||
title_sort: None,
|
||||
artist: row.get(offset + 3)?,
|
||||
artist_id: row.get(offset + 4)?,
|
||||
album: row.get(offset + 5)?,
|
||||
album_id: row.get(offset + 6)?,
|
||||
album_artist: row.get(offset + 7)?,
|
||||
duration_sec: row.get(offset + 8)?,
|
||||
track_number: row.get(offset + 9)?,
|
||||
disc_number: row.get(offset + 10)?,
|
||||
year: row.get(offset + 11)?,
|
||||
genre: row.get(offset + 12)?,
|
||||
suffix: row.get(offset + 13)?,
|
||||
bit_rate: row.get(offset + 14)?,
|
||||
size_bytes: row.get(offset + 15)?,
|
||||
cover_art_id: row.get(offset + 16)?,
|
||||
starred_at: row.get(offset + 17)?,
|
||||
user_rating: row.get(offset + 18)?,
|
||||
play_count: row.get(offset + 19)?,
|
||||
bpm: row.get(offset + 20)?,
|
||||
played_at: None,
|
||||
server_path: None,
|
||||
library_id: None,
|
||||
isrc: None,
|
||||
mbid_recording: None,
|
||||
replay_gain_track_db: None,
|
||||
replay_gain_album_db: None,
|
||||
server_updated_at: None,
|
||||
server_created_at: None,
|
||||
synced_at: row.get(offset + 21)?,
|
||||
enrichment: None,
|
||||
raw_json: serde_json::Value::Null,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::repos::{TrackRepository, TrackRow};
|
||||
use serde_json::json;
|
||||
|
||||
fn track(
|
||||
server: &str,
|
||||
id: &str,
|
||||
title: &str,
|
||||
artist: &str,
|
||||
album: &str,
|
||||
album_id: &str,
|
||||
artist_id: &str,
|
||||
) -> TrackRow {
|
||||
TrackRow {
|
||||
server_id: server.into(),
|
||||
id: id.into(),
|
||||
title: title.into(),
|
||||
title_sort: None,
|
||||
artist: Some(artist.into()),
|
||||
artist_id: Some(artist_id.into()),
|
||||
album: album.into(),
|
||||
album_id: Some(album_id.into()),
|
||||
album_artist: Some(artist.into()),
|
||||
duration_sec: 200,
|
||||
track_number: Some(1),
|
||||
disc_number: Some(1),
|
||||
year: None,
|
||||
genre: None,
|
||||
suffix: None,
|
||||
bit_rate: None,
|
||||
size_bytes: None,
|
||||
cover_art_id: Some(format!("cv_{album_id}")),
|
||||
starred_at: None,
|
||||
user_rating: None,
|
||||
play_count: None,
|
||||
played_at: None,
|
||||
server_path: None,
|
||||
library_id: None,
|
||||
isrc: None,
|
||||
mbid_recording: None,
|
||||
bpm: None,
|
||||
replay_gain_track_db: None,
|
||||
replay_gain_album_db: None,
|
||||
content_hash: None,
|
||||
server_updated_at: None,
|
||||
server_created_at: None,
|
||||
deleted: false,
|
||||
synced_at: 1,
|
||||
raw_json: "{}".into(),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn live_search_prefix_matches_partial_artist_name() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
TrackRepository::new(&store)
|
||||
.upsert_batch(&[
|
||||
track(
|
||||
"s1",
|
||||
"t1",
|
||||
"Enter Sandman",
|
||||
"Metallica",
|
||||
"Metallica",
|
||||
"al1",
|
||||
"ar_meta",
|
||||
),
|
||||
track("s1", "t2", "Other", "Other Artist", "Other Album", "al2", "ar2"),
|
||||
])
|
||||
.unwrap();
|
||||
let resp = run_live_search(&store, "s1", "metal", None, 5, 5, 10).unwrap();
|
||||
assert!(
|
||||
resp.artists.iter().any(|a| a.name == "Metallica"),
|
||||
"expected Metallica from prefix query metal"
|
||||
);
|
||||
assert!(resp.tracks.iter().any(|t| t.artist.as_deref() == Some("Metallica")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn live_search_returns_songs_albums_artists_from_scoped_fts() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
TrackRepository::new(&store)
|
||||
.upsert_batch(&[
|
||||
track("s1", "t1", "Aurora Song", "Aurora Quartet", "Aurora Nights", "al1", "ar1"),
|
||||
track("s1", "t2", "Other", "Other Artist", "Other Album", "al2", "ar2"),
|
||||
])
|
||||
.unwrap();
|
||||
let resp = run_live_search(&store, "s1", "aurora", None, 5, 5, 10).unwrap();
|
||||
assert_eq!(resp.tracks.len(), 1);
|
||||
assert_eq!(resp.albums.len(), 1);
|
||||
assert_eq!(resp.albums[0].id, "al1");
|
||||
assert_eq!(resp.artists.len(), 1);
|
||||
assert_eq!(resp.artists[0].id, "ar1");
|
||||
assert!(resp.tracks[0].raw_json.is_null());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn live_search_does_not_surface_artist_from_unrelated_track_hit() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
TrackRepository::new(&store)
|
||||
.upsert_batch(&[
|
||||
track(
|
||||
"s1",
|
||||
"t1",
|
||||
"Battle Hymn",
|
||||
"Arch Enemy",
|
||||
"Manowar Covers Vol 1",
|
||||
"al1",
|
||||
"ar_arch",
|
||||
),
|
||||
track(
|
||||
"s1",
|
||||
"t2",
|
||||
"Heart Of Steel",
|
||||
"Manowar",
|
||||
"Fighting the World",
|
||||
"al2",
|
||||
"ar_mano",
|
||||
),
|
||||
])
|
||||
.unwrap();
|
||||
let resp = run_live_search(&store, "s1", "manowar", None, 5, 5, 10).unwrap();
|
||||
assert!(
|
||||
resp.artists.iter().any(|a| a.name == "Manowar"),
|
||||
"expected Manowar artist"
|
||||
);
|
||||
assert!(
|
||||
!resp.artists.iter().any(|a| a.name == "Arch Enemy"),
|
||||
"Arch Enemy must not appear when only the album title mentions Manowar"
|
||||
);
|
||||
assert!(resp.albums.iter().any(|a| a.name.contains("Manowar")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn live_search_short_query_returns_empty_without_scanning() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
TrackRepository::new(&store)
|
||||
.upsert_batch(&[track(
|
||||
"s1",
|
||||
"t1",
|
||||
"Аура",
|
||||
"Artist",
|
||||
"Album",
|
||||
"al1",
|
||||
"ar1",
|
||||
)])
|
||||
.unwrap();
|
||||
let resp = run_live_search(&store, "s1", "а", None, 5, 5, 10).unwrap();
|
||||
assert!(resp.tracks.is_empty());
|
||||
assert!(resp.artists.is_empty());
|
||||
assert!(resp.albums.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn live_search_library_scope_narrows_results() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let mut in_lib = track(
|
||||
"s1",
|
||||
"t1",
|
||||
"Scoped Song",
|
||||
"Scoped Artist",
|
||||
"Scoped Album",
|
||||
"al1",
|
||||
"ar1",
|
||||
);
|
||||
in_lib.library_id = Some("lib1".into());
|
||||
let mut other = track(
|
||||
"s1",
|
||||
"t2",
|
||||
"Scoped Song",
|
||||
"Other Artist",
|
||||
"Other Album",
|
||||
"al2",
|
||||
"ar2",
|
||||
);
|
||||
other.library_id = Some("lib2".into());
|
||||
TrackRepository::new(&store)
|
||||
.upsert_batch(&[in_lib, other])
|
||||
.unwrap();
|
||||
let resp = run_live_search(&store, "s1", "scoped", Some("lib1"), 5, 5, 10).unwrap();
|
||||
assert_eq!(resp.tracks.len(), 1);
|
||||
assert_eq!(resp.tracks[0].id, "t1");
|
||||
assert_eq!(resp.artists.len(), 1);
|
||||
assert_eq!(resp.artists[0].name, "Scoped Artist");
|
||||
assert_eq!(resp.albums.len(), 1);
|
||||
assert_eq!(resp.albums[0].name, "Scoped Album");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn live_search_library_scope_matches_raw_json_when_column_null() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let mut row = track(
|
||||
"s1",
|
||||
"t1",
|
||||
"Scoped Song",
|
||||
"Scoped Artist",
|
||||
"Scoped Album",
|
||||
"al1",
|
||||
"ar1",
|
||||
);
|
||||
row.raw_json = json!({"libraryId": 3}).to_string();
|
||||
TrackRepository::new(&store).upsert_batch(&[row]).unwrap();
|
||||
let resp = run_live_search(&store, "s1", "scoped", Some("3"), 5, 5, 10).unwrap();
|
||||
assert_eq!(resp.tracks.len(), 1);
|
||||
assert_eq!(resp.tracks[0].id, "t1");
|
||||
}
|
||||
|
||||
/// Manual: `cargo test -p psysonic-library bench_disk_live_search --release -- --ignored --nocapture`
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn bench_disk_live_search() {
|
||||
use std::path::PathBuf;
|
||||
use std::time::Instant;
|
||||
|
||||
let path: PathBuf = std::env::var("HOME")
|
||||
.map(|h| PathBuf::from(h).join(".local/share/dev.psysonic.player/library.sqlite"))
|
||||
.expect("HOME");
|
||||
if !path.exists() {
|
||||
eprintln!("skip: no db at {}", path.display());
|
||||
return;
|
||||
}
|
||||
let conn = rusqlite::Connection::open_with_flags(
|
||||
&path,
|
||||
rusqlite::OpenFlags::SQLITE_OPEN_READ_ONLY,
|
||||
)
|
||||
.expect("open db");
|
||||
conn.pragma_update(None, "cache_size", -64000).unwrap();
|
||||
|
||||
let server_id = std::env::var("PSYSONIC_BENCH_SERVER_ID").unwrap_or_else(|_| {
|
||||
conn.query_row(
|
||||
"SELECT server_id FROM track WHERE deleted = 0 LIMIT 1",
|
||||
[],
|
||||
|r| r.get::<_, String>(0),
|
||||
)
|
||||
.expect("server_id")
|
||||
});
|
||||
|
||||
for q in ["manowar", "metallica", "arch enemy", "metal", "meta"] {
|
||||
let t0 = Instant::now();
|
||||
let songs = query_songs(&conn, q, &server_id, None, 10).unwrap();
|
||||
let t1 = Instant::now();
|
||||
let artists = query_artists(&conn, q, &server_id, None, 5).unwrap();
|
||||
let t2 = Instant::now();
|
||||
let albums = query_albums(&conn, q, &server_id, None, 5).unwrap();
|
||||
let t3 = Instant::now();
|
||||
eprintln!(
|
||||
"{q:?}: songs={} ({:.1}ms) artists={} ({:.1}ms) albums={} ({:.1}ms) total={:.1}ms",
|
||||
songs.len(),
|
||||
t1.duration_since(t0).as_secs_f64() * 1000.0,
|
||||
artists.len(),
|
||||
t2.duration_since(t1).as_secs_f64() * 1000.0,
|
||||
albums.len(),
|
||||
t3.duration_since(t2).as_secs_f64() * 1000.0,
|
||||
t3.duration_since(t0).as_secs_f64() * 1000.0,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,240 @@
|
||||
//! `ProgressEvent` → Tauri event payload mapper. PR-5a ships the
|
||||
//! transformation as a pure function so it's unit-testable without
|
||||
//! Tauri / supervisor wiring; PR-5b plugs the mpsc receiver into
|
||||
//! `AppHandle::emit("library:sync-progress", payload)`.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::sync::progress::ProgressEvent;
|
||||
|
||||
/// Wire shape for the `library:sync-progress` / `library:sync-idle`
|
||||
/// Tauri events. Carries the `serverId` + `libraryScope` so the
|
||||
/// frontend can demultiplex across multiple servers.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct LibrarySyncProgressPayload {
|
||||
pub server_id: String,
|
||||
pub library_scope: String,
|
||||
/// Discriminator: `"phase_changed"` / `"ingest_page"` /
|
||||
/// `"remapped"` / `"tombstoned"` / `"completed"` / `"error"`.
|
||||
pub kind: String,
|
||||
pub phase: Option<String>,
|
||||
pub ingested_total: Option<u32>,
|
||||
pub batch_count: Option<u32>,
|
||||
pub remapped_count: Option<u32>,
|
||||
pub tombstones_checked: Option<u32>,
|
||||
pub tombstones_deleted: Option<u32>,
|
||||
pub completed_kind: Option<String>,
|
||||
pub message: Option<String>,
|
||||
/// Per-batch ingest timings (S1 initial sync).
|
||||
pub ingest_metrics: Option<crate::sync::progress::IngestBatchMetrics>,
|
||||
}
|
||||
|
||||
impl LibrarySyncProgressPayload {
|
||||
pub fn from_event(event: &ProgressEvent, server_id: &str, library_scope: &str) -> Self {
|
||||
let mut payload = Self {
|
||||
server_id: server_id.to_string(),
|
||||
library_scope: library_scope.to_string(),
|
||||
kind: String::new(),
|
||||
phase: None,
|
||||
ingested_total: None,
|
||||
batch_count: None,
|
||||
remapped_count: None,
|
||||
tombstones_checked: None,
|
||||
tombstones_deleted: None,
|
||||
completed_kind: None,
|
||||
message: None,
|
||||
ingest_metrics: None,
|
||||
};
|
||||
match event {
|
||||
ProgressEvent::PhaseChanged { phase } => {
|
||||
payload.kind = "phase_changed".into();
|
||||
payload.phase = Some(phase.clone());
|
||||
}
|
||||
ProgressEvent::IngestPage {
|
||||
ingested_total,
|
||||
batch_count,
|
||||
metrics,
|
||||
} => {
|
||||
payload.kind = "ingest_page".into();
|
||||
payload.ingested_total = Some(*ingested_total);
|
||||
payload.batch_count = Some(*batch_count);
|
||||
payload.ingest_metrics = metrics.clone();
|
||||
}
|
||||
ProgressEvent::Remapped { entries } => {
|
||||
payload.kind = "remapped".into();
|
||||
payload.remapped_count = Some(entries.len() as u32);
|
||||
}
|
||||
ProgressEvent::Tombstoned {
|
||||
deleted_count,
|
||||
checked_count,
|
||||
} => {
|
||||
payload.kind = "tombstoned".into();
|
||||
payload.tombstones_deleted = Some(*deleted_count);
|
||||
payload.tombstones_checked = Some(*checked_count);
|
||||
}
|
||||
ProgressEvent::Completed { kind } => {
|
||||
payload.kind = "completed".into();
|
||||
payload.completed_kind = Some(kind.clone());
|
||||
}
|
||||
ProgressEvent::Error { message } => {
|
||||
payload.kind = "error".into();
|
||||
payload.message = Some(message.clone());
|
||||
}
|
||||
}
|
||||
payload
|
||||
}
|
||||
|
||||
/// Convenience constant for the event-name Tauri emits.
|
||||
pub const PROGRESS_EVENT_NAME: &'static str = "library:sync-progress";
|
||||
pub const IDLE_EVENT_NAME: &'static str = "library:sync-idle";
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::repos::RemapEntry;
|
||||
|
||||
#[test]
|
||||
fn phase_changed_maps_to_phase_kind() {
|
||||
let p = LibrarySyncProgressPayload::from_event(
|
||||
&ProgressEvent::PhaseChanged { phase: "ingest".into() },
|
||||
"s1",
|
||||
"",
|
||||
);
|
||||
assert_eq!(p.kind, "phase_changed");
|
||||
assert_eq!(p.phase.as_deref(), Some("ingest"));
|
||||
assert_eq!(p.server_id, "s1");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ingest_page_carries_total_and_batch_count() {
|
||||
let p = LibrarySyncProgressPayload::from_event(
|
||||
&ProgressEvent::IngestPage {
|
||||
ingested_total: 2500,
|
||||
batch_count: 5,
|
||||
metrics: None,
|
||||
},
|
||||
"s1",
|
||||
"lib-1",
|
||||
);
|
||||
assert_eq!(p.kind, "ingest_page");
|
||||
assert_eq!(p.ingested_total, Some(2500));
|
||||
assert_eq!(p.batch_count, Some(5));
|
||||
assert_eq!(p.library_scope, "lib-1");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn remapped_count_reflects_entries_length() {
|
||||
let p = LibrarySyncProgressPayload::from_event(
|
||||
&ProgressEvent::Remapped {
|
||||
entries: vec![
|
||||
RemapEntry {
|
||||
server_id: "s1".into(),
|
||||
old_id: "a".into(),
|
||||
new_id: "b".into(),
|
||||
},
|
||||
RemapEntry {
|
||||
server_id: "s1".into(),
|
||||
old_id: "c".into(),
|
||||
new_id: "d".into(),
|
||||
},
|
||||
],
|
||||
},
|
||||
"s1",
|
||||
"",
|
||||
);
|
||||
assert_eq!(p.kind, "remapped");
|
||||
assert_eq!(p.remapped_count, Some(2));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tombstoned_carries_checked_and_deleted() {
|
||||
let p = LibrarySyncProgressPayload::from_event(
|
||||
&ProgressEvent::Tombstoned {
|
||||
deleted_count: 3,
|
||||
checked_count: 10,
|
||||
},
|
||||
"s1",
|
||||
"",
|
||||
);
|
||||
assert_eq!(p.kind, "tombstoned");
|
||||
assert_eq!(p.tombstones_deleted, Some(3));
|
||||
assert_eq!(p.tombstones_checked, Some(10));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn completed_event_records_kind_string() {
|
||||
let p = LibrarySyncProgressPayload::from_event(
|
||||
&ProgressEvent::Completed { kind: "initial_sync".into() },
|
||||
"s1",
|
||||
"",
|
||||
);
|
||||
assert_eq!(p.kind, "completed");
|
||||
assert_eq!(p.completed_kind.as_deref(), Some("initial_sync"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn error_event_records_message() {
|
||||
let p = LibrarySyncProgressPayload::from_event(
|
||||
&ProgressEvent::Error { message: "timeout".into() },
|
||||
"s1",
|
||||
"",
|
||||
);
|
||||
assert_eq!(p.kind, "error");
|
||||
assert_eq!(p.message.as_deref(), Some("timeout"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ingest_metrics_serialize_camel_case() {
|
||||
use crate::sync::progress::IngestBatchMetrics;
|
||||
|
||||
let p = LibrarySyncProgressPayload::from_event(
|
||||
&ProgressEvent::IngestPage {
|
||||
ingested_total: 500,
|
||||
batch_count: 1,
|
||||
metrics: Some(IngestBatchMetrics {
|
||||
offset: 4500,
|
||||
strategy: "s1".into(),
|
||||
fetch_ms: 120,
|
||||
write_ms: 8,
|
||||
lock_wait_ms: 0,
|
||||
sql_exec_ms: 7,
|
||||
persist_ms: 1,
|
||||
row_count: 500,
|
||||
bulk_ingest_active: true,
|
||||
}),
|
||||
},
|
||||
"s1",
|
||||
"",
|
||||
);
|
||||
let json = serde_json::to_value(&p).unwrap();
|
||||
let metrics = json.get("ingestMetrics").unwrap();
|
||||
assert_eq!(metrics.get("fetchMs").and_then(|v| v.as_u64()), Some(120));
|
||||
assert_eq!(metrics.get("lockWaitMs").and_then(|v| v.as_u64()), Some(0));
|
||||
assert_eq!(metrics.get("bulkIngestActive").and_then(|v| v.as_bool()), Some(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serialization_uses_camel_case_keys() {
|
||||
let p = LibrarySyncProgressPayload::from_event(
|
||||
&ProgressEvent::IngestPage {
|
||||
ingested_total: 1,
|
||||
batch_count: 1,
|
||||
metrics: None,
|
||||
},
|
||||
"s1",
|
||||
"",
|
||||
);
|
||||
let json = serde_json::to_value(&p).unwrap();
|
||||
for key in [
|
||||
"serverId",
|
||||
"libraryScope",
|
||||
"kind",
|
||||
"ingestedTotal",
|
||||
"batchCount",
|
||||
] {
|
||||
assert!(json.get(key).is_some(), "missing camelCase key `{key}`");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,453 @@
|
||||
//! E4 — `ArtifactRepository`: typed CRUD over `track_artifact` with the
|
||||
//! §5.11 / §5.12 storage rules.
|
||||
//!
|
||||
//! - **Lazy expiry (P34):** no background GC; a `get` first deletes the
|
||||
//! track's expired artifacts of that kind (negative-cache rows included),
|
||||
//! then returns the best surviving match.
|
||||
//! - **Size cap (§5.12 write path):** an artifact over 512 KB is rejected
|
||||
//! unless it is a user import (`source_kind == "user"`), so a runaway
|
||||
//! lyrics/blob fetch can't bloat `library.sqlite`.
|
||||
//!
|
||||
//! The Tauri commands (`library_get_artifact` / `library_put_artifact`)
|
||||
//! delegate here so there's one source of truth for the storage rules.
|
||||
|
||||
use rusqlite::types::Value;
|
||||
use rusqlite::{params, OptionalExtension};
|
||||
|
||||
use crate::dto::{ArtifactInputDto, TrackArtifactDto};
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
/// §5.12: cap a single artifact at 512 KB unless the user imported it.
|
||||
pub const MAX_ARTIFACT_BYTES: usize = 512 * 1024;
|
||||
|
||||
pub struct ArtifactRepository<'a> {
|
||||
store: &'a LibraryStore,
|
||||
}
|
||||
|
||||
impl<'a> ArtifactRepository<'a> {
|
||||
pub fn new(store: &'a LibraryStore) -> Self {
|
||||
Self { store }
|
||||
}
|
||||
|
||||
/// Lazily drop the track's expired artifacts of this kind (§5.12
|
||||
/// read-path GC — negative-cache rows expire too), then return the most
|
||||
/// recent surviving match. `source_kind` / `source_id` / `format` narrow
|
||||
/// the lookup when provided so the lyrics path can resolve a specific
|
||||
/// source or take the first match.
|
||||
// Mirrors the flat parameter list of the `library_get_artifact` Tauri
|
||||
// command (a fixed public contract); grouping them here would only
|
||||
// diverge the two signatures.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub fn get(
|
||||
&self,
|
||||
server_id: &str,
|
||||
track_id: &str,
|
||||
artifact_kind: &str,
|
||||
source_kind: Option<&str>,
|
||||
source_id: Option<&str>,
|
||||
format: Option<&str>,
|
||||
now: i64,
|
||||
) -> Result<Option<TrackArtifactDto>, String> {
|
||||
if self.store.bulk_ingest_active() {
|
||||
return self.get_readonly(
|
||||
server_id,
|
||||
track_id,
|
||||
artifact_kind,
|
||||
source_kind,
|
||||
source_id,
|
||||
format,
|
||||
);
|
||||
}
|
||||
self.store
|
||||
.with_conn_mut("artifact.get_gc", |conn| {
|
||||
// Lazy TTL cleanup, scoped to the looked-up kind.
|
||||
conn.execute(
|
||||
"DELETE FROM track_artifact \
|
||||
WHERE server_id = ?1 AND track_id = ?2 AND artifact_kind = ?3 \
|
||||
AND expires_at IS NOT NULL AND expires_at < ?4",
|
||||
params![server_id, track_id, artifact_kind, now],
|
||||
)?;
|
||||
|
||||
Self::query_one(
|
||||
conn,
|
||||
server_id,
|
||||
track_id,
|
||||
artifact_kind,
|
||||
source_kind,
|
||||
source_id,
|
||||
format,
|
||||
)
|
||||
})
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn get_readonly(
|
||||
&self,
|
||||
server_id: &str,
|
||||
track_id: &str,
|
||||
artifact_kind: &str,
|
||||
source_kind: Option<&str>,
|
||||
source_id: Option<&str>,
|
||||
format: Option<&str>,
|
||||
) -> Result<Option<TrackArtifactDto>, String> {
|
||||
self.store
|
||||
.with_read_conn(|conn| {
|
||||
Self::query_one(
|
||||
conn,
|
||||
server_id,
|
||||
track_id,
|
||||
artifact_kind,
|
||||
source_kind,
|
||||
source_id,
|
||||
format,
|
||||
)
|
||||
})
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn query_one(
|
||||
conn: &rusqlite::Connection,
|
||||
server_id: &str,
|
||||
track_id: &str,
|
||||
artifact_kind: &str,
|
||||
source_kind: Option<&str>,
|
||||
source_id: Option<&str>,
|
||||
format: Option<&str>,
|
||||
) -> rusqlite::Result<Option<TrackArtifactDto>> {
|
||||
let mut sql = String::from(
|
||||
"SELECT server_id, track_id, artifact_kind, format, source_kind, source_id, \
|
||||
language, content_text, content_bytes, not_found, content_hash, fetched_at, \
|
||||
expires_at FROM track_artifact \
|
||||
WHERE server_id = ?1 AND track_id = ?2 AND artifact_kind = ?3",
|
||||
);
|
||||
let mut bound: Vec<Value> = vec![
|
||||
Value::Text(server_id.to_string()),
|
||||
Value::Text(track_id.to_string()),
|
||||
Value::Text(artifact_kind.to_string()),
|
||||
];
|
||||
let mut next = 4;
|
||||
if let Some(sk) = source_kind {
|
||||
sql.push_str(&format!(" AND source_kind = ?{next}"));
|
||||
bound.push(Value::Text(sk.to_string()));
|
||||
next += 1;
|
||||
}
|
||||
if let Some(si) = source_id {
|
||||
sql.push_str(&format!(" AND source_id = ?{next}"));
|
||||
bound.push(Value::Text(si.to_string()));
|
||||
next += 1;
|
||||
}
|
||||
if let Some(fmt) = format {
|
||||
sql.push_str(&format!(" AND format = ?{next}"));
|
||||
bound.push(Value::Text(fmt.to_string()));
|
||||
}
|
||||
sql.push_str(" ORDER BY fetched_at DESC LIMIT 1");
|
||||
|
||||
let mut stmt = conn.prepare(&sql)?;
|
||||
stmt.query_row(rusqlite::params_from_iter(bound.iter()), row_to_artifact_dto)
|
||||
.optional()
|
||||
}
|
||||
|
||||
/// E3 readiness: is there a valid (non-expired, non-`not_found`) lyrics
|
||||
/// artifact for `(server_id, track_id)`? Pure read — no lazy GC, no writes —
|
||||
/// so the `library_get_track` enrichment summary stays read-only.
|
||||
pub fn lyrics_cached(&self, server_id: &str, track_id: &str, now: i64) -> Result<bool, String> {
|
||||
let query = |conn: &rusqlite::Connection| {
|
||||
conn.query_row(
|
||||
"SELECT EXISTS ( \
|
||||
SELECT 1 FROM track_artifact \
|
||||
WHERE server_id = ?1 AND track_id = ?2 \
|
||||
AND artifact_kind = 'lyrics' \
|
||||
AND not_found = 0 \
|
||||
AND (expires_at IS NULL OR expires_at >= ?3) \
|
||||
)",
|
||||
params![server_id, track_id, now],
|
||||
|r| r.get::<_, i64>(0),
|
||||
)
|
||||
};
|
||||
let exists = if self.store.bulk_ingest_active() {
|
||||
self.store.with_read_conn(query)?
|
||||
} else {
|
||||
self.store.with_conn("artifact.lyrics_cached", query)?
|
||||
};
|
||||
Ok(exists != 0)
|
||||
}
|
||||
|
||||
/// Upsert an artifact. Rejects content over [`MAX_ARTIFACT_BYTES`] unless
|
||||
/// it is a user import (§5.12). A negative-cache row (`not_found`) carries
|
||||
/// no content, so it always fits.
|
||||
pub fn put(
|
||||
&self,
|
||||
server_id: &str,
|
||||
track_id: &str,
|
||||
artifact: &ArtifactInputDto,
|
||||
now: i64,
|
||||
) -> Result<(), String> {
|
||||
let actual = artifact.content_text.as_ref().map_or(0, |s| s.len())
|
||||
+ artifact.content_blob.as_ref().map_or(0, |b| b.len());
|
||||
let declared = usize::try_from(artifact.content_bytes).unwrap_or(0);
|
||||
let size = actual.max(declared);
|
||||
if size > MAX_ARTIFACT_BYTES && artifact.source_kind != "user" {
|
||||
return Err(format!(
|
||||
"artifact too large: {size} bytes exceeds {MAX_ARTIFACT_BYTES} cap \
|
||||
(source_kind={})",
|
||||
artifact.source_kind
|
||||
));
|
||||
}
|
||||
|
||||
self.store
|
||||
.with_conn("artifact.put", |conn| {
|
||||
conn.execute(
|
||||
UPSERT_ARTIFACT,
|
||||
params![
|
||||
server_id,
|
||||
track_id,
|
||||
artifact.artifact_kind,
|
||||
artifact.format,
|
||||
artifact.language,
|
||||
artifact.source_kind,
|
||||
artifact.source_id,
|
||||
artifact.content_text,
|
||||
artifact.content_blob,
|
||||
artifact.content_bytes,
|
||||
if artifact.not_found { 1_i64 } else { 0 },
|
||||
artifact.content_hash,
|
||||
now,
|
||||
artifact.expires_at,
|
||||
],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
}
|
||||
|
||||
fn row_to_artifact_dto(row: &rusqlite::Row<'_>) -> rusqlite::Result<TrackArtifactDto> {
|
||||
Ok(TrackArtifactDto {
|
||||
server_id: row.get(0)?,
|
||||
track_id: row.get(1)?,
|
||||
artifact_kind: row.get(2)?,
|
||||
format: row.get(3)?,
|
||||
source_kind: row.get(4)?,
|
||||
source_id: row.get(5)?,
|
||||
language: row.get(6)?,
|
||||
content_text: row.get(7)?,
|
||||
content_bytes: row.get(8)?,
|
||||
not_found: row.get::<_, i64>(9)? != 0,
|
||||
content_hash: row.get(10)?,
|
||||
fetched_at: row.get(11)?,
|
||||
expires_at: row.get(12)?,
|
||||
})
|
||||
}
|
||||
|
||||
const UPSERT_ARTIFACT: &str = "INSERT INTO track_artifact \
|
||||
(server_id, track_id, artifact_kind, format, language, source_kind, source_id, \
|
||||
content_text, content_blob, content_bytes, not_found, content_hash, fetched_at, expires_at) \
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14) \
|
||||
ON CONFLICT(server_id, track_id, artifact_kind, source_kind, source_id, format) DO UPDATE SET \
|
||||
language = excluded.language, \
|
||||
content_text = excluded.content_text, \
|
||||
content_blob = excluded.content_blob, \
|
||||
content_bytes = excluded.content_bytes, \
|
||||
not_found = excluded.not_found, \
|
||||
content_hash = excluded.content_hash, \
|
||||
fetched_at = excluded.fetched_at, \
|
||||
expires_at = excluded.expires_at";
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn artifact(
|
||||
kind: &str,
|
||||
format: &str,
|
||||
source_kind: &str,
|
||||
source_id: &str,
|
||||
) -> ArtifactInputDto {
|
||||
ArtifactInputDto {
|
||||
artifact_kind: kind.into(),
|
||||
format: format.into(),
|
||||
source_kind: source_kind.into(),
|
||||
source_id: source_id.into(),
|
||||
language: None,
|
||||
content_text: Some("la la la".into()),
|
||||
content_blob: None,
|
||||
content_bytes: 0,
|
||||
not_found: false,
|
||||
content_hash: None,
|
||||
expires_at: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn seed_track(store: &LibraryStore, server: &str, id: &str) {
|
||||
store
|
||||
.with_conn("misc", |c| {
|
||||
c.execute(
|
||||
"INSERT INTO track (server_id, id, title, synced_at, raw_json) \
|
||||
VALUES (?1, ?2, 'T', 1, '{}')",
|
||||
params![server, id],
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn put_then_get_roundtrips() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "s1", "t1");
|
||||
let repo = ArtifactRepository::new(&store);
|
||||
repo.put("s1", "t1", &artifact("lyrics", "plain", "lrclib", "lrclib"), 100)
|
||||
.unwrap();
|
||||
let got = repo
|
||||
.get("s1", "t1", "lyrics", None, None, None, 200)
|
||||
.unwrap()
|
||||
.expect("artifact present");
|
||||
assert_eq!(got.artifact_kind, "lyrics");
|
||||
assert_eq!(got.content_text.as_deref(), Some("la la la"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn get_lazily_deletes_expired_artifact() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "s1", "t1");
|
||||
let repo = ArtifactRepository::new(&store);
|
||||
let mut a = artifact("lyrics", "plain", "lrclib", "lrclib");
|
||||
a.expires_at = Some(150);
|
||||
repo.put("s1", "t1", &a, 100).unwrap();
|
||||
|
||||
// read at t=200 → expired, dropped + treated as a miss.
|
||||
let got = repo.get("s1", "t1", "lyrics", None, None, None, 200).unwrap();
|
||||
assert!(got.is_none());
|
||||
|
||||
let total: i64 = store
|
||||
.with_conn("misc", |c| c.query_row("SELECT COUNT(*) FROM track_artifact", [], |r| r.get(0)))
|
||||
.unwrap();
|
||||
assert_eq!(total, 0, "expired row deleted, not just filtered");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn negative_cache_row_is_returned_until_it_expires() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "s1", "t1");
|
||||
let repo = ArtifactRepository::new(&store);
|
||||
let mut miss = artifact("lyrics", "plain", "lrclib", "lrclib");
|
||||
miss.content_text = None;
|
||||
miss.not_found = true;
|
||||
miss.expires_at = Some(1000);
|
||||
repo.put("s1", "t1", &miss, 100).unwrap();
|
||||
|
||||
let got = repo
|
||||
.get("s1", "t1", "lyrics", None, None, None, 500)
|
||||
.unwrap()
|
||||
.expect("negative-cache row still live");
|
||||
assert!(got.not_found, "caller sees the cached miss instead of refetching");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lyrics_cached_only_for_live_found_row() {
|
||||
// Live lyrics row → cached; nothing / negative-cache / expired → not.
|
||||
let live = LibraryStore::open_in_memory();
|
||||
seed_track(&live, "s1", "t1");
|
||||
let repo = ArtifactRepository::new(&live);
|
||||
assert!(!repo.lyrics_cached("s1", "t1", 200).unwrap(), "no row → not cached");
|
||||
repo.put("s1", "t1", &artifact("lyrics", "plain", "lrclib", "lrclib"), 100)
|
||||
.unwrap();
|
||||
assert!(repo.lyrics_cached("s1", "t1", 200).unwrap(), "live row → cached");
|
||||
|
||||
let neg = LibraryStore::open_in_memory();
|
||||
seed_track(&neg, "s1", "t1");
|
||||
let repo_neg = ArtifactRepository::new(&neg);
|
||||
let mut miss = artifact("lyrics", "plain", "lrclib", "lrclib");
|
||||
miss.content_text = None;
|
||||
miss.not_found = true;
|
||||
miss.expires_at = Some(1000);
|
||||
repo_neg.put("s1", "t1", &miss, 100).unwrap();
|
||||
assert!(
|
||||
!repo_neg.lyrics_cached("s1", "t1", 500).unwrap(),
|
||||
"negative-cache row is not 'cached'"
|
||||
);
|
||||
|
||||
let exp = LibraryStore::open_in_memory();
|
||||
seed_track(&exp, "s1", "t1");
|
||||
let repo_exp = ArtifactRepository::new(&exp);
|
||||
let mut expired = artifact("lyrics", "plain", "lrclib", "lrclib");
|
||||
expired.expires_at = Some(150);
|
||||
repo_exp.put("s1", "t1", &expired, 100).unwrap();
|
||||
assert!(
|
||||
!repo_exp.lyrics_cached("s1", "t1", 200).unwrap(),
|
||||
"expired lyrics not cached"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn get_filters_by_source_id_only() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "s1", "t1");
|
||||
let repo = ArtifactRepository::new(&store);
|
||||
repo.put("s1", "t1", &artifact("lyrics", "plain", "lrclib", "lrclib"), 1)
|
||||
.unwrap();
|
||||
repo.put("s1", "t1", &artifact("lyrics", "plain", "netease", "netease"), 2)
|
||||
.unwrap();
|
||||
|
||||
// source_id without source_kind must bind correctly (running indices).
|
||||
let got = repo
|
||||
.get("s1", "t1", "lyrics", None, Some("netease"), None, 3)
|
||||
.unwrap()
|
||||
.expect("netease row");
|
||||
assert_eq!(got.source_id, "netease");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn get_returns_most_recent_when_unfiltered() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "s1", "t1");
|
||||
let repo = ArtifactRepository::new(&store);
|
||||
repo.put("s1", "t1", &artifact("lyrics", "plain", "lrclib", "lrclib"), 1)
|
||||
.unwrap();
|
||||
repo.put("s1", "t1", &artifact("lyrics", "plain", "netease", "netease"), 9)
|
||||
.unwrap();
|
||||
let got = repo
|
||||
.get("s1", "t1", "lyrics", None, None, None, 10)
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
assert_eq!(got.source_id, "netease", "ORDER BY fetched_at DESC");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn put_rejects_oversized_non_user_artifact() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "s1", "t1");
|
||||
let repo = ArtifactRepository::new(&store);
|
||||
let mut big = artifact("lyrics", "plain", "lrclib", "lrclib");
|
||||
big.content_text = Some("x".repeat(MAX_ARTIFACT_BYTES + 1));
|
||||
let err = repo.put("s1", "t1", &big, 1).unwrap_err();
|
||||
assert!(err.contains("too large"), "got: {err}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn put_allows_oversized_user_import() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "s1", "t1");
|
||||
let repo = ArtifactRepository::new(&store);
|
||||
let mut big = artifact("lyrics", "plain", "user", "user");
|
||||
big.content_text = Some("x".repeat(MAX_ARTIFACT_BYTES + 1));
|
||||
repo.put("s1", "t1", &big, 1).expect("user import bypasses the cap");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn put_is_idempotent_on_same_source_key() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "s1", "t1");
|
||||
let repo = ArtifactRepository::new(&store);
|
||||
repo.put("s1", "t1", &artifact("lyrics", "plain", "lrclib", "lrclib"), 1)
|
||||
.unwrap();
|
||||
let mut updated = artifact("lyrics", "plain", "lrclib", "lrclib");
|
||||
updated.content_text = Some("new words".into());
|
||||
repo.put("s1", "t1", &updated, 2).unwrap();
|
||||
let count: i64 = store
|
||||
.with_conn("misc", |c| c.query_row("SELECT COUNT(*) FROM track_artifact", [], |r| r.get(0)))
|
||||
.unwrap();
|
||||
assert_eq!(count, 1, "same (kind, source, format) updates in place");
|
||||
let got = repo.get("s1", "t1", "lyrics", None, None, None, 3).unwrap().unwrap();
|
||||
assert_eq!(got.content_text.as_deref(), Some("new words"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,302 @@
|
||||
//! E4 — `FactRepository`: typed CRUD over `track_fact` with the §5.12
|
||||
//! provenance / TTL rules.
|
||||
//!
|
||||
//! - **Lazy expiry (P34):** there's no background GC; a `get` first deletes
|
||||
//! the track's expired facts, then returns the survivors.
|
||||
//! - **User override (R6-3.4):** a `user` BPM fact also writes `track.bpm` so
|
||||
//! the override wins and survives a server resync.
|
||||
//!
|
||||
//! The Tauri commands (`library_get_facts` / `library_put_fact`) delegate
|
||||
//! here so there's one source of truth for the storage rules.
|
||||
|
||||
use rusqlite::params;
|
||||
|
||||
use crate::dto::{FactInputDto, TrackFactDto};
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
pub struct FactRepository<'a> {
|
||||
store: &'a LibraryStore,
|
||||
}
|
||||
|
||||
impl<'a> FactRepository<'a> {
|
||||
pub fn new(store: &'a LibraryStore) -> Self {
|
||||
Self { store }
|
||||
}
|
||||
|
||||
/// Lazily drop the track's expired facts (§5.12 read-path GC), then
|
||||
/// return the survivors — optionally filtered to `fact_kinds`. Ordered
|
||||
/// `fact_kind ASC, fetched_at DESC` so the highest-priority source per
|
||||
/// kind (caller resolves) comes first within its group.
|
||||
pub fn get(
|
||||
&self,
|
||||
server_id: &str,
|
||||
track_id: &str,
|
||||
fact_kinds: &[String],
|
||||
now: i64,
|
||||
) -> Result<Vec<TrackFactDto>, String> {
|
||||
if self.store.bulk_ingest_active() {
|
||||
return self.get_readonly(server_id, track_id, fact_kinds);
|
||||
}
|
||||
self.store
|
||||
.with_conn_mut("fact.get_gc", |conn| {
|
||||
// Lazy TTL cleanup for this track.
|
||||
conn.execute(
|
||||
"DELETE FROM track_fact \
|
||||
WHERE server_id = ?1 AND track_id = ?2 \
|
||||
AND expires_at IS NOT NULL AND expires_at < ?3",
|
||||
params![server_id, track_id, now],
|
||||
)?;
|
||||
|
||||
Self::query_facts(conn, server_id, track_id, fact_kinds)
|
||||
})
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn get_readonly(
|
||||
&self,
|
||||
server_id: &str,
|
||||
track_id: &str,
|
||||
fact_kinds: &[String],
|
||||
) -> Result<Vec<TrackFactDto>, String> {
|
||||
self.store
|
||||
.with_read_conn(|conn| Self::query_facts(conn, server_id, track_id, fact_kinds))
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn query_facts(
|
||||
conn: &rusqlite::Connection,
|
||||
server_id: &str,
|
||||
track_id: &str,
|
||||
fact_kinds: &[String],
|
||||
) -> rusqlite::Result<Vec<TrackFactDto>> {
|
||||
if fact_kinds.is_empty() {
|
||||
let mut stmt = conn.prepare(SELECT_FACTS)?;
|
||||
let rows: rusqlite::Result<Vec<TrackFactDto>> = stmt
|
||||
.query_map(params![server_id, track_id], row_to_fact_dto)?
|
||||
.collect();
|
||||
rows
|
||||
} else {
|
||||
let placeholders = (0..fact_kinds.len())
|
||||
.map(|i| format!("?{}", i + 3))
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ");
|
||||
let sql = format!(
|
||||
"{SELECT_FACTS_BASE} AND fact_kind IN ({placeholders}) \
|
||||
ORDER BY fact_kind ASC, fetched_at DESC"
|
||||
);
|
||||
let mut bound: Vec<rusqlite::types::Value> = vec![
|
||||
rusqlite::types::Value::Text(server_id.to_string()),
|
||||
rusqlite::types::Value::Text(track_id.to_string()),
|
||||
];
|
||||
for k in fact_kinds {
|
||||
bound.push(rusqlite::types::Value::Text(k.clone()));
|
||||
}
|
||||
let mut stmt = conn.prepare(&sql)?;
|
||||
let rows: rusqlite::Result<Vec<TrackFactDto>> = stmt
|
||||
.query_map(rusqlite::params_from_iter(bound.iter()), row_to_fact_dto)?
|
||||
.collect();
|
||||
rows
|
||||
}
|
||||
}
|
||||
|
||||
/// Upsert a fact. A `user` BPM fact also writes the hot `track.bpm`
|
||||
/// column so the manual override beats the server tag and survives a
|
||||
/// resync (§5.12 write path / R6-3.4).
|
||||
pub fn put(
|
||||
&self,
|
||||
server_id: &str,
|
||||
track_id: &str,
|
||||
fact: &FactInputDto,
|
||||
now: i64,
|
||||
) -> Result<(), String> {
|
||||
self.store
|
||||
.with_conn("fact.put", |conn| {
|
||||
conn.execute(
|
||||
UPSERT_FACT,
|
||||
params![
|
||||
server_id,
|
||||
track_id,
|
||||
fact.fact_kind,
|
||||
fact.value_real,
|
||||
fact.value_int,
|
||||
fact.value_text,
|
||||
fact.unit,
|
||||
fact.source_kind,
|
||||
fact.source_id,
|
||||
fact.confidence,
|
||||
fact.content_hash,
|
||||
now,
|
||||
fact.expires_at,
|
||||
],
|
||||
)?;
|
||||
if fact.fact_kind == "bpm" && fact.source_kind == "user" {
|
||||
if let Some(bpm) = fact.value_int {
|
||||
conn.execute(
|
||||
"UPDATE track SET bpm = ?3 WHERE server_id = ?1 AND id = ?2",
|
||||
params![server_id, track_id, bpm],
|
||||
)?;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
}
|
||||
|
||||
fn row_to_fact_dto(row: &rusqlite::Row<'_>) -> rusqlite::Result<TrackFactDto> {
|
||||
Ok(TrackFactDto {
|
||||
server_id: row.get(0)?,
|
||||
track_id: row.get(1)?,
|
||||
fact_kind: row.get(2)?,
|
||||
value_real: row.get(3)?,
|
||||
value_int: row.get(4)?,
|
||||
value_text: row.get(5)?,
|
||||
unit: row.get(6)?,
|
||||
source_kind: row.get(7)?,
|
||||
source_id: row.get(8)?,
|
||||
confidence: row.get(9)?,
|
||||
content_hash: row.get(10)?,
|
||||
fetched_at: row.get(11)?,
|
||||
expires_at: row.get(12)?,
|
||||
})
|
||||
}
|
||||
|
||||
const SELECT_FACTS_BASE: &str = "SELECT server_id, track_id, fact_kind, value_real, value_int, \
|
||||
value_text, unit, source_kind, source_id, confidence, content_hash, fetched_at, expires_at \
|
||||
FROM track_fact WHERE server_id = ?1 AND track_id = ?2";
|
||||
|
||||
const SELECT_FACTS: &str = "SELECT server_id, track_id, fact_kind, value_real, value_int, \
|
||||
value_text, unit, source_kind, source_id, confidence, content_hash, fetched_at, expires_at \
|
||||
FROM track_fact WHERE server_id = ?1 AND track_id = ?2 \
|
||||
ORDER BY fact_kind ASC, fetched_at DESC";
|
||||
|
||||
const UPSERT_FACT: &str = "INSERT INTO track_fact \
|
||||
(server_id, track_id, fact_kind, value_real, value_int, value_text, unit, \
|
||||
source_kind, source_id, source_detail, confidence, content_hash, fetched_at, expires_at) \
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, NULL, ?10, ?11, ?12, ?13) \
|
||||
ON CONFLICT(server_id, track_id, fact_kind, source_kind, source_id) DO UPDATE SET \
|
||||
value_real = excluded.value_real, \
|
||||
value_int = excluded.value_int, \
|
||||
value_text = excluded.value_text, \
|
||||
unit = excluded.unit, \
|
||||
confidence = excluded.confidence, \
|
||||
content_hash = excluded.content_hash, \
|
||||
fetched_at = excluded.fetched_at, \
|
||||
expires_at = excluded.expires_at";
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn fact(kind: &str, source_kind: &str, value_int: Option<i64>, expires_at: Option<i64>) -> FactInputDto {
|
||||
FactInputDto {
|
||||
fact_kind: kind.into(),
|
||||
value_real: None,
|
||||
value_int,
|
||||
value_text: None,
|
||||
unit: None,
|
||||
source_kind: source_kind.into(),
|
||||
source_id: "seed".into(),
|
||||
confidence: 1.0,
|
||||
content_hash: None,
|
||||
expires_at,
|
||||
}
|
||||
}
|
||||
|
||||
fn seed_track(store: &LibraryStore, server: &str, id: &str) {
|
||||
store
|
||||
.with_conn("misc", |c| {
|
||||
c.execute(
|
||||
"INSERT INTO track (server_id, id, title, synced_at, raw_json) \
|
||||
VALUES (?1, ?2, 'T', 1, '{}')",
|
||||
params![server, id],
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn put_then_get_roundtrips() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "s1", "t1");
|
||||
let repo = FactRepository::new(&store);
|
||||
repo.put("s1", "t1", &fact("bpm", "analysis", Some(120), None), 100).unwrap();
|
||||
let facts = repo.get("s1", "t1", &[], 200).unwrap();
|
||||
assert_eq!(facts.len(), 1);
|
||||
assert_eq!(facts[0].fact_kind, "bpm");
|
||||
assert_eq!(facts[0].value_int, Some(120));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn get_lazily_deletes_expired_facts() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "s1", "t1");
|
||||
let repo = FactRepository::new(&store);
|
||||
// expires at t=150
|
||||
repo.put("s1", "t1", &fact("energy", "external_api", Some(5), Some(150)), 100).unwrap();
|
||||
repo.put("s1", "t1", &fact("bpm", "analysis", Some(120), None), 100).unwrap();
|
||||
|
||||
// read at t=200 → energy expired, dropped + excluded; bpm survives.
|
||||
let facts = repo.get("s1", "t1", &[], 200).unwrap();
|
||||
assert_eq!(facts.len(), 1);
|
||||
assert_eq!(facts[0].fact_kind, "bpm");
|
||||
|
||||
// and it was actually deleted from the table (not just filtered).
|
||||
let total: i64 = store
|
||||
.with_conn("misc", |c| c.query_row("SELECT COUNT(*) FROM track_fact", [], |r| r.get(0)))
|
||||
.unwrap();
|
||||
assert_eq!(total, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn get_filters_by_kind() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "s1", "t1");
|
||||
let repo = FactRepository::new(&store);
|
||||
repo.put("s1", "t1", &fact("bpm", "analysis", Some(120), None), 1).unwrap();
|
||||
repo.put("s1", "t1", &fact("energy", "analysis", Some(7), None), 1).unwrap();
|
||||
let facts = repo.get("s1", "t1", &["bpm".into()], 2).unwrap();
|
||||
assert_eq!(facts.len(), 1);
|
||||
assert_eq!(facts[0].fact_kind, "bpm");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn user_bpm_fact_also_writes_hot_track_column() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "s1", "t1");
|
||||
let repo = FactRepository::new(&store);
|
||||
repo.put("s1", "t1", &fact("bpm", "user", Some(128), None), 1).unwrap();
|
||||
let bpm: Option<i64> = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row("SELECT bpm FROM track WHERE server_id='s1' AND id='t1'", [], |r| r.get(0))
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(bpm, Some(128), "user bpm override must write track.bpm");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn analysis_bpm_fact_does_not_touch_hot_track_column() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "s1", "t1");
|
||||
let repo = FactRepository::new(&store);
|
||||
repo.put("s1", "t1", &fact("bpm", "analysis", Some(99), None), 1).unwrap();
|
||||
let bpm: Option<i64> = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row("SELECT bpm FROM track WHERE server_id='s1' AND id='t1'", [], |r| r.get(0))
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(bpm, None, "only a user override writes the hot column (§5.12)");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn put_is_idempotent_on_same_source_key() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "s1", "t1");
|
||||
let repo = FactRepository::new(&store);
|
||||
repo.put("s1", "t1", &fact("bpm", "analysis", Some(120), None), 1).unwrap();
|
||||
repo.put("s1", "t1", &fact("bpm", "analysis", Some(124), None), 2).unwrap();
|
||||
let facts = repo.get("s1", "t1", &[], 3).unwrap();
|
||||
assert_eq!(facts.len(), 1, "same (kind, source) updates in place");
|
||||
assert_eq!(facts[0].value_int, Some(124));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
pub mod artifact;
|
||||
pub mod fact;
|
||||
pub mod sync_state;
|
||||
pub mod track;
|
||||
pub mod track_id_history;
|
||||
|
||||
pub use artifact::ArtifactRepository;
|
||||
pub use fact::FactRepository;
|
||||
pub use sync_state::SyncStateRepository;
|
||||
pub use track::{RemapEntry, RemapStats, TrackRepository, TrackRow};
|
||||
pub use track_id_history::TrackIdHistoryRepository;
|
||||
|
||||
// Shared row-mapper + column list so the Advanced Search builder can project
|
||||
// the same hot columns as the repositories without re-declaring them.
|
||||
pub(crate) use track::{row_to_track_row, track_columns};
|
||||
@@ -0,0 +1,826 @@
|
||||
use rusqlite::{params, OptionalExtension};
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
/// Repository over the `sync_state` row identified by `(server_id, library_scope)`.
|
||||
/// PR-1b exposes just enough of the row to drive resumable initial sync — the
|
||||
/// orchestrator-side helpers (poll stats, phase transitions, …) land with
|
||||
/// PR-3 when there's actual sync code to consume them.
|
||||
pub struct SyncStateRepository<'a> {
|
||||
store: &'a LibraryStore,
|
||||
}
|
||||
|
||||
impl<'a> SyncStateRepository<'a> {
|
||||
pub fn new(store: &'a LibraryStore) -> Self {
|
||||
Self { store }
|
||||
}
|
||||
|
||||
/// Read-only queries — must not take the write mutex (ingest holds it for
|
||||
/// long stretches during IS-3).
|
||||
fn read<R>(
|
||||
&self,
|
||||
f: impl FnOnce(&rusqlite::Connection) -> rusqlite::Result<R>,
|
||||
) -> Result<R, String> {
|
||||
self.store.with_read_conn(f)
|
||||
}
|
||||
|
||||
/// Insert a default-valued row for this `(server_id, library_scope)` pair
|
||||
/// if none exists. All non-PK columns fall back to their schema DEFAULTs
|
||||
/// (`sync_phase='idle'`, `initial_sync_cursor_json='{}'`, …).
|
||||
pub fn ensure(&self, server_id: &str, library_scope: &str) -> Result<(), String> {
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.execute(
|
||||
"INSERT OR IGNORE INTO sync_state (server_id, library_scope) VALUES (?1, ?2)",
|
||||
params![server_id, library_scope],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// Read `initial_sync_cursor_json`. Returns `None` when the row doesn't
|
||||
/// exist yet, `Some(Value)` otherwise (the schema DEFAULT is `'{}'`, so
|
||||
/// a freshly-ensured row reads back as `Some(Object({}))`).
|
||||
pub fn get_initial_sync_cursor(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
) -> Result<Option<Value>, String> {
|
||||
let raw: Option<String> = self.read(|conn| {
|
||||
conn.query_row(
|
||||
"SELECT initial_sync_cursor_json FROM sync_state \
|
||||
WHERE server_id = ?1 AND library_scope = ?2",
|
||||
params![server_id, library_scope],
|
||||
|row| row.get(0),
|
||||
)
|
||||
.optional()
|
||||
})?;
|
||||
match raw {
|
||||
None => Ok(None),
|
||||
Some(s) => serde_json::from_str(&s)
|
||||
.map(Some)
|
||||
.map_err(|e| format!("invalid initial_sync_cursor_json: {e}")),
|
||||
}
|
||||
}
|
||||
|
||||
/// Write `initial_sync_cursor_json`. Creates the row if needed; only the
|
||||
/// cursor column is touched, all other columns keep their current values
|
||||
/// (or their DEFAULTs on first insert).
|
||||
pub fn set_initial_sync_cursor(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
cursor: &Value,
|
||||
) -> Result<(), String> {
|
||||
let json = serde_json::to_string(cursor).map_err(|e| e.to_string())?;
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, initial_sync_cursor_json) \
|
||||
VALUES (?1, ?2, ?3) \
|
||||
ON CONFLICT(server_id, library_scope) DO UPDATE SET \
|
||||
initial_sync_cursor_json = excluded.initial_sync_cursor_json",
|
||||
params![server_id, library_scope, json],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// Single write-lock acquisition for cursor + local count during ingest.
|
||||
pub fn set_initial_sync_cursor_and_local_track_count(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
cursor: &Value,
|
||||
local_track_count: i64,
|
||||
) -> Result<(), String> {
|
||||
let json = serde_json::to_string(cursor).map_err(|e| e.to_string())?;
|
||||
self.store.with_conn("sync_state.persist_cursor", |conn| {
|
||||
conn.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, initial_sync_cursor_json, local_track_count) \
|
||||
VALUES (?1, ?2, ?3, ?4) \
|
||||
ON CONFLICT(server_id, library_scope) DO UPDATE SET \
|
||||
initial_sync_cursor_json = excluded.initial_sync_cursor_json, \
|
||||
local_track_count = excluded.local_track_count",
|
||||
params![server_id, library_scope, json, local_track_count],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// Read `capability_flags` (spec §6.1.1). Returns `None` when the
|
||||
/// row doesn't exist; SQL DEFAULT is 0 so a freshly-ensured row
|
||||
/// reads back as `Some(0)`.
|
||||
pub fn get_capability_flags(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
) -> Result<Option<u32>, String> {
|
||||
let raw: Option<i64> = self.read(|conn| {
|
||||
conn.query_row(
|
||||
"SELECT capability_flags FROM sync_state \
|
||||
WHERE server_id = ?1 AND library_scope = ?2",
|
||||
params![server_id, library_scope],
|
||||
|row| row.get(0),
|
||||
)
|
||||
.optional()
|
||||
})?;
|
||||
Ok(raw.map(|v| v.max(0) as u32))
|
||||
}
|
||||
|
||||
/// Write `capability_flags`. Upsert scoped to that one column.
|
||||
pub fn set_capability_flags(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
flags: u32,
|
||||
) -> Result<(), String> {
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, capability_flags) \
|
||||
VALUES (?1, ?2, ?3) \
|
||||
ON CONFLICT(server_id, library_scope) DO UPDATE SET \
|
||||
capability_flags = excluded.capability_flags",
|
||||
params![server_id, library_scope, flags as i64],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// Read `sync_phase` (state-machine values per spec §6.2:
|
||||
/// `idle` / `probing` / `initial_sync` / `ready` / `error`).
|
||||
/// Returns `None` when the row doesn't exist; SQL DEFAULT is
|
||||
/// `'idle'` so a freshly-ensured row reads back as `Some("idle")`.
|
||||
pub fn get_sync_phase(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
) -> Result<Option<String>, String> {
|
||||
self.read(|conn| {
|
||||
conn.query_row(
|
||||
"SELECT sync_phase FROM sync_state \
|
||||
WHERE server_id = ?1 AND library_scope = ?2",
|
||||
params![server_id, library_scope],
|
||||
|row| row.get::<_, String>(0),
|
||||
)
|
||||
.optional()
|
||||
})
|
||||
}
|
||||
|
||||
/// True when a full sync has completed at least once.
|
||||
pub fn has_last_full_sync_at(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
) -> Result<bool, String> {
|
||||
self.read(|conn| {
|
||||
let ts: Option<Option<i64>> = conn
|
||||
.query_row(
|
||||
"SELECT last_full_sync_at FROM sync_state \
|
||||
WHERE server_id = ?1 AND library_scope = ?2",
|
||||
params![server_id, library_scope],
|
||||
|row| row.get(0),
|
||||
)
|
||||
.optional()?;
|
||||
Ok(ts.flatten().is_some())
|
||||
})
|
||||
}
|
||||
|
||||
/// Write `sync_phase`. Upsert scoped to that one column.
|
||||
pub fn set_sync_phase(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
phase: &str,
|
||||
) -> Result<(), String> {
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, sync_phase) \
|
||||
VALUES (?1, ?2, ?3) \
|
||||
ON CONFLICT(server_id, library_scope) DO UPDATE SET \
|
||||
sync_phase = excluded.sync_phase",
|
||||
params![server_id, library_scope, phase],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// Write `server_last_scan_iso` — server-reported timestamp of the
|
||||
/// last completed scan, captured from `getScanStatus.lastScan`.
|
||||
pub fn set_server_last_scan_iso(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
last_scan_iso: Option<&str>,
|
||||
) -> Result<(), String> {
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, server_last_scan_iso) \
|
||||
VALUES (?1, ?2, ?3) \
|
||||
ON CONFLICT(server_id, library_scope) DO UPDATE SET \
|
||||
server_last_scan_iso = excluded.server_last_scan_iso",
|
||||
params![server_id, library_scope, last_scan_iso],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// Write `indexes_last_modified_ms` — watermark for the file-tree
|
||||
/// browse path (`getIndexes.lastModified`).
|
||||
pub fn set_indexes_last_modified_ms(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
last_modified_ms: i64,
|
||||
) -> Result<(), String> {
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, indexes_last_modified_ms) \
|
||||
VALUES (?1, ?2, ?3) \
|
||||
ON CONFLICT(server_id, library_scope) DO UPDATE SET \
|
||||
indexes_last_modified_ms = excluded.indexes_last_modified_ms",
|
||||
params![server_id, library_scope, last_modified_ms],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// Read `artists_last_modified_ms`. Returns `None` when the row
|
||||
/// doesn't exist or the column is `NULL`. DS-2 in §6.4 compares
|
||||
/// the live `getArtists.lastModified` against this to decide
|
||||
/// whether a delta pass is needed.
|
||||
pub fn get_artists_last_modified_ms(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
) -> Result<Option<i64>, String> {
|
||||
self.read(|conn| {
|
||||
conn.query_row(
|
||||
"SELECT artists_last_modified_ms FROM sync_state \
|
||||
WHERE server_id = ?1 AND library_scope = ?2",
|
||||
params![server_id, library_scope],
|
||||
|row| row.get::<_, Option<i64>>(0),
|
||||
)
|
||||
.optional()
|
||||
})
|
||||
.map(|opt| opt.flatten())
|
||||
}
|
||||
|
||||
/// Read `server_last_scan_iso`. Returns `None` when row missing
|
||||
/// or column null. DS-2 uses this against `getScanStatus.lastScan`
|
||||
/// for the Huge-tier short-circuit.
|
||||
pub fn get_server_last_scan_iso(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
) -> Result<Option<String>, String> {
|
||||
self.read(|conn| {
|
||||
conn.query_row(
|
||||
"SELECT server_last_scan_iso FROM sync_state \
|
||||
WHERE server_id = ?1 AND library_scope = ?2",
|
||||
params![server_id, library_scope],
|
||||
|row| row.get::<_, Option<String>>(0),
|
||||
)
|
||||
.optional()
|
||||
})
|
||||
.map(|opt| opt.flatten())
|
||||
}
|
||||
|
||||
/// Read `library_tier`. Returns `None` when row missing. DS-0
|
||||
/// picks between `getArtists` (small/medium) and `getScanStatus`
|
||||
/// (huge) based on this.
|
||||
pub fn get_library_tier(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
) -> Result<Option<String>, String> {
|
||||
self.read(|conn| {
|
||||
conn.query_row(
|
||||
"SELECT library_tier FROM sync_state \
|
||||
WHERE server_id = ?1 AND library_scope = ?2",
|
||||
params![server_id, library_scope],
|
||||
|row| row.get::<_, String>(0),
|
||||
)
|
||||
.optional()
|
||||
})
|
||||
}
|
||||
|
||||
/// Read `next_poll_at` — epoch ms scheduling target. `None` when
|
||||
/// the row is missing or the column is `NULL` (no schedule yet).
|
||||
pub fn get_next_poll_at(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
) -> Result<Option<i64>, String> {
|
||||
self.read(|conn| {
|
||||
conn.query_row(
|
||||
"SELECT next_poll_at FROM sync_state \
|
||||
WHERE server_id = ?1 AND library_scope = ?2",
|
||||
params![server_id, library_scope],
|
||||
|row| row.get::<_, Option<i64>>(0),
|
||||
)
|
||||
.optional()
|
||||
})
|
||||
.map(|opt| opt.flatten())
|
||||
}
|
||||
|
||||
/// Write `next_poll_at`. Upsert scoped to that one column.
|
||||
pub fn set_next_poll_at(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
epoch_ms: i64,
|
||||
) -> Result<(), String> {
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, next_poll_at) \
|
||||
VALUES (?1, ?2, ?3) \
|
||||
ON CONFLICT(server_id, library_scope) DO UPDATE SET \
|
||||
next_poll_at = excluded.next_poll_at",
|
||||
params![server_id, library_scope, epoch_ms],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// Read `poll_stats_json`. Returns `Some(Value)` for an existing
|
||||
/// row (SQL DEFAULT is `'{}'`, so a freshly-ensured row reads
|
||||
/// back as `Some(Object({}))`), `None` when the row is absent.
|
||||
pub fn get_poll_stats_json(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
) -> Result<Option<Value>, String> {
|
||||
let raw: Option<String> = self.read(|conn| {
|
||||
conn.query_row(
|
||||
"SELECT poll_stats_json FROM sync_state \
|
||||
WHERE server_id = ?1 AND library_scope = ?2",
|
||||
params![server_id, library_scope],
|
||||
|row| row.get(0),
|
||||
)
|
||||
.optional()
|
||||
})?;
|
||||
match raw {
|
||||
None => Ok(None),
|
||||
Some(s) => serde_json::from_str(&s)
|
||||
.map(Some)
|
||||
.map_err(|e| format!("invalid poll_stats_json: {e}")),
|
||||
}
|
||||
}
|
||||
|
||||
/// Write `poll_stats_json`. Upsert scoped to that one column.
|
||||
pub fn set_poll_stats_json(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
stats: &Value,
|
||||
) -> Result<(), String> {
|
||||
let json = serde_json::to_string(stats).map_err(|e| e.to_string())?;
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, poll_stats_json) \
|
||||
VALUES (?1, ?2, ?3) \
|
||||
ON CONFLICT(server_id, library_scope) DO UPDATE SET \
|
||||
poll_stats_json = excluded.poll_stats_json",
|
||||
params![server_id, library_scope, json],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// Read `local_track_count` snapshot (counts kept in sync by C8 /
|
||||
/// PR-3d2 scheduler ticks). Returns `None` when unset.
|
||||
pub fn get_local_track_count(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
) -> Result<Option<i64>, String> {
|
||||
self.read(|conn| {
|
||||
conn.query_row(
|
||||
"SELECT local_track_count FROM sync_state \
|
||||
WHERE server_id = ?1 AND library_scope = ?2",
|
||||
params![server_id, library_scope],
|
||||
|row| row.get::<_, Option<i64>>(0),
|
||||
)
|
||||
.optional()
|
||||
})
|
||||
.map(|opt| opt.flatten())
|
||||
}
|
||||
|
||||
pub fn set_local_track_count(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
count: i64,
|
||||
) -> Result<(), String> {
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, local_track_count) \
|
||||
VALUES (?1, ?2, ?3) \
|
||||
ON CONFLICT(server_id, library_scope) DO UPDATE SET \
|
||||
local_track_count = excluded.local_track_count",
|
||||
params![server_id, library_scope, count],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
pub fn get_server_track_count(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
) -> Result<Option<i64>, String> {
|
||||
self.read(|conn| {
|
||||
conn.query_row(
|
||||
"SELECT server_track_count FROM sync_state \
|
||||
WHERE server_id = ?1 AND library_scope = ?2",
|
||||
params![server_id, library_scope],
|
||||
|row| row.get::<_, Option<i64>>(0),
|
||||
)
|
||||
.optional()
|
||||
})
|
||||
.map(|opt| opt.flatten())
|
||||
}
|
||||
|
||||
pub fn set_server_track_count(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
count: i64,
|
||||
) -> Result<(), String> {
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, server_track_count) \
|
||||
VALUES (?1, ?2, ?3) \
|
||||
ON CONFLICT(server_id, library_scope) DO UPDATE SET \
|
||||
server_track_count = excluded.server_track_count",
|
||||
params![server_id, library_scope, count],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// Read `n1_bulk_unreliable` — per-server learned flag (R7-15). When
|
||||
/// set, the strategy selector stops choosing N1 for this server (the
|
||||
/// native `/api/song` endpoint 500'd beyond a deep offset). Returns
|
||||
/// `None` when the row doesn't exist; SQL DEFAULT is 0 so a
|
||||
/// freshly-ensured row reads back as `Some(false)`.
|
||||
pub fn get_n1_bulk_unreliable(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
) -> Result<Option<bool>, String> {
|
||||
let raw: Option<i64> = self.read(|conn| {
|
||||
conn.query_row(
|
||||
"SELECT n1_bulk_unreliable FROM sync_state \
|
||||
WHERE server_id = ?1 AND library_scope = ?2",
|
||||
params![server_id, library_scope],
|
||||
|row| row.get(0),
|
||||
)
|
||||
.optional()
|
||||
})?;
|
||||
Ok(raw.map(|v| v != 0))
|
||||
}
|
||||
|
||||
/// Write `n1_bulk_unreliable`. Upsert scoped to that one column.
|
||||
pub fn set_n1_bulk_unreliable(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
unreliable: bool,
|
||||
) -> Result<(), String> {
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, n1_bulk_unreliable) \
|
||||
VALUES (?1, ?2, ?3) \
|
||||
ON CONFLICT(server_id, library_scope) DO UPDATE SET \
|
||||
n1_bulk_unreliable = excluded.n1_bulk_unreliable",
|
||||
params![server_id, library_scope, unreliable as i64],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// Stamp `last_full_sync_at = now` (epoch ms). Called by IS-6 when
|
||||
/// the initial full ingest completes successfully.
|
||||
pub fn set_last_full_sync_at(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
epoch_ms: i64,
|
||||
) -> Result<(), String> {
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, last_full_sync_at) \
|
||||
VALUES (?1, ?2, ?3) \
|
||||
ON CONFLICT(server_id, library_scope) DO UPDATE SET \
|
||||
last_full_sync_at = excluded.last_full_sync_at",
|
||||
params![server_id, library_scope, epoch_ms],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// Stamp `last_delta_sync_at = now` (epoch ms). Called by DS-9 at
|
||||
/// the end of every successful delta pass.
|
||||
pub fn set_last_delta_sync_at(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
epoch_ms: i64,
|
||||
) -> Result<(), String> {
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, last_delta_sync_at) \
|
||||
VALUES (?1, ?2, ?3) \
|
||||
ON CONFLICT(server_id, library_scope) DO UPDATE SET \
|
||||
last_delta_sync_at = excluded.last_delta_sync_at",
|
||||
params![server_id, library_scope, epoch_ms],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// Write `artists_last_modified_ms` — watermark for the ID3 path
|
||||
/// (`getArtists.lastModified`); §2.2.1 background poll keys off
|
||||
/// this.
|
||||
pub fn set_artists_last_modified_ms(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
last_modified_ms: i64,
|
||||
) -> Result<(), String> {
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, artists_last_modified_ms) \
|
||||
VALUES (?1, ?2, ?3) \
|
||||
ON CONFLICT(server_id, library_scope) DO UPDATE SET \
|
||||
artists_last_modified_ms = excluded.artists_last_modified_ms",
|
||||
params![server_id, library_scope, last_modified_ms],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// Write `library_tier` (spec §6.2.2 — `small` / `medium` / `huge`
|
||||
/// / `unknown`). Drives the adaptive poll interval; PR-3d wires
|
||||
/// the EWMA loop that picks this.
|
||||
pub fn set_library_tier(
|
||||
&self,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
tier: &str,
|
||||
) -> Result<(), String> {
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, library_tier) \
|
||||
VALUES (?1, ?2, ?3) \
|
||||
ON CONFLICT(server_id, library_scope) DO UPDATE SET \
|
||||
library_tier = excluded.library_tier",
|
||||
params![server_id, library_scope, tier],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn ensure_creates_row_with_default_cursor() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
repo.ensure("s1", "").unwrap();
|
||||
|
||||
let cursor = repo.get_initial_sync_cursor("s1", "").unwrap();
|
||||
assert_eq!(cursor, Some(json!({})), "DEFAULT must read back as empty object");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ensure_is_idempotent() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
repo.ensure("s1", "").unwrap();
|
||||
repo.ensure("s1", "").unwrap();
|
||||
|
||||
let count: i64 = store
|
||||
.with_conn("misc", |c| c.query_row("SELECT COUNT(*) FROM sync_state", [], |r| r.get(0)))
|
||||
.unwrap();
|
||||
assert_eq!(count, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn get_returns_none_for_missing_row() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
assert_eq!(repo.get_initial_sync_cursor("absent", "").unwrap(), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn set_roundtrips_nested_cursor_value() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
let cursor = json!({
|
||||
"phase": "ingest_tracks",
|
||||
"offset": 12_500,
|
||||
"last_seen_id": "tr_abc",
|
||||
"filters": { "library_id": "lib-1" },
|
||||
});
|
||||
repo.set_initial_sync_cursor("s1", "", &cursor).unwrap();
|
||||
let got = repo.get_initial_sync_cursor("s1", "").unwrap();
|
||||
assert_eq!(got, Some(cursor));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn set_overwrites_prior_cursor() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
repo.set_initial_sync_cursor("s1", "", &json!({"offset": 1})).unwrap();
|
||||
repo.set_initial_sync_cursor("s1", "", &json!({"offset": 2})).unwrap();
|
||||
let got = repo.get_initial_sync_cursor("s1", "").unwrap();
|
||||
assert_eq!(got, Some(json!({"offset": 2})));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn set_preserves_other_columns_on_upsert() {
|
||||
// The ON CONFLICT clause must only touch the cursor column. Other
|
||||
// DEFAULT-backed fields stay at their initial values across upserts.
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
repo.set_initial_sync_cursor("s1", "", &json!({"x": 1})).unwrap();
|
||||
|
||||
// Mutate a sibling column out-of-band to detect any accidental reset.
|
||||
store
|
||||
.with_conn("misc", |c| {
|
||||
c.execute(
|
||||
"UPDATE sync_state SET sync_phase = 'ingesting' WHERE server_id = 's1'",
|
||||
[],
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
|
||||
// Second cursor write must not touch sync_phase.
|
||||
repo.set_initial_sync_cursor("s1", "", &json!({"x": 2})).unwrap();
|
||||
let phase: String = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT sync_phase FROM sync_state WHERE server_id = 's1'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(phase, "ingesting");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn library_scope_separates_rows_per_server() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
repo.set_initial_sync_cursor("s1", "", &json!({"all": true})).unwrap();
|
||||
repo.set_initial_sync_cursor("s1", "lib-1", &json!({"lib": "one"})).unwrap();
|
||||
|
||||
assert_eq!(
|
||||
repo.get_initial_sync_cursor("s1", "").unwrap(),
|
||||
Some(json!({"all": true}))
|
||||
);
|
||||
assert_eq!(
|
||||
repo.get_initial_sync_cursor("s1", "lib-1").unwrap(),
|
||||
Some(json!({"lib": "one"}))
|
||||
);
|
||||
}
|
||||
|
||||
// ── PR-3a: capability_flags, sync_phase, watermarks, library_tier ────
|
||||
|
||||
#[test]
|
||||
fn capability_flags_roundtrip() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
repo.ensure("s1", "").unwrap();
|
||||
assert_eq!(repo.get_capability_flags("s1", "").unwrap(), Some(0));
|
||||
repo.set_capability_flags("s1", "", 0x002 | 0x010).unwrap();
|
||||
assert_eq!(repo.get_capability_flags("s1", "").unwrap(), Some(0x012));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn capability_flags_returns_none_for_missing_row() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
assert_eq!(repo.get_capability_flags("absent", "").unwrap(), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn capability_flags_set_creates_row_with_other_defaults() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
repo.set_capability_flags("s1", "", 0x008).unwrap();
|
||||
// sync_phase defaulted to 'idle' on the implicit insert.
|
||||
assert_eq!(repo.get_sync_phase("s1", "").unwrap().as_deref(), Some("idle"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sync_phase_default_is_idle_after_ensure() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
repo.ensure("s1", "").unwrap();
|
||||
assert_eq!(repo.get_sync_phase("s1", "").unwrap().as_deref(), Some("idle"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sync_phase_transitions_through_state_machine_values() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
for phase in ["probing", "initial_sync", "ready", "error", "idle"] {
|
||||
repo.set_sync_phase("s1", "", phase).unwrap();
|
||||
assert_eq!(
|
||||
repo.get_sync_phase("s1", "").unwrap().as_deref(),
|
||||
Some(phase)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn watermark_setters_preserve_each_other() {
|
||||
// Each setter must scope its `ON CONFLICT … DO UPDATE` to its own
|
||||
// column. Set three and read all three back unchanged.
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
repo.set_server_last_scan_iso("s1", "", Some("2026-05-01T12:00:00Z")).unwrap();
|
||||
repo.set_indexes_last_modified_ms("s1", "", 1_700_000_000_000).unwrap();
|
||||
repo.set_artists_last_modified_ms("s1", "", 1_700_000_500_000).unwrap();
|
||||
|
||||
let (iso, idx_ms, art_ms): (Option<String>, Option<i64>, Option<i64>) = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT server_last_scan_iso, indexes_last_modified_ms, artists_last_modified_ms \
|
||||
FROM sync_state WHERE server_id = 's1'",
|
||||
[],
|
||||
|r| Ok((r.get(0)?, r.get(1)?, r.get(2)?)),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(iso.as_deref(), Some("2026-05-01T12:00:00Z"));
|
||||
assert_eq!(idx_ms, Some(1_700_000_000_000));
|
||||
assert_eq!(art_ms, Some(1_700_000_500_000));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn library_tier_roundtrip() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
repo.set_library_tier("s1", "", "huge").unwrap();
|
||||
let tier: String = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT library_tier FROM sync_state WHERE server_id = 's1'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(tier, "huge");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn n1_bulk_unreliable_defaults_false_and_roundtrips() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
repo.ensure("s1", "").unwrap();
|
||||
// DEFAULT 0 reads back as Some(false); existing servers stay N1-eligible.
|
||||
assert_eq!(repo.get_n1_bulk_unreliable("s1", "").unwrap(), Some(false));
|
||||
assert_eq!(repo.get_n1_bulk_unreliable("absent", "").unwrap(), None);
|
||||
|
||||
repo.set_n1_bulk_unreliable("s1", "", true).unwrap();
|
||||
assert_eq!(repo.get_n1_bulk_unreliable("s1", "").unwrap(), Some(true));
|
||||
repo.set_n1_bulk_unreliable("s1", "", false).unwrap();
|
||||
assert_eq!(repo.get_n1_bulk_unreliable("s1", "").unwrap(), Some(false));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn n1_bulk_unreliable_set_does_not_clobber_cursor() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
repo.set_initial_sync_cursor("s1", "", &json!({"offset": 7})).unwrap();
|
||||
repo.set_n1_bulk_unreliable("s1", "", true).unwrap();
|
||||
assert_eq!(
|
||||
repo.get_initial_sync_cursor("s1", "").unwrap(),
|
||||
Some(json!({"offset": 7}))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn capability_flags_set_does_not_clobber_cursor() {
|
||||
// Cross-check: setting flags must not reset
|
||||
// `initial_sync_cursor_json` back to '{}'.
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = SyncStateRepository::new(&store);
|
||||
repo.set_initial_sync_cursor("s1", "", &json!({"offset": 42})).unwrap();
|
||||
repo.set_capability_flags("s1", "", 0x002).unwrap();
|
||||
assert_eq!(
|
||||
repo.get_initial_sync_cursor("s1", "").unwrap(),
|
||||
Some(json!({"offset": 42}))
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,980 @@
|
||||
use rusqlite::{params, OptionalExtension};
|
||||
|
||||
use crate::store::{LibraryStore, WriteOpTiming};
|
||||
|
||||
/// One row of the `track` table — every hot column from spec §5.1 plus
|
||||
/// `raw_json` (the full normalized SubsonicSong). Sync code (PR-2/PR-3) is
|
||||
/// expected to project ingested payloads into this shape, not to talk SQL
|
||||
/// directly.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct TrackRow {
|
||||
pub server_id: String,
|
||||
pub id: String,
|
||||
pub title: String,
|
||||
pub title_sort: Option<String>,
|
||||
pub artist: Option<String>,
|
||||
pub artist_id: Option<String>,
|
||||
pub album: String,
|
||||
pub album_id: Option<String>,
|
||||
pub album_artist: Option<String>,
|
||||
pub duration_sec: i64,
|
||||
pub track_number: Option<i64>,
|
||||
pub disc_number: Option<i64>,
|
||||
pub year: Option<i64>,
|
||||
pub genre: Option<String>,
|
||||
pub suffix: Option<String>,
|
||||
pub bit_rate: Option<i64>,
|
||||
pub size_bytes: Option<i64>,
|
||||
pub cover_art_id: Option<String>,
|
||||
pub starred_at: Option<i64>,
|
||||
pub user_rating: Option<i64>,
|
||||
pub play_count: Option<i64>,
|
||||
pub played_at: Option<i64>,
|
||||
pub server_path: Option<String>,
|
||||
pub library_id: Option<String>,
|
||||
pub isrc: Option<String>,
|
||||
pub mbid_recording: Option<String>,
|
||||
pub bpm: Option<i64>,
|
||||
pub replay_gain_track_db: Option<f64>,
|
||||
pub replay_gain_album_db: Option<f64>,
|
||||
pub content_hash: Option<String>,
|
||||
pub server_updated_at: Option<i64>,
|
||||
pub server_created_at: Option<i64>,
|
||||
pub deleted: bool,
|
||||
pub synced_at: i64,
|
||||
pub raw_json: String,
|
||||
}
|
||||
|
||||
/// One detected remap during an upsert batch. Sync code can use this
|
||||
/// to emit `library:tracks-changed { remapped: [{from, to}] }` (spec
|
||||
/// §6.9) so the UI can refresh open per-track views.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct RemapEntry {
|
||||
pub server_id: String,
|
||||
pub old_id: String,
|
||||
pub new_id: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct RemapStats {
|
||||
pub remapped: Vec<RemapEntry>,
|
||||
}
|
||||
|
||||
pub struct TrackRepository<'a> {
|
||||
store: &'a LibraryStore,
|
||||
}
|
||||
|
||||
impl<'a> TrackRepository<'a> {
|
||||
pub fn new(store: &'a LibraryStore) -> Self {
|
||||
Self { store }
|
||||
}
|
||||
|
||||
/// Batch upsert without remap detection. Suitable for generic
|
||||
/// Subsonic servers where `UnstableTrackIds` is clear (track ids
|
||||
/// are stable across reindexing). Wrapped in a single transaction.
|
||||
pub fn upsert_batch(&self, rows: &[TrackRow]) -> Result<(), String> {
|
||||
self.upsert_batch_with_remap(rows, false).map(|_| ())
|
||||
}
|
||||
|
||||
/// IS-3 initial-sync fast path: upsert rows only. Skips §6.9 remap
|
||||
/// detection and inline canonical linking — both run on delta sync
|
||||
/// or in a post-ingest canonical pass so 500-row batches stay fast.
|
||||
pub fn upsert_batch_initial_ingest(&self, rows: &[TrackRow]) -> Result<(), String> {
|
||||
self.upsert_batch_initial_ingest_timed(rows).map(|_| ())
|
||||
}
|
||||
|
||||
pub fn upsert_batch_initial_ingest_timed(
|
||||
&self,
|
||||
rows: &[TrackRow],
|
||||
) -> Result<WriteOpTiming, String> {
|
||||
if rows.is_empty() {
|
||||
return Ok(WriteOpTiming::default());
|
||||
}
|
||||
let (_, timing) = self.store.with_conn_mut_timed("track.upsert_initial_ingest", |conn| {
|
||||
let tx = conn.transaction()?;
|
||||
let mut upsert = tx.prepare_cached(UPSERT_SQL)?;
|
||||
for r in rows {
|
||||
upsert.execute(params![
|
||||
r.server_id,
|
||||
r.id,
|
||||
r.title,
|
||||
r.title_sort,
|
||||
r.artist,
|
||||
r.artist_id,
|
||||
r.album,
|
||||
r.album_id,
|
||||
r.album_artist,
|
||||
r.duration_sec,
|
||||
r.track_number,
|
||||
r.disc_number,
|
||||
r.year,
|
||||
r.genre,
|
||||
r.suffix,
|
||||
r.bit_rate,
|
||||
r.size_bytes,
|
||||
r.cover_art_id,
|
||||
r.starred_at,
|
||||
r.user_rating,
|
||||
r.play_count,
|
||||
r.played_at,
|
||||
r.server_path,
|
||||
r.library_id,
|
||||
r.isrc,
|
||||
r.mbid_recording,
|
||||
r.bpm,
|
||||
r.replay_gain_track_db,
|
||||
r.replay_gain_album_db,
|
||||
r.content_hash,
|
||||
r.server_updated_at,
|
||||
r.server_created_at,
|
||||
if r.deleted { 1_i64 } else { 0 },
|
||||
r.synced_at,
|
||||
r.raw_json,
|
||||
])?;
|
||||
}
|
||||
drop(upsert);
|
||||
tx.commit()?;
|
||||
Ok(())
|
||||
})?;
|
||||
Ok(timing)
|
||||
}
|
||||
|
||||
/// SELECT a single track by `(server_id, id)`. Returns `None`
|
||||
/// when missing or deleted (`deleted = 1`). Used by
|
||||
/// `library_get_track` and the offline-path command.
|
||||
pub fn find_one(
|
||||
&self,
|
||||
server_id: &str,
|
||||
track_id: &str,
|
||||
) -> Result<Option<TrackRow>, String> {
|
||||
self.store.with_read_conn(|conn| {
|
||||
let mut stmt = conn.prepare(SELECT_TRACK_BY_ID)?;
|
||||
stmt.query_row(params![server_id, track_id], row_to_track_row)
|
||||
.optional()
|
||||
})
|
||||
}
|
||||
|
||||
/// Batch SELECT — `library_get_tracks_batch`. Caller-supplied refs
|
||||
/// preserve their order in the result; unknown / deleted refs
|
||||
/// are silently dropped (frontend reads `tracks.length` against
|
||||
/// `refs.length` to detect partial responses).
|
||||
pub fn find_batch(
|
||||
&self,
|
||||
refs: &[(String, String)],
|
||||
) -> Result<Vec<TrackRow>, String> {
|
||||
if refs.is_empty() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
self.store.with_read_conn(|conn| {
|
||||
let mut stmt = conn.prepare(SELECT_TRACK_BY_ID)?;
|
||||
let mut out: Vec<TrackRow> = Vec::with_capacity(refs.len());
|
||||
for (server_id, track_id) in refs {
|
||||
if let Some(row) = stmt
|
||||
.query_row(params![server_id, track_id], row_to_track_row)
|
||||
.optional()?
|
||||
{
|
||||
out.push(row);
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
})
|
||||
}
|
||||
|
||||
/// SELECT every non-deleted track on this album, ordered by
|
||||
/// `disc_number ASC, track_number ASC` for stable display.
|
||||
pub fn find_by_album(
|
||||
&self,
|
||||
server_id: &str,
|
||||
album_id: &str,
|
||||
) -> Result<Vec<TrackRow>, String> {
|
||||
self.store.with_read_conn(|conn| {
|
||||
let mut stmt = conn.prepare(SELECT_TRACKS_BY_ALBUM)?;
|
||||
let rows: rusqlite::Result<Vec<TrackRow>> = stmt
|
||||
.query_map(params![server_id, album_id], row_to_track_row)?
|
||||
.collect();
|
||||
rows
|
||||
})
|
||||
}
|
||||
|
||||
/// Batch upsert with optional §6.9 id-remap detection. When
|
||||
/// `unstable_track_ids` is `true`, each incoming row is checked
|
||||
/// against the existing `track` table for a collision via
|
||||
/// `content_hash` or `server_path` carrying a different id. On
|
||||
/// collision, child tables (`track_offline` and the FK-bound
|
||||
/// extension / fact / artifact / canonical_link tables) are
|
||||
/// retargeted onto the new id, a `track_id_history` row is
|
||||
/// recorded, and the old `track` row is deleted — all inside the
|
||||
/// same SQLite transaction so partial remaps can't leak.
|
||||
pub fn upsert_batch_with_remap(
|
||||
&self,
|
||||
rows: &[TrackRow],
|
||||
unstable_track_ids: bool,
|
||||
) -> Result<RemapStats, String> {
|
||||
if rows.is_empty() {
|
||||
return Ok(RemapStats::default());
|
||||
}
|
||||
self.store.with_conn_mut("misc", |conn| {
|
||||
let tx = conn.transaction()?;
|
||||
let mut remapped: Vec<RemapEntry> = Vec::new();
|
||||
let mut upsert = tx.prepare_cached(UPSERT_SQL)?;
|
||||
let mut remap_lookup = if unstable_track_ids {
|
||||
Some(tx.prepare_cached(REMAP_LOOKUP_SQL)?)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
for r in rows {
|
||||
// Spec §6.9: detect collision BEFORE the upsert so the
|
||||
// old id is known. The upsert itself comes next; only
|
||||
// then do we retarget children to the new id, since
|
||||
// child tables FK→track(server_id, id) and would refuse
|
||||
// an UPDATE pointing at an id that doesn't exist yet.
|
||||
let detected_old: Option<String> = if let Some(ref mut lookup) = remap_lookup {
|
||||
detect_remap_target_cached(lookup, r)?
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
upsert.execute(params![
|
||||
r.server_id,
|
||||
r.id,
|
||||
r.title,
|
||||
r.title_sort,
|
||||
r.artist,
|
||||
r.artist_id,
|
||||
r.album,
|
||||
r.album_id,
|
||||
r.album_artist,
|
||||
r.duration_sec,
|
||||
r.track_number,
|
||||
r.disc_number,
|
||||
r.year,
|
||||
r.genre,
|
||||
r.suffix,
|
||||
r.bit_rate,
|
||||
r.size_bytes,
|
||||
r.cover_art_id,
|
||||
r.starred_at,
|
||||
r.user_rating,
|
||||
r.play_count,
|
||||
r.played_at,
|
||||
r.server_path,
|
||||
r.library_id,
|
||||
r.isrc,
|
||||
r.mbid_recording,
|
||||
r.bpm,
|
||||
r.replay_gain_track_db,
|
||||
r.replay_gain_album_db,
|
||||
r.content_hash,
|
||||
r.server_updated_at,
|
||||
r.server_created_at,
|
||||
if r.deleted { 1_i64 } else { 0 },
|
||||
r.synced_at,
|
||||
r.raw_json,
|
||||
])?;
|
||||
|
||||
if let Some(old_id) = detected_old {
|
||||
remap_existing_to_new(
|
||||
&tx,
|
||||
&r.server_id,
|
||||
&old_id,
|
||||
&r.id,
|
||||
r.content_hash.as_deref(),
|
||||
r.server_path.as_deref(),
|
||||
r.synced_at,
|
||||
)?;
|
||||
remapped.push(RemapEntry {
|
||||
server_id: r.server_id.clone(),
|
||||
old_id,
|
||||
new_id: r.id.clone(),
|
||||
});
|
||||
}
|
||||
|
||||
// H2 (§5.5A): link this track to its canonical id by its
|
||||
// strong key (ISRC, else MBID recording). Inline + O(1);
|
||||
// a no-op for tracks that carry neither.
|
||||
crate::canonical::link_track(
|
||||
&tx,
|
||||
&r.server_id,
|
||||
&r.id,
|
||||
r.isrc.as_deref(),
|
||||
r.mbid_recording.as_deref(),
|
||||
r.synced_at,
|
||||
)?;
|
||||
}
|
||||
|
||||
drop(upsert);
|
||||
drop(remap_lookup);
|
||||
|
||||
tx.commit()?;
|
||||
Ok(RemapStats { remapped })
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
const REMAP_LOOKUP_SQL: &str = r#"
|
||||
SELECT id FROM track
|
||||
WHERE server_id = ?1
|
||||
AND deleted = 0
|
||||
AND id != ?2
|
||||
AND (
|
||||
(?3 IS NOT NULL AND content_hash = ?3)
|
||||
OR (?4 IS NOT NULL AND server_path = ?4)
|
||||
)
|
||||
LIMIT 1
|
||||
"#;
|
||||
|
||||
/// Run the `SELECT old.id` half of §6.9 — returns `Some(old_id)` if a
|
||||
/// non-deleted row with a different id on this server matches the
|
||||
/// incoming row's `content_hash` or `server_path`.
|
||||
fn detect_remap_target_cached(
|
||||
lookup: &mut rusqlite::Statement<'_>,
|
||||
incoming: &TrackRow,
|
||||
) -> rusqlite::Result<Option<String>> {
|
||||
// Empty-string sentinels are *not* eligible — spec §6.9 explicitly
|
||||
// excludes them so the file-tree default never collides.
|
||||
let hash = incoming.content_hash.as_deref().filter(|s| !s.is_empty());
|
||||
let path = incoming.server_path.as_deref().filter(|s| !s.is_empty());
|
||||
if hash.is_none() && path.is_none() {
|
||||
return Ok(None);
|
||||
}
|
||||
lookup
|
||||
.query_row(
|
||||
params![incoming.server_id, incoming.id, hash, path],
|
||||
|row| row.get::<_, String>(0),
|
||||
)
|
||||
.optional()
|
||||
}
|
||||
|
||||
/// Run the §6.9 retarget half — UPDATE every FK-bound child to the
|
||||
/// new id, INSERT into `track_id_history`, DELETE the old `track` row.
|
||||
/// `track_offline` has no FK to `track` (spec §5.14) but still needs
|
||||
/// its row retargeted so the cached file resolves under the new id.
|
||||
fn remap_existing_to_new(
|
||||
tx: &rusqlite::Transaction<'_>,
|
||||
server_id: &str,
|
||||
old_id: &str,
|
||||
new_id: &str,
|
||||
content_hash: Option<&str>,
|
||||
server_path: Option<&str>,
|
||||
remapped_at: i64,
|
||||
) -> rusqlite::Result<()> {
|
||||
for table in [
|
||||
"track_offline",
|
||||
"track_extension",
|
||||
"track_fact",
|
||||
"track_artifact",
|
||||
"track_canonical_link",
|
||||
] {
|
||||
tx.execute(
|
||||
&format!(
|
||||
"UPDATE {table} SET track_id = ?1 \
|
||||
WHERE server_id = ?2 AND track_id = ?3"
|
||||
),
|
||||
params![new_id, server_id, old_id],
|
||||
)?;
|
||||
}
|
||||
tx.execute(
|
||||
"INSERT INTO track_id_history \
|
||||
(server_id, old_id, new_id, content_hash, server_path, remapped_at) \
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6) \
|
||||
ON CONFLICT(server_id, old_id) DO UPDATE SET \
|
||||
new_id = excluded.new_id, \
|
||||
content_hash = excluded.content_hash, \
|
||||
server_path = excluded.server_path, \
|
||||
remapped_at = excluded.remapped_at",
|
||||
params![server_id, old_id, new_id, content_hash, server_path, remapped_at],
|
||||
)?;
|
||||
tx.execute(
|
||||
"DELETE FROM track WHERE server_id = ?1 AND id = ?2",
|
||||
params![server_id, old_id],
|
||||
)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Column list mirroring the `track` schema (§5.1) — used by every
|
||||
/// `SELECT … FROM track` so the row-mapper can index by position.
|
||||
const TRACK_COLUMNS: &str = "\
|
||||
server_id, id, title, title_sort, artist, artist_id, album, album_id, \
|
||||
album_artist, duration_sec, track_number, disc_number, year, genre, suffix, \
|
||||
bit_rate, size_bytes, cover_art_id, starred_at, user_rating, play_count, \
|
||||
played_at, server_path, library_id, isrc, mbid_recording, bpm, \
|
||||
replay_gain_track_db, replay_gain_album_db, content_hash, server_updated_at, \
|
||||
server_created_at, deleted, synced_at, raw_json";
|
||||
|
||||
const SELECT_TRACK_BY_ID: &str = "SELECT server_id, id, title, title_sort, artist, artist_id, \
|
||||
album, album_id, album_artist, duration_sec, track_number, disc_number, year, genre, suffix, \
|
||||
bit_rate, size_bytes, cover_art_id, starred_at, user_rating, play_count, played_at, \
|
||||
server_path, library_id, isrc, mbid_recording, bpm, replay_gain_track_db, replay_gain_album_db, \
|
||||
content_hash, server_updated_at, server_created_at, deleted, synced_at, raw_json \
|
||||
FROM track WHERE server_id = ?1 AND id = ?2 AND deleted = 0";
|
||||
|
||||
const SELECT_TRACKS_BY_ALBUM: &str = "SELECT server_id, id, title, title_sort, artist, artist_id, \
|
||||
album, album_id, album_artist, duration_sec, track_number, disc_number, year, genre, suffix, \
|
||||
bit_rate, size_bytes, cover_art_id, starred_at, user_rating, play_count, played_at, \
|
||||
server_path, library_id, isrc, mbid_recording, bpm, replay_gain_track_db, replay_gain_album_db, \
|
||||
content_hash, server_updated_at, server_created_at, deleted, synced_at, raw_json \
|
||||
FROM track WHERE server_id = ?1 AND album_id = ?2 AND deleted = 0 \
|
||||
ORDER BY disc_number ASC NULLS LAST, track_number ASC NULLS LAST, id ASC";
|
||||
|
||||
pub(crate) fn track_columns() -> &'static str {
|
||||
TRACK_COLUMNS
|
||||
}
|
||||
|
||||
pub(crate) fn row_to_track_row(row: &rusqlite::Row<'_>) -> rusqlite::Result<TrackRow> {
|
||||
Ok(TrackRow {
|
||||
server_id: row.get(0)?,
|
||||
id: row.get(1)?,
|
||||
title: row.get(2)?,
|
||||
title_sort: row.get(3)?,
|
||||
artist: row.get(4)?,
|
||||
artist_id: row.get(5)?,
|
||||
album: row.get(6)?,
|
||||
album_id: row.get(7)?,
|
||||
album_artist: row.get(8)?,
|
||||
duration_sec: row.get(9)?,
|
||||
track_number: row.get(10)?,
|
||||
disc_number: row.get(11)?,
|
||||
year: row.get(12)?,
|
||||
genre: row.get(13)?,
|
||||
suffix: row.get(14)?,
|
||||
bit_rate: row.get(15)?,
|
||||
size_bytes: row.get(16)?,
|
||||
cover_art_id: row.get(17)?,
|
||||
starred_at: row.get(18)?,
|
||||
user_rating: row.get(19)?,
|
||||
play_count: row.get(20)?,
|
||||
played_at: row.get(21)?,
|
||||
server_path: row.get(22)?,
|
||||
library_id: row.get(23)?,
|
||||
isrc: row.get(24)?,
|
||||
mbid_recording: row.get(25)?,
|
||||
bpm: row.get(26)?,
|
||||
replay_gain_track_db: row.get(27)?,
|
||||
replay_gain_album_db: row.get(28)?,
|
||||
content_hash: row.get(29)?,
|
||||
server_updated_at: row.get(30)?,
|
||||
server_created_at: row.get(31)?,
|
||||
deleted: row.get::<_, i64>(32)? != 0,
|
||||
synced_at: row.get(33)?,
|
||||
raw_json: row.get(34)?,
|
||||
})
|
||||
}
|
||||
|
||||
const UPSERT_SQL: &str = r#"
|
||||
INSERT INTO track (
|
||||
server_id, id, title, title_sort, artist, artist_id, album, album_id,
|
||||
album_artist, duration_sec, track_number, disc_number, year, genre, suffix,
|
||||
bit_rate, size_bytes, cover_art_id, starred_at, user_rating, play_count,
|
||||
played_at, server_path, library_id, isrc, mbid_recording, bpm,
|
||||
replay_gain_track_db, replay_gain_album_db, content_hash, server_updated_at,
|
||||
server_created_at, deleted, synced_at, raw_json
|
||||
) VALUES (
|
||||
?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16, ?17,
|
||||
?18, ?19, ?20, ?21, ?22, ?23, ?24, ?25, ?26, ?27, ?28, ?29, ?30, ?31, ?32,
|
||||
?33, ?34, ?35
|
||||
)
|
||||
ON CONFLICT(server_id, id) DO UPDATE SET
|
||||
title = excluded.title,
|
||||
title_sort = excluded.title_sort,
|
||||
artist = excluded.artist,
|
||||
artist_id = excluded.artist_id,
|
||||
album = excluded.album,
|
||||
album_id = excluded.album_id,
|
||||
album_artist = excluded.album_artist,
|
||||
duration_sec = excluded.duration_sec,
|
||||
track_number = excluded.track_number,
|
||||
disc_number = excluded.disc_number,
|
||||
year = excluded.year,
|
||||
genre = excluded.genre,
|
||||
suffix = excluded.suffix,
|
||||
bit_rate = excluded.bit_rate,
|
||||
size_bytes = excluded.size_bytes,
|
||||
cover_art_id = excluded.cover_art_id,
|
||||
starred_at = excluded.starred_at,
|
||||
user_rating = excluded.user_rating,
|
||||
play_count = excluded.play_count,
|
||||
played_at = excluded.played_at,
|
||||
server_path = excluded.server_path,
|
||||
library_id = excluded.library_id,
|
||||
isrc = excluded.isrc,
|
||||
mbid_recording = excluded.mbid_recording,
|
||||
bpm = excluded.bpm,
|
||||
replay_gain_track_db = excluded.replay_gain_track_db,
|
||||
replay_gain_album_db = excluded.replay_gain_album_db,
|
||||
-- E2: never let a sync (which passes NULL content_hash) clobber the
|
||||
-- playback-derived md5_16kb written via library_patch_track / the analysis
|
||||
-- bridge. A non-empty incoming hash still wins.
|
||||
content_hash = COALESCE(NULLIF(excluded.content_hash, ''), track.content_hash),
|
||||
server_updated_at = excluded.server_updated_at,
|
||||
server_created_at = excluded.server_created_at,
|
||||
deleted = excluded.deleted,
|
||||
synced_at = excluded.synced_at,
|
||||
raw_json = excluded.raw_json
|
||||
"#;
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn row(server: &str, id: &str, title: &str) -> TrackRow {
|
||||
TrackRow {
|
||||
server_id: server.into(),
|
||||
id: id.into(),
|
||||
title: title.into(),
|
||||
title_sort: None,
|
||||
artist: Some("The Artist".into()),
|
||||
artist_id: Some("ar1".into()),
|
||||
album: "An Album".into(),
|
||||
album_id: Some("al1".into()),
|
||||
album_artist: Some("The Artist".into()),
|
||||
duration_sec: 240,
|
||||
track_number: Some(3),
|
||||
disc_number: Some(1),
|
||||
year: Some(2024),
|
||||
genre: Some("Ambient".into()),
|
||||
suffix: Some("flac".into()),
|
||||
bit_rate: Some(1000),
|
||||
size_bytes: Some(32_000_000),
|
||||
cover_art_id: Some("cv1".into()),
|
||||
starred_at: None,
|
||||
user_rating: None,
|
||||
play_count: Some(0),
|
||||
played_at: None,
|
||||
server_path: Some("Artist/Album/03.flac".into()),
|
||||
library_id: Some("lib-1".into()),
|
||||
isrc: None,
|
||||
mbid_recording: None,
|
||||
bpm: None,
|
||||
replay_gain_track_db: None,
|
||||
replay_gain_album_db: None,
|
||||
content_hash: Some("hash-abc".into()),
|
||||
server_updated_at: Some(1_700_000_000),
|
||||
server_created_at: Some(1_699_000_000),
|
||||
deleted: false,
|
||||
synced_at: 1_700_000_500,
|
||||
raw_json: r#"{"id":"t1"}"#.into(),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resync_does_not_clobber_playback_content_hash() {
|
||||
// E2 safety property: a sync (which passes content_hash = None) must
|
||||
// never wipe the playback-derived md5 written via patch / the bridge.
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
|
||||
let mut initial = row("s1", "t1", "First");
|
||||
initial.content_hash = None;
|
||||
repo.upsert_batch(&[initial]).unwrap();
|
||||
|
||||
// Playback records the content fingerprint.
|
||||
store
|
||||
.with_conn("misc", |c| {
|
||||
c.execute(
|
||||
"UPDATE track SET content_hash = 'playback-md5' WHERE server_id='s1' AND id='t1'",
|
||||
[],
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
|
||||
let read = |store: &LibraryStore| -> Option<String> {
|
||||
store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT content_hash FROM track WHERE server_id='s1' AND id='t1'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
})
|
||||
.unwrap()
|
||||
};
|
||||
|
||||
// Resync with a NULL hash preserves the playback value.
|
||||
let mut resync = row("s1", "t1", "First (resynced)");
|
||||
resync.content_hash = None;
|
||||
repo.upsert_batch(&[resync]).unwrap();
|
||||
assert_eq!(read(&store).as_deref(), Some("playback-md5"));
|
||||
|
||||
// A non-empty incoming hash still wins.
|
||||
let mut with_hash = row("s1", "t1", "First");
|
||||
with_hash.content_hash = Some("server-hash".into());
|
||||
repo.upsert_batch(&[with_hash]).unwrap();
|
||||
assert_eq!(read(&store).as_deref(), Some("server-hash"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn upsert_inserts_new_rows() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
repo.upsert_batch(&[row("s1", "t1", "First"), row("s1", "t2", "Second")])
|
||||
.unwrap();
|
||||
let count: i64 = store
|
||||
.with_conn("misc", |c| c.query_row("SELECT COUNT(*) FROM track", [], |r| r.get(0)))
|
||||
.unwrap();
|
||||
assert_eq!(count, 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn upsert_updates_existing_rows() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
repo.upsert_batch(&[row("s1", "t1", "Original")]).unwrap();
|
||||
|
||||
let mut updated = row("s1", "t1", "Updated");
|
||||
updated.bpm = Some(128);
|
||||
updated.starred_at = Some(1_700_000_999);
|
||||
repo.upsert_batch(&[updated]).unwrap();
|
||||
|
||||
let (title, bpm, starred): (String, Option<i64>, Option<i64>) = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT title, bpm, starred_at FROM track WHERE server_id='s1' AND id='t1'",
|
||||
[],
|
||||
|r| Ok((r.get(0)?, r.get(1)?, r.get(2)?)),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(title, "Updated");
|
||||
assert_eq!(bpm, Some(128));
|
||||
assert_eq!(starred, Some(1_700_000_999));
|
||||
|
||||
let count: i64 = store
|
||||
.with_conn("misc", |c| c.query_row("SELECT COUNT(*) FROM track", [], |r| r.get(0)))
|
||||
.unwrap();
|
||||
assert_eq!(count, 1, "upsert must not duplicate the row");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn upsert_empty_batch_is_noop() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
repo.upsert_batch(&[]).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn upsert_keeps_server_scope_separate() {
|
||||
// Same `id` on two different servers must produce two rows
|
||||
// (PRIMARY KEY is composite).
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
repo.upsert_batch(&[row("s1", "t1", "From S1"), row("s2", "t1", "From S2")])
|
||||
.unwrap();
|
||||
let count: i64 = store
|
||||
.with_conn("misc", |c| c.query_row("SELECT COUNT(*) FROM track", [], |r| r.get(0)))
|
||||
.unwrap();
|
||||
assert_eq!(count, 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn upsert_populates_fts_via_trigger() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
repo.upsert_batch(&[row("s1", "t1", "Aurora Boreal")]).unwrap();
|
||||
let fts_hit: i64 = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT COUNT(*) FROM track_fts WHERE track_fts MATCH 'aurora'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(fts_hit, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn upsert_update_refreshes_fts_via_trigger() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
repo.upsert_batch(&[row("s1", "t1", "Old Title")]).unwrap();
|
||||
repo.upsert_batch(&[row("s1", "t1", "Brand New Title")]).unwrap();
|
||||
|
||||
let old_hit: i64 = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT COUNT(*) FROM track_fts WHERE track_fts MATCH 'old'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
let new_hit: i64 = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT COUNT(*) FROM track_fts WHERE track_fts MATCH 'brand'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(old_hit, 0, "delete-trigger must drop the stale FTS row");
|
||||
assert_eq!(new_hit, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn initial_ingest_batch_skips_remap_and_canonical() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
let rows: Vec<TrackRow> = (0..500)
|
||||
.map(|i| {
|
||||
let mut r = row("s1", &format!("t{i:04}"), &format!("Track {i:04}"));
|
||||
r.server_path = Some(format!("/music/track{i:04}.flac"));
|
||||
r.isrc = Some(format!("USRC{i:06}"));
|
||||
r.raw_json = format!(r#"{{"id":"t{i:04}","payload":"#)
|
||||
+ &"x".repeat(512)
|
||||
+ r#""}"#;
|
||||
r
|
||||
})
|
||||
.collect();
|
||||
let start = std::time::Instant::now();
|
||||
repo.upsert_batch_initial_ingest(&rows).unwrap();
|
||||
let elapsed = start.elapsed();
|
||||
assert!(
|
||||
elapsed < std::time::Duration::from_millis(500),
|
||||
"initial ingest batch(500) took {elapsed:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn upsert_500_rows_completes_well_under_perf_budget() {
|
||||
// Spec §5.1 / AC A3: `upsert_batch` should land 500 rows under 100ms
|
||||
// typical. The CI threshold is 5× that to absorb slow runners and
|
||||
// the difference between debug and release; any regression past it
|
||||
// is real signal.
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
let rows: Vec<TrackRow> = (0..500)
|
||||
.map(|i| row("s1", &format!("t{i:04}"), &format!("Track {i:04}")))
|
||||
.collect();
|
||||
|
||||
let start = std::time::Instant::now();
|
||||
repo.upsert_batch(&rows).unwrap();
|
||||
let elapsed = start.elapsed();
|
||||
|
||||
let stored: i64 = store
|
||||
.with_conn("misc", |c| c.query_row("SELECT COUNT(*) FROM track", [], |r| r.get(0)))
|
||||
.unwrap();
|
||||
assert_eq!(stored, 500);
|
||||
|
||||
assert!(
|
||||
elapsed < std::time::Duration::from_millis(500),
|
||||
"upsert_batch(500 rows) took {elapsed:?}; AC A3 target is <100ms typical, \
|
||||
test fails past 5× that"
|
||||
);
|
||||
}
|
||||
|
||||
// ── PR-3b: §6.9 id remap detection ────────────────────────────────────
|
||||
|
||||
fn row_with_id_hash(server: &str, id: &str, hash: &str, path: &str) -> TrackRow {
|
||||
let mut r = row(server, id, "Title");
|
||||
r.content_hash = if hash.is_empty() { None } else { Some(hash.into()) };
|
||||
r.server_path = if path.is_empty() { None } else { Some(path.into()) };
|
||||
r
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn remap_disabled_never_records_history_even_on_hash_collision() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
repo.upsert_batch(&[row_with_id_hash("s1", "tr_old", "deadbeef", "")])
|
||||
.unwrap();
|
||||
|
||||
// Generic Subsonic path: caller passes `unstable_track_ids = false`.
|
||||
let stats = repo
|
||||
.upsert_batch_with_remap(
|
||||
&[row_with_id_hash("s1", "tr_new", "deadbeef", "")],
|
||||
false,
|
||||
)
|
||||
.unwrap();
|
||||
assert!(stats.remapped.is_empty());
|
||||
|
||||
let track_count: i64 = store
|
||||
.with_conn("misc", |c| c.query_row("SELECT COUNT(*) FROM track", [], |r| r.get(0)))
|
||||
.unwrap();
|
||||
let hist_count: i64 = store
|
||||
.with_conn("misc", |c| c.query_row("SELECT COUNT(*) FROM track_id_history", [], |r| r.get(0)))
|
||||
.unwrap();
|
||||
assert_eq!(track_count, 2, "both ids coexist when remap is off");
|
||||
assert_eq!(hist_count, 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn remap_via_content_hash_replaces_old_row_and_records_history() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
// Seed with the old id; child tables get a row each that must
|
||||
// follow the remap.
|
||||
repo.upsert_batch(&[row_with_id_hash("s1", "tr_old", "deadbeef", "/path/x.flac")])
|
||||
.unwrap();
|
||||
store
|
||||
.with_conn("misc", |c| {
|
||||
c.execute(
|
||||
"INSERT INTO track_offline \
|
||||
(server_id, track_id, local_path, cached_at) \
|
||||
VALUES ('s1', 'tr_old', '/local/x.flac', 1)",
|
||||
[],
|
||||
)?;
|
||||
c.execute(
|
||||
"INSERT INTO track_extension \
|
||||
(server_id, track_id, kind, payload, updated_at) \
|
||||
VALUES ('s1', 'tr_old', 'user_note', X'7B7D', 1)",
|
||||
[],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
.unwrap();
|
||||
|
||||
let stats = repo
|
||||
.upsert_batch_with_remap(
|
||||
&[row_with_id_hash("s1", "tr_new", "deadbeef", "/path/x.flac")],
|
||||
true,
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(stats.remapped.len(), 1);
|
||||
assert_eq!(stats.remapped[0].old_id, "tr_old");
|
||||
assert_eq!(stats.remapped[0].new_id, "tr_new");
|
||||
|
||||
// Old track row gone, new one in place.
|
||||
let ids: Vec<String> = store
|
||||
.with_conn("misc", |c| {
|
||||
let mut stmt = c.prepare("SELECT id FROM track WHERE server_id = 's1'")?;
|
||||
let r: rusqlite::Result<Vec<String>> = stmt.query_map([], |r| r.get(0))?.collect();
|
||||
r
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(ids, vec!["tr_new"]);
|
||||
|
||||
// Child tables follow the new id.
|
||||
let offline_id: String = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT track_id FROM track_offline WHERE server_id = 's1'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(offline_id, "tr_new");
|
||||
let ext_id: String = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT track_id FROM track_extension WHERE server_id = 's1'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(ext_id, "tr_new");
|
||||
|
||||
// History row recorded.
|
||||
let hist = crate::repos::TrackIdHistoryRepository::new(&store);
|
||||
assert_eq!(
|
||||
hist.lookup_new_id("s1", "tr_old").unwrap().as_deref(),
|
||||
Some("tr_new")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn remap_via_server_path_only_works_when_hash_missing() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
repo.upsert_batch(&[row_with_id_hash("s1", "tr_old", "", "/path/y.mp3")])
|
||||
.unwrap();
|
||||
// Server only ships server_path on the new row — no hash yet.
|
||||
let stats = repo
|
||||
.upsert_batch_with_remap(
|
||||
&[row_with_id_hash("s1", "tr_new", "", "/path/y.mp3")],
|
||||
true,
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(stats.remapped.len(), 1, "path-based remap must trigger");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn remap_skips_when_neither_hash_nor_path_present() {
|
||||
// Defensive: empty-string sentinels must not cause spurious
|
||||
// remaps across unrelated rows that happen to lack hash + path.
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
repo.upsert_batch(&[row_with_id_hash("s1", "tr_old", "", "")]).unwrap();
|
||||
let stats = repo
|
||||
.upsert_batch_with_remap(&[row_with_id_hash("s1", "tr_new", "", "")], true)
|
||||
.unwrap();
|
||||
assert!(stats.remapped.is_empty());
|
||||
let count: i64 = store
|
||||
.with_conn("misc", |c| c.query_row("SELECT COUNT(*) FROM track", [], |r| r.get(0)))
|
||||
.unwrap();
|
||||
assert_eq!(count, 2, "both rows kept; identity-less rows can't shadow");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn remap_is_noop_when_new_id_matches_existing_id() {
|
||||
// Standard delta-sync: same id, same hash. Must not trigger
|
||||
// remap (SELECT excludes id = T.id).
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
repo.upsert_batch(&[row_with_id_hash("s1", "tr_1", "h", "/p")]).unwrap();
|
||||
let stats = repo
|
||||
.upsert_batch_with_remap(&[row_with_id_hash("s1", "tr_1", "h", "/p")], true)
|
||||
.unwrap();
|
||||
assert!(stats.remapped.is_empty());
|
||||
}
|
||||
|
||||
// ── H2: canonical linking on the upsert path (§5.5A) ───────────────
|
||||
|
||||
#[test]
|
||||
fn upsert_links_track_to_canonical_by_isrc() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let mut r = row("s1", "t1", "Title");
|
||||
r.isrc = Some("USRC100".into());
|
||||
TrackRepository::new(&store).upsert_batch(&[r]).unwrap();
|
||||
let cid: Option<String> = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT canonical_id FROM track_canonical_link \
|
||||
WHERE server_id='s1' AND track_id='t1'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
.optional()
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(cid.as_deref(), Some("isrc:USRC100"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn upsert_shares_canonical_across_servers_with_same_isrc() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let mut a = row("s1", "t1", "T");
|
||||
a.isrc = Some("USRC200".into());
|
||||
let mut b = row("s2", "t9", "T");
|
||||
b.isrc = Some("USRC200".into());
|
||||
TrackRepository::new(&store).upsert_batch(&[a, b]).unwrap();
|
||||
let distinct: i64 = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT COUNT(DISTINCT canonical_id) FROM track_canonical_link",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(distinct, 1, "same ISRC on two servers → one canonical id");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn upsert_without_strong_key_creates_no_canonical_link() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
// `row(...)` leaves isrc / mbid_recording as None.
|
||||
TrackRepository::new(&store)
|
||||
.upsert_batch(&[row("s1", "t1", "T")])
|
||||
.unwrap();
|
||||
let count: i64 = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row("SELECT COUNT(*) FROM track_canonical_link", [], |r| r.get(0))
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(count, 0);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,123 @@
|
||||
//! Read-side helper for `track_id_history`. Writes live inside the
|
||||
//! upsert transaction in `TrackRepository::upsert_batch_with_remap`
|
||||
//! (spec §6.9) — keeping them there avoids splitting the remap across
|
||||
//! two SQLite transactions, which would leave a window where child
|
||||
//! tables point at a removed track id.
|
||||
|
||||
use rusqlite::{params, OptionalExtension};
|
||||
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
pub struct TrackIdHistoryRepository<'a> {
|
||||
store: &'a LibraryStore,
|
||||
}
|
||||
|
||||
impl<'a> TrackIdHistoryRepository<'a> {
|
||||
pub fn new(store: &'a LibraryStore) -> Self {
|
||||
Self { store }
|
||||
}
|
||||
|
||||
/// Resolve a remapped id forward to the current server id, if a
|
||||
/// remap was recorded. Returns `None` when no row exists. Analysis
|
||||
/// cache lookups (Phase E) go through this so cached waveform /
|
||||
/// loudness rows stay reachable after the server's id space shifts.
|
||||
pub fn lookup_new_id(
|
||||
&self,
|
||||
server_id: &str,
|
||||
old_id: &str,
|
||||
) -> Result<Option<String>, String> {
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.query_row(
|
||||
"SELECT new_id FROM track_id_history \
|
||||
WHERE server_id = ?1 AND old_id = ?2",
|
||||
params![server_id, old_id],
|
||||
|row| row.get::<_, String>(0),
|
||||
)
|
||||
.optional()
|
||||
})
|
||||
}
|
||||
|
||||
/// Count the rows recorded for this server — used by tests and by
|
||||
/// post-sync diagnostics (Settings „Library index" panel later).
|
||||
pub fn count_for_server(&self, server_id: &str) -> Result<i64, String> {
|
||||
self.store.with_conn("misc", |conn| {
|
||||
conn.query_row(
|
||||
"SELECT COUNT(*) FROM track_id_history WHERE server_id = ?1",
|
||||
params![server_id],
|
||||
|row| row.get(0),
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use rusqlite::params;
|
||||
|
||||
fn seed_history(store: &LibraryStore, rows: &[(&str, &str, &str)]) {
|
||||
store
|
||||
.with_conn("misc", |c| {
|
||||
for (server, old, new) in rows {
|
||||
c.execute(
|
||||
"INSERT INTO track_id_history \
|
||||
(server_id, old_id, new_id, content_hash, server_path, remapped_at) \
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6)",
|
||||
params![server, old, new, "hash", "path", 1_700_000_000_i64],
|
||||
)?;
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lookup_returns_none_when_no_remap_recorded() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackIdHistoryRepository::new(&store);
|
||||
assert_eq!(repo.lookup_new_id("s1", "tr_old").unwrap(), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lookup_finds_new_id_for_recorded_remap() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_history(&store, &[("s1", "tr_old", "tr_new")]);
|
||||
let repo = TrackIdHistoryRepository::new(&store);
|
||||
assert_eq!(
|
||||
repo.lookup_new_id("s1", "tr_old").unwrap().as_deref(),
|
||||
Some("tr_new")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lookup_scopes_by_server_id() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_history(&store, &[("s1", "tr_x", "tr_new1"), ("s2", "tr_x", "tr_new2")]);
|
||||
let repo = TrackIdHistoryRepository::new(&store);
|
||||
assert_eq!(
|
||||
repo.lookup_new_id("s1", "tr_x").unwrap().as_deref(),
|
||||
Some("tr_new1")
|
||||
);
|
||||
assert_eq!(
|
||||
repo.lookup_new_id("s2", "tr_x").unwrap().as_deref(),
|
||||
Some("tr_new2")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn count_for_server_scopes_correctly() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_history(
|
||||
&store,
|
||||
&[
|
||||
("s1", "a", "b"),
|
||||
("s1", "c", "d"),
|
||||
("s2", "e", "f"),
|
||||
],
|
||||
);
|
||||
let repo = TrackIdHistoryRepository::new(&store);
|
||||
assert_eq!(repo.count_for_server("s1").unwrap(), 2);
|
||||
assert_eq!(repo.count_for_server("s2").unwrap(), 1);
|
||||
assert_eq!(repo.count_for_server("absent").unwrap(), 0);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,258 @@
|
||||
//! `LibraryRuntime` — Tauri State shared by every library command.
|
||||
//!
|
||||
//! PR-5a held only the store. PR-5b extends with the per-server sync
|
||||
//! session map (credentials live in process memory only — same trust
|
||||
//! boundary as today's WebView-held passwords), the current playback
|
||||
//! hint, an `Option<SyncSupervisor>` for in-flight start/cancel, and
|
||||
//! a long-lived cancellation flag for the background-scheduler task
|
||||
//! the top crate spawns in `setup()`.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use tokio::sync::Notify;
|
||||
|
||||
use crate::store::LibraryStore;
|
||||
use crate::sync::bandwidth::PlaybackHint;
|
||||
|
||||
/// Per-server credentials cache for the sync runner. Lives only in
|
||||
/// `LibraryRuntime` process memory; `library_sync_clear_session`
|
||||
/// removes it on logout / index disable / purge.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SyncSession {
|
||||
pub server_id: String,
|
||||
pub base_url: String,
|
||||
pub username: String,
|
||||
pub password: String,
|
||||
/// Navidrome native API bearer cached from the `/auth/login`
|
||||
/// response at bind time. `None` when the server isn't Navidrome
|
||||
/// or the optional Navidrome auth failed (Subsonic-only path).
|
||||
pub navidrome_token: Option<String>,
|
||||
pub library_scope: Option<String>,
|
||||
}
|
||||
|
||||
/// Currently-running initial / delta / manual integrity job
|
||||
/// metadata. Holding the `SyncSupervisor` in the mutex (as the
|
||||
/// PR-5 kickoff sketch suggested) would block `library_sync_cancel`
|
||||
/// behind whoever's running the supervisor's join — instead we keep
|
||||
/// just the cancel handle + identity, and the job-orchestrator task
|
||||
/// owns the supervisor / receiver / join.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CurrentJob {
|
||||
pub job_id: String,
|
||||
pub server_id: String,
|
||||
/// `"initial_sync"` or `"delta_sync"`.
|
||||
pub kind: String,
|
||||
pub cancel: Arc<AtomicBool>,
|
||||
/// Signaled when this job's runner task finishes (success, error, or cancel).
|
||||
pub done: Arc<Notify>,
|
||||
}
|
||||
|
||||
pub struct LibraryRuntime {
|
||||
pub store: Arc<LibraryStore>,
|
||||
/// Per-`server_id` sync session. Mutex over a `HashMap` — single
|
||||
/// writer at a time is fine for the command surface; the
|
||||
/// background scheduler tick reads a snapshot.
|
||||
pub sync_sessions: Mutex<HashMap<String, SyncSession>>,
|
||||
pub playback_hint: Mutex<PlaybackHint>,
|
||||
/// Currently running initial / delta / manual integrity job, if
|
||||
/// any. `library_sync_start` populates, `library_sync_cancel`
|
||||
/// trips `cancel`; the orchestrator task clears the slot when
|
||||
/// the job's `join` returns.
|
||||
pub current_job: Mutex<Option<CurrentJob>>,
|
||||
/// Top-crate scheduler tick task watches this flag; set true on
|
||||
/// app shutdown / library index disabled.
|
||||
pub scheduler_cancel: Arc<AtomicBool>,
|
||||
/// Latest `library_live_search` epoch from the UI — stale commands
|
||||
/// skip FTS when a newer keystroke generation was registered.
|
||||
live_search_epoch: AtomicU64,
|
||||
}
|
||||
|
||||
impl LibraryRuntime {
|
||||
pub fn new(store: Arc<LibraryStore>) -> Self {
|
||||
Self {
|
||||
store,
|
||||
sync_sessions: Mutex::new(HashMap::new()),
|
||||
playback_hint: Mutex::new(PlaybackHint::default()),
|
||||
current_job: Mutex::new(None),
|
||||
scheduler_cancel: Arc::new(AtomicBool::new(false)),
|
||||
live_search_epoch: AtomicU64::new(0),
|
||||
}
|
||||
}
|
||||
|
||||
/// UI bumps `epoch` on every debounced search start / cancel.
|
||||
pub fn register_live_search_epoch(&self, epoch: u64) {
|
||||
let _ = self.live_search_epoch.fetch_max(epoch, Ordering::SeqCst);
|
||||
}
|
||||
|
||||
pub fn live_search_still_current(&self, epoch: u64) -> bool {
|
||||
self.live_search_epoch.load(Ordering::Acquire) == epoch
|
||||
}
|
||||
|
||||
pub fn set_current_job(&self, job: CurrentJob) {
|
||||
if let Ok(mut slot) = self.current_job.lock() {
|
||||
// Best-effort cancel any in-flight job before we replace.
|
||||
if let Some(prev) = slot.as_ref() {
|
||||
prev.cancel.store(true, std::sync::atomic::Ordering::SeqCst);
|
||||
}
|
||||
*slot = Some(job);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn current_job(&self) -> Option<CurrentJob> {
|
||||
self.current_job.lock().ok().and_then(|s| s.clone())
|
||||
}
|
||||
|
||||
pub fn clear_current_job_if_matches(&self, job_id: &str) {
|
||||
if let Ok(mut slot) = self.current_job.lock() {
|
||||
if slot.as_ref().is_some_and(|j| j.job_id == job_id) {
|
||||
*slot = None;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn cancel_current_job(&self) -> bool {
|
||||
if let Ok(slot) = self.current_job.lock() {
|
||||
if let Some(job) = slot.as_ref() {
|
||||
job.cancel.store(true, std::sync::atomic::Ordering::SeqCst);
|
||||
return true;
|
||||
}
|
||||
}
|
||||
false
|
||||
}
|
||||
|
||||
/// Snapshot all bound sessions — used by the scheduler tick task
|
||||
/// in the top crate so it doesn't hold the mutex across an `await`.
|
||||
pub fn snapshot_sessions(&self) -> Vec<SyncSession> {
|
||||
self.sync_sessions
|
||||
.lock()
|
||||
.map(|sessions| sessions.values().cloned().collect())
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
pub fn get_session(&self, server_id: &str) -> Option<SyncSession> {
|
||||
self.sync_sessions
|
||||
.lock()
|
||||
.ok()
|
||||
.and_then(|s| s.get(server_id).cloned())
|
||||
}
|
||||
|
||||
pub fn set_session(&self, session: SyncSession) {
|
||||
if let Ok(mut sessions) = self.sync_sessions.lock() {
|
||||
sessions.insert(session.server_id.clone(), session);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn clear_session(&self, server_id: &str) {
|
||||
if let Ok(mut sessions) = self.sync_sessions.lock() {
|
||||
sessions.remove(server_id);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn current_playback_hint(&self) -> PlaybackHint {
|
||||
self.playback_hint
|
||||
.lock()
|
||||
.map(|h| *h)
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
pub fn set_playback_hint(&self, hint: PlaybackHint) {
|
||||
if let Ok(mut h) = self.playback_hint.lock() {
|
||||
*h = hint;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn sample_session(server_id: &str) -> SyncSession {
|
||||
SyncSession {
|
||||
server_id: server_id.into(),
|
||||
base_url: "https://nas.example.com".into(),
|
||||
username: "u".into(),
|
||||
password: "p".into(),
|
||||
navidrome_token: None,
|
||||
library_scope: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn new_runtime_has_empty_sessions_and_idle_hint() {
|
||||
let store = Arc::new(LibraryStore::open_in_memory());
|
||||
let rt = LibraryRuntime::new(store);
|
||||
assert!(rt.snapshot_sessions().is_empty());
|
||||
assert_eq!(rt.current_playback_hint(), PlaybackHint::Idle);
|
||||
assert!(!rt
|
||||
.scheduler_cancel
|
||||
.load(std::sync::atomic::Ordering::SeqCst));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn set_and_get_session_roundtrip() {
|
||||
let store = Arc::new(LibraryStore::open_in_memory());
|
||||
let rt = LibraryRuntime::new(store);
|
||||
rt.set_session(sample_session("s1"));
|
||||
let got = rt.get_session("s1").unwrap();
|
||||
assert_eq!(got.base_url, "https://nas.example.com");
|
||||
assert_eq!(got.username, "u");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn clear_session_removes_one_server_only() {
|
||||
let store = Arc::new(LibraryStore::open_in_memory());
|
||||
let rt = LibraryRuntime::new(store);
|
||||
rt.set_session(sample_session("s1"));
|
||||
rt.set_session(sample_session("s2"));
|
||||
rt.clear_session("s1");
|
||||
assert!(rt.get_session("s1").is_none());
|
||||
assert!(rt.get_session("s2").is_some());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn snapshot_returns_clones_so_lock_drops_after_call() {
|
||||
let store = Arc::new(LibraryStore::open_in_memory());
|
||||
let rt = LibraryRuntime::new(store);
|
||||
rt.set_session(sample_session("s1"));
|
||||
let snap = rt.snapshot_sessions();
|
||||
// Should be free to mutate after the snapshot.
|
||||
rt.set_session(sample_session("s2"));
|
||||
assert_eq!(snap.len(), 1);
|
||||
assert_eq!(rt.snapshot_sessions().len(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn playback_hint_default_is_idle_and_setter_updates() {
|
||||
let store = Arc::new(LibraryStore::open_in_memory());
|
||||
let rt = LibraryRuntime::new(store);
|
||||
assert_eq!(rt.current_playback_hint(), PlaybackHint::Idle);
|
||||
rt.set_playback_hint(PlaybackHint::Playing);
|
||||
assert_eq!(rt.current_playback_hint(), PlaybackHint::Playing);
|
||||
rt.set_playback_hint(PlaybackHint::PrefetchActive);
|
||||
assert_eq!(rt.current_playback_hint(), PlaybackHint::PrefetchActive);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn job_done_notify_one_survives_early_signal_before_await() {
|
||||
let done = Arc::new(Notify::new());
|
||||
done.notify_one();
|
||||
tokio::time::timeout(std::time::Duration::from_millis(50), done.notified())
|
||||
.await
|
||||
.expect("notify_one must store a permit for a later waiter");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn job_done_notify_waiters_loses_early_signal_before_await() {
|
||||
let done = Arc::new(Notify::new());
|
||||
done.notify_waiters();
|
||||
let waited = tokio::time::timeout(std::time::Duration::from_millis(20), done.notified())
|
||||
.await
|
||||
.is_ok();
|
||||
assert!(
|
||||
!waited,
|
||||
"notify_waiters must not store a permit — resync drain uses notify_one instead"
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,338 @@
|
||||
//! FTS5-backed track search. Skeleton landed in PR-1a — the multi-server +
|
||||
//! libraryScope + bm25 ranking shape from spec §5.9 will be filled in by the
|
||||
//! sync / search PRs.
|
||||
|
||||
use rusqlite::params;
|
||||
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct TrackHit {
|
||||
pub server_id: String,
|
||||
pub id: String,
|
||||
pub title: String,
|
||||
pub artist: Option<String>,
|
||||
pub album: String,
|
||||
}
|
||||
|
||||
/// Run a single-server FTS5 match against `track_fts`, returning rows in
|
||||
/// bm25 order. `query` is passed straight to FTS5 — callers are expected to
|
||||
/// sanitise / quote user input (see §5.13.5: parameterised only).
|
||||
pub fn search_tracks(
|
||||
store: &LibraryStore,
|
||||
server_id: &str,
|
||||
query: &str,
|
||||
limit: i64,
|
||||
) -> Result<Vec<TrackHit>, String> {
|
||||
if !fts_query_meets_min_len(query) {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
let fts = fts_track_match_query(query).ok_or_else(|| "empty query".to_string())?;
|
||||
store.with_read_conn(|conn| {
|
||||
let mut stmt = conn.prepare(
|
||||
r#"
|
||||
SELECT t.server_id, t.id, t.title, t.artist, t.album
|
||||
FROM track_fts f
|
||||
JOIN track t ON t.rowid = f.rowid
|
||||
WHERE track_fts MATCH ?1
|
||||
AND t.server_id = ?2
|
||||
AND t.deleted = 0
|
||||
ORDER BY bm25(track_fts)
|
||||
LIMIT ?3
|
||||
"#,
|
||||
)?;
|
||||
let rows = stmt
|
||||
.query_map(params![fts, server_id, limit], |r| {
|
||||
Ok(TrackHit {
|
||||
server_id: r.get(0)?,
|
||||
id: r.get(1)?,
|
||||
title: r.get(2)?,
|
||||
artist: r.get(3)?,
|
||||
album: r.get(4)?,
|
||||
})
|
||||
})?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||
Ok(rows)
|
||||
})
|
||||
}
|
||||
|
||||
// ── shared search SQL helpers (Advanced Search §5.13 + cross-server §5.5B) ──
|
||||
|
||||
/// Hard ceiling on a single search page — keeps the FTS5 p95 budget (§5.9).
|
||||
/// Callers clamp their requested `limit` into `1..=PAGE_LIMIT_MAX`.
|
||||
pub(crate) const PAGE_LIMIT_MAX: u32 = 500;
|
||||
|
||||
/// Local FTS is skipped below this length — single-character queries (e.g. Cyrillic
|
||||
/// «а», Latin «a») match huge fractions of a large library and bm25+LIMIT can
|
||||
/// take tens of seconds (§5.9: no heavy work on every keystroke).
|
||||
pub const LOCAL_FTS_MIN_QUERY_CHARS: usize = 2;
|
||||
|
||||
/// True when `raw` has enough graphemes for a scoped FTS MATCH.
|
||||
pub fn fts_query_meets_min_len(raw: &str) -> bool {
|
||||
raw.trim().chars().count() >= LOCAL_FTS_MIN_QUERY_CHARS
|
||||
}
|
||||
|
||||
/// Build a safe FTS5 MATCH string: each whitespace token is quoted (and its
|
||||
/// internal `"` doubled) so arbitrary user input can't trip FTS5 query
|
||||
/// syntax. Tokens are implicitly AND-ed. `None` when the input has no tokens.
|
||||
pub(crate) fn fts_query(raw: &str) -> Option<String> {
|
||||
let tokens = fts_token_expr(raw)?;
|
||||
Some(tokens)
|
||||
}
|
||||
|
||||
/// Token expression only (`"a" "b"`), shared by column-scoped builders.
|
||||
pub(crate) fn fts_token_expr(raw: &str) -> Option<String> {
|
||||
fts_token_expr_with(raw, false)
|
||||
}
|
||||
|
||||
/// Prefix token expression (`"a"* "b"*`) for Live Search as-you-type matching.
|
||||
pub(crate) fn fts_prefix_token_expr(raw: &str) -> Option<String> {
|
||||
fts_token_expr_with(raw, true)
|
||||
}
|
||||
|
||||
fn fts_token_expr_with(raw: &str, prefix: bool) -> Option<String> {
|
||||
let tokens: Vec<String> = raw
|
||||
.split_whitespace()
|
||||
.map(|t| {
|
||||
let quoted = format!("\"{}\"", t.replace('"', "\"\""));
|
||||
if prefix {
|
||||
format!("{quoted}*")
|
||||
} else {
|
||||
quoted
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
if tokens.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(tokens.join(" "))
|
||||
}
|
||||
}
|
||||
|
||||
/// Column-scoped prefix match (`artist : "met"*` → Metallica).
|
||||
pub(crate) fn fts_column_prefix_query(column: &str, raw: &str) -> Option<String> {
|
||||
fts_prefix_token_expr(raw).map(|tokens| format!("{column} : {tokens}"))
|
||||
}
|
||||
|
||||
/// Prefix variants for Live Search / Advanced Search as-you-type matching.
|
||||
pub(crate) fn fts_track_prefix_match_query(raw: &str) -> Option<String> {
|
||||
fts_prefix_token_expr(raw).map(|tokens| {
|
||||
["title", "artist", "album", "album_artist"]
|
||||
.iter()
|
||||
.map(|col| format!("{col} : {tokens}"))
|
||||
.collect::<Vec<_>>()
|
||||
.join(" OR ")
|
||||
})
|
||||
}
|
||||
|
||||
pub(crate) fn fts_album_prefix_match_query(raw: &str) -> Option<String> {
|
||||
fts_prefix_token_expr(raw).map(|tokens| {
|
||||
format!("(album : {tokens} OR album_artist : {tokens})")
|
||||
})
|
||||
}
|
||||
|
||||
/// Song / track entity: match primary display fields (excludes `genre` to cut
|
||||
/// noise and FTS fan-out on large libraries).
|
||||
pub(crate) fn fts_track_match_query(raw: &str) -> Option<String> {
|
||||
fts_token_expr(raw).map(|tokens| {
|
||||
["title", "artist", "album", "album_artist"]
|
||||
.iter()
|
||||
.map(|col| format!("{col} : {tokens}"))
|
||||
.collect::<Vec<_>>()
|
||||
.join(" OR ")
|
||||
})
|
||||
}
|
||||
|
||||
/// Project the `track` hot columns prefixed with `alias` (e.g. `t.title`),
|
||||
/// in `repos::row_to_track_row`'s positional order so the Advanced Search /
|
||||
/// cross-server builders can reuse the shared row mapper.
|
||||
/// Effective library id for scoped search — hot column first, then common
|
||||
/// OpenSubsonic / Navidrome keys in `raw_json` (legacy rows may only have JSON).
|
||||
pub(crate) fn library_scope_match_sql(table_alias: &str) -> String {
|
||||
format!(
|
||||
"COALESCE(NULLIF({table_alias}.library_id, ''), \
|
||||
CAST(json_extract({table_alias}.raw_json, '$.libraryId') AS TEXT), \
|
||||
CAST(json_extract({table_alias}.raw_json, '$.library_id') AS TEXT), \
|
||||
CAST(json_extract({table_alias}.raw_json, '$.musicFolderId') AS TEXT))"
|
||||
)
|
||||
}
|
||||
|
||||
pub(crate) fn library_scope_equals_sql(table_alias: &str) -> String {
|
||||
format!("{} = ?", library_scope_match_sql(table_alias))
|
||||
}
|
||||
|
||||
pub(crate) fn aliased_track_columns(alias: &str) -> String {
|
||||
crate::repos::track_columns()
|
||||
.split(',')
|
||||
.map(|c| format!("{alias}.{}", c.trim()))
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ")
|
||||
}
|
||||
|
||||
/// Build a `%…%` LIKE pattern with the LIKE wildcards (`%`, `_`) and the
|
||||
/// `\` escape char escaped, for use with `LIKE ? ESCAPE '\'`. Shared by the
|
||||
/// Advanced Search album/artist name match and the cross-server fuzzy
|
||||
/// title fallback (§5.9).
|
||||
pub(crate) fn like_contains(raw: &str) -> String {
|
||||
let escaped = raw
|
||||
.replace('\\', "\\\\")
|
||||
.replace('%', "\\%")
|
||||
.replace('_', "\\_");
|
||||
format!("%{escaped}%")
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::repos::{TrackRepository, TrackRow};
|
||||
|
||||
fn row(server: &str, id: &str, title: &str, artist: &str, album: &str) -> TrackRow {
|
||||
TrackRow {
|
||||
server_id: server.into(),
|
||||
id: id.into(),
|
||||
title: title.into(),
|
||||
title_sort: None,
|
||||
artist: Some(artist.into()),
|
||||
artist_id: None,
|
||||
album: album.into(),
|
||||
album_id: None,
|
||||
album_artist: Some(artist.into()),
|
||||
duration_sec: 200,
|
||||
track_number: None,
|
||||
disc_number: None,
|
||||
year: None,
|
||||
genre: None,
|
||||
suffix: None,
|
||||
bit_rate: None,
|
||||
size_bytes: None,
|
||||
cover_art_id: None,
|
||||
starred_at: None,
|
||||
user_rating: None,
|
||||
play_count: None,
|
||||
played_at: None,
|
||||
server_path: None,
|
||||
library_id: None,
|
||||
isrc: None,
|
||||
mbid_recording: None,
|
||||
bpm: None,
|
||||
replay_gain_track_db: None,
|
||||
replay_gain_album_db: None,
|
||||
content_hash: None,
|
||||
server_updated_at: None,
|
||||
server_created_at: None,
|
||||
deleted: false,
|
||||
synced_at: 1,
|
||||
raw_json: "{}".into(),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn match_finds_track_by_title() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
TrackRepository::new(&store)
|
||||
.upsert_batch(&[
|
||||
row("s1", "t1", "Aurora", "Anna", "Skylines"),
|
||||
row("s1", "t2", "Sunset", "Beth", "Skylines"),
|
||||
])
|
||||
.unwrap();
|
||||
let hits = search_tracks(&store, "s1", "aurora", 10).unwrap();
|
||||
assert_eq!(hits.len(), 1);
|
||||
assert_eq!(hits[0].id, "t1");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn match_filters_by_server_id() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
TrackRepository::new(&store)
|
||||
.upsert_batch(&[
|
||||
row("s1", "t1", "Aurora", "Anna", "Skylines"),
|
||||
row("s2", "t1", "Aurora", "Anna", "Skylines"),
|
||||
])
|
||||
.unwrap();
|
||||
let hits = search_tracks(&store, "s2", "aurora", 10).unwrap();
|
||||
assert_eq!(hits.len(), 1);
|
||||
assert_eq!(hits[0].server_id, "s2");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn match_skips_deleted_rows() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let repo = TrackRepository::new(&store);
|
||||
repo.upsert_batch(&[row("s1", "t1", "Aurora", "Anna", "Skylines")])
|
||||
.unwrap();
|
||||
let mut gone = row("s1", "t1", "Aurora", "Anna", "Skylines");
|
||||
gone.deleted = true;
|
||||
repo.upsert_batch(&[gone]).unwrap();
|
||||
let hits = search_tracks(&store, "s1", "aurora", 10).unwrap();
|
||||
assert!(hits.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fts_column_prefix_query_scopes_to_one_column() {
|
||||
assert_eq!(
|
||||
fts_column_prefix_query("artist", "metal").as_deref(),
|
||||
Some("artist : \"metal\"*")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fts_prefix_token_expr_ands_multiword_prefixes() {
|
||||
assert_eq!(
|
||||
fts_prefix_token_expr("arch enemy").as_deref(),
|
||||
Some("\"arch\"* \"enemy\"*")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fts_track_prefix_match_query_or_across_display_columns() {
|
||||
let q = fts_track_prefix_match_query("metal").unwrap();
|
||||
assert!(q.contains("title : \"metal\"*"));
|
||||
assert!(q.contains("artist : \"metal\"*"));
|
||||
assert!(!q.contains("genre"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fts_album_prefix_match_query_includes_album_artist() {
|
||||
assert_eq!(
|
||||
fts_album_prefix_match_query("metal").as_deref(),
|
||||
Some("(album : \"metal\"* OR album_artist : \"metal\"*)")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fts_track_match_query_or_across_display_columns() {
|
||||
let q = fts_track_match_query("manowar").unwrap();
|
||||
assert!(q.contains("title : \"manowar\""));
|
||||
assert!(q.contains("artist : \"manowar\""));
|
||||
assert!(!q.contains("genre"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fts_query_meets_min_len_requires_two_graphemes() {
|
||||
assert!(!fts_query_meets_min_len("a"));
|
||||
assert!(!fts_query_meets_min_len("а"));
|
||||
assert!(fts_query_meets_min_len("ab"));
|
||||
assert!(fts_query_meets_min_len("ма"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fts_query_quotes_tokens_and_doubles_inner_quotes() {
|
||||
assert_eq!(fts_query("hello world").as_deref(), Some("\"hello\" \"world\""));
|
||||
assert_eq!(fts_query("a\"b").as_deref(), Some("\"a\"\"b\""));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fts_query_is_none_for_blank_input() {
|
||||
assert!(fts_query("").is_none());
|
||||
assert!(fts_query(" ").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn aliased_track_columns_prefixes_every_column() {
|
||||
let cols = aliased_track_columns("t");
|
||||
assert!(cols.starts_with("t.server_id, t.id, t.title"));
|
||||
assert!(cols.ends_with("t.raw_json"));
|
||||
// One alias per column — count matches the shared column list.
|
||||
assert_eq!(cols.matches("t.").count(), crate::repos::track_columns().split(',').count());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,532 @@
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
|
||||
use std::sync::Mutex;
|
||||
use std::time::Duration;
|
||||
|
||||
use rusqlite::{params, Connection, OpenFlags};
|
||||
use tauri::Manager;
|
||||
|
||||
/// Current head of the embedded migrations. Bump each time a new
|
||||
/// `migrations/NNN_*.sql` is added.
|
||||
pub const LIBRARY_DB_SCHEMA_VERSION: i64 = 5;
|
||||
|
||||
/// Lowest applied schema version the current code can advance from purely
|
||||
/// additively. If a DB carries a version below this, the breaking-bump hook
|
||||
/// fires (spec §5.7 / P22): the library is treated as incompatible, must be
|
||||
/// dropped, and initial sync must restart.
|
||||
///
|
||||
/// At v1 launch this equals `LIBRARY_DB_SCHEMA_VERSION` — no real DB can
|
||||
/// trip the hook. Bump independently of `SCHEMA_VERSION` only when a
|
||||
/// migration cannot be expressed additively.
|
||||
pub const LIBRARY_DB_MIN_COMPATIBLE_VERSION: i64 = 1;
|
||||
|
||||
pub(crate) const INITIAL_SQL: &str = include_str!("../migrations/001_initial.sql");
|
||||
const MIGRATION_002_N1_BULK_UNRELIABLE: &str =
|
||||
include_str!("../migrations/002_n1_bulk_unreliable.sql");
|
||||
const MIGRATION_003_TRACK_REMAP_INDEXES: &str =
|
||||
include_str!("../migrations/003_track_remap_indexes.sql");
|
||||
const MIGRATION_004_TRACK_TITLE_INDEX: &str =
|
||||
include_str!("../migrations/004_track_title_index.sql");
|
||||
const MIGRATION_005_TRACK_GENRE_YEAR_INDEXES: &str =
|
||||
include_str!("../migrations/005_track_genre_year_indexes.sql");
|
||||
|
||||
/// Embedded migrations. Ordered ascending by `version`; the runner sorts
|
||||
/// defensively before applying so the source order can stay readable.
|
||||
const MIGRATIONS: &[(i64, &str)] = &[
|
||||
(1, INITIAL_SQL),
|
||||
(2, MIGRATION_002_N1_BULK_UNRELIABLE),
|
||||
(3, MIGRATION_003_TRACK_REMAP_INDEXES),
|
||||
(4, MIGRATION_004_TRACK_TITLE_INDEX),
|
||||
(5, MIGRATION_005_TRACK_GENRE_YEAR_INDEXES),
|
||||
];
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub(crate) enum MigrationOutcome {
|
||||
/// Every missing migration was applied (or the DB was already at head).
|
||||
Applied,
|
||||
/// The DB carried a schema below `LIBRARY_DB_MIN_COMPATIBLE_VERSION`,
|
||||
/// so the breaking-bump hook fired. Callers should treat the library
|
||||
/// data as discarded and trigger a fresh initial sync (P22).
|
||||
BreakingBump,
|
||||
}
|
||||
|
||||
/// In-memory tests share one DB across the read/write pair in a single store.
|
||||
static IN_MEMORY_DB_COUNTER: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
fn in_memory_uri() -> String {
|
||||
let n = IN_MEMORY_DB_COUNTER.fetch_add(1, Ordering::Relaxed);
|
||||
format!("file:psysonic_library_mem_{n}?mode=memory&cache=shared")
|
||||
}
|
||||
|
||||
pub struct LibraryStore {
|
||||
/// Writes, migrations, and sync ingest (single writer).
|
||||
write_conn: Mutex<Connection>,
|
||||
/// Read-only handle for search / status / hydrate while sync writes (WAL).
|
||||
read_conn: Mutex<Connection>,
|
||||
/// IS-3 bulk ingest in progress — read paths skip write-lock work.
|
||||
bulk_ingest_active: AtomicBool,
|
||||
}
|
||||
|
||||
impl LibraryStore {
|
||||
pub fn init(app: &tauri::AppHandle) -> Result<Self, String> {
|
||||
let db_path = library_db_path(app)?;
|
||||
if let Some(parent) = db_path.parent() {
|
||||
std::fs::create_dir_all(parent).map_err(|e| e.to_string())?;
|
||||
}
|
||||
Self::open_file(&db_path)
|
||||
}
|
||||
|
||||
fn open_file(db_path: &Path) -> Result<Self, String> {
|
||||
let write_conn = Connection::open(db_path).map_err(|e| e.to_string())?;
|
||||
configure_write_connection(&write_conn).map_err(|e| e.to_string())?;
|
||||
run_migrations(&write_conn).map_err(|e| e.to_string())?;
|
||||
let read_conn = Connection::open_with_flags(db_path, OpenFlags::SQLITE_OPEN_READ_ONLY)
|
||||
.map_err(|e| e.to_string())?;
|
||||
configure_read_connection(&read_conn).map_err(|e| e.to_string())?;
|
||||
Ok(Self {
|
||||
write_conn: Mutex::new(write_conn),
|
||||
read_conn: Mutex::new(read_conn),
|
||||
bulk_ingest_active: AtomicBool::new(false),
|
||||
})
|
||||
}
|
||||
|
||||
/// Build an in-memory DB with the production schema applied.
|
||||
pub fn open_in_memory() -> Self {
|
||||
let uri = in_memory_uri();
|
||||
let write_conn = Connection::open(&uri).expect("in-memory write connection");
|
||||
configure_write_connection(&write_conn).expect("write pragmas");
|
||||
run_migrations(&write_conn).expect("schema migration");
|
||||
let read_conn = Connection::open(&uri).expect("in-memory read connection");
|
||||
configure_read_connection(&read_conn).expect("read pragmas");
|
||||
Self {
|
||||
write_conn: Mutex::new(write_conn),
|
||||
read_conn: Mutex::new(read_conn),
|
||||
bulk_ingest_active: AtomicBool::new(false),
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn set_bulk_ingest_active(&self, active: bool) {
|
||||
self.bulk_ingest_active
|
||||
.store(active, Ordering::Release);
|
||||
}
|
||||
|
||||
pub(crate) fn bulk_ingest_active(&self) -> bool {
|
||||
self.bulk_ingest_active.load(Ordering::Acquire)
|
||||
}
|
||||
|
||||
/// Writer connection — sync ingest, migrations, mutations.
|
||||
pub(crate) fn with_conn<R>(
|
||||
&self,
|
||||
op: &'static str,
|
||||
f: impl FnOnce(&Connection) -> rusqlite::Result<R>,
|
||||
) -> Result<R, String> {
|
||||
let lock_start = std::time::Instant::now();
|
||||
let conn = self
|
||||
.write_conn
|
||||
.lock()
|
||||
.map_err(|_| "library store write lock poisoned".to_string())?;
|
||||
let lock_wait_ms = lock_start.elapsed().as_millis();
|
||||
let exec_start = std::time::Instant::now();
|
||||
let out = f(&conn).map_err(|e| e.to_string());
|
||||
let exec_ms = exec_start.elapsed().as_millis();
|
||||
log_write_op(op, lock_wait_ms, exec_ms);
|
||||
out
|
||||
}
|
||||
|
||||
/// Read-only connection — search, status, hydrate; does not block on sync writes.
|
||||
pub(crate) fn with_read_conn<R>(
|
||||
&self,
|
||||
f: impl FnOnce(&Connection) -> rusqlite::Result<R>,
|
||||
) -> Result<R, String> {
|
||||
let conn = self
|
||||
.read_conn
|
||||
.lock()
|
||||
.map_err(|_| "library store read lock poisoned".to_string())?;
|
||||
f(&conn).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
pub(crate) fn with_conn_mut<R>(
|
||||
&self,
|
||||
op: &'static str,
|
||||
f: impl FnOnce(&mut Connection) -> rusqlite::Result<R>,
|
||||
) -> Result<R, String> {
|
||||
self.with_conn_mut_timed(op, f).map(|(value, _)| value)
|
||||
}
|
||||
|
||||
pub(crate) fn with_conn_mut_timed<R>(
|
||||
&self,
|
||||
op: &'static str,
|
||||
f: impl FnOnce(&mut Connection) -> rusqlite::Result<R>,
|
||||
) -> Result<(R, WriteOpTiming), String> {
|
||||
let lock_start = std::time::Instant::now();
|
||||
let mut conn = self
|
||||
.write_conn
|
||||
.lock()
|
||||
.map_err(|_| "library store write lock poisoned".to_string())?;
|
||||
let lock_wait_ms = lock_start.elapsed().as_millis() as u64;
|
||||
let exec_start = std::time::Instant::now();
|
||||
let out = f(&mut conn).map_err(|e| e.to_string())?;
|
||||
let exec_ms = exec_start.elapsed().as_millis() as u64;
|
||||
log_write_op(op, lock_wait_ms as u128, exec_ms as u128);
|
||||
Ok((out, WriteOpTiming { lock_wait_ms, exec_ms }))
|
||||
}
|
||||
}
|
||||
|
||||
/// Timing split returned to ingest progress (DevTools / terminal).
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
pub struct WriteOpTiming {
|
||||
pub lock_wait_ms: u64,
|
||||
pub exec_ms: u64,
|
||||
}
|
||||
|
||||
impl WriteOpTiming {
|
||||
pub fn total_ms(&self) -> u64 {
|
||||
self.lock_wait_ms.saturating_add(self.exec_ms)
|
||||
}
|
||||
}
|
||||
|
||||
fn log_write_op(op: &str, lock_wait_ms: u128, exec_ms: u128) {
|
||||
if lock_wait_ms >= 1000 || exec_ms >= 1000 {
|
||||
crate::app_eprintln!(
|
||||
"[library-db] SLOW write op={op} lock_wait_ms={lock_wait_ms} exec_ms={exec_ms}"
|
||||
);
|
||||
} else if lock_wait_ms >= 50 || exec_ms >= 200 {
|
||||
crate::app_eprintln!("[library-db] write op={op} lock_wait_ms={lock_wait_ms} exec_ms={exec_ms}");
|
||||
}
|
||||
}
|
||||
|
||||
fn library_db_path(app: &tauri::AppHandle) -> Result<PathBuf, String> {
|
||||
let base = app.path().app_data_dir().map_err(|e| e.to_string())?;
|
||||
Ok(base.join("library.sqlite"))
|
||||
}
|
||||
|
||||
fn configure_write_connection(conn: &Connection) -> rusqlite::Result<()> {
|
||||
conn.busy_timeout(Duration::from_secs(30))?;
|
||||
conn.pragma_update(None, "journal_mode", "WAL")?;
|
||||
conn.pragma_update(None, "synchronous", "NORMAL")?;
|
||||
conn.pragma_update(None, "temp_store", "MEMORY")?;
|
||||
conn.pragma_update(None, "foreign_keys", "ON")?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn configure_read_connection(conn: &Connection) -> rusqlite::Result<()> {
|
||||
conn.busy_timeout(Duration::from_secs(5))?;
|
||||
conn.pragma_update(None, "foreign_keys", "ON")?;
|
||||
// Search / browse hot path on large libraries (read-only handle).
|
||||
conn.pragma_update(None, "cache_size", -64_000)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn run_migrations(conn: &Connection) -> rusqlite::Result<MigrationOutcome> {
|
||||
run_migrations_with(
|
||||
conn,
|
||||
MIGRATIONS,
|
||||
LIBRARY_DB_MIN_COMPATIBLE_VERSION,
|
||||
handle_breaking_schema_bump,
|
||||
)
|
||||
}
|
||||
|
||||
/// Test-friendly entry point. Production code goes through `run_migrations`,
|
||||
/// which fixes `migrations`, `min_compatible`, and `hook` to the prod values.
|
||||
pub(crate) fn run_migrations_with(
|
||||
conn: &Connection,
|
||||
migrations: &[(i64, &str)],
|
||||
min_compatible: i64,
|
||||
hook: fn(&Connection, i64, i64) -> rusqlite::Result<()>,
|
||||
) -> rusqlite::Result<MigrationOutcome> {
|
||||
conn.execute_batch(
|
||||
"CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
version INTEGER PRIMARY KEY,
|
||||
applied_at INTEGER NOT NULL
|
||||
);",
|
||||
)?;
|
||||
|
||||
// Breaking-bump detection only meaningful for already-initialised DBs.
|
||||
let max_applied: Option<i64> = conn.query_row(
|
||||
"SELECT MAX(version) FROM schema_migrations",
|
||||
[],
|
||||
|row| row.get::<_, Option<i64>>(0),
|
||||
)?;
|
||||
if let Some(max_applied) = max_applied {
|
||||
if max_applied < min_compatible {
|
||||
hook(conn, max_applied, LIBRARY_DB_SCHEMA_VERSION)?;
|
||||
return Ok(MigrationOutcome::BreakingBump);
|
||||
}
|
||||
}
|
||||
|
||||
let mut ordered: Vec<(i64, &str)> = migrations.iter().map(|(v, s)| (*v, *s)).collect();
|
||||
ordered.sort_by_key(|(v, _)| *v);
|
||||
for (version, sql) in ordered {
|
||||
let already: i64 = conn.query_row(
|
||||
"SELECT COUNT(*) FROM schema_migrations WHERE version = ?1",
|
||||
params![version],
|
||||
|row| row.get(0),
|
||||
)?;
|
||||
if already > 0 {
|
||||
continue;
|
||||
}
|
||||
conn.execute_batch(sql)?;
|
||||
conn.execute(
|
||||
"INSERT INTO schema_migrations (version, applied_at) VALUES (?1, strftime('%s','now'))",
|
||||
params![version],
|
||||
)?;
|
||||
}
|
||||
Ok(MigrationOutcome::Applied)
|
||||
}
|
||||
|
||||
/// P22 breaking-schema-bump hook. PR-1b ships a no-op stub: the function
|
||||
/// signature, call site, and `MigrationOutcome::BreakingBump` signal are in
|
||||
/// place, but the actual library-drop + sync-reset logic lands when the
|
||||
/// first real breaking bump happens. Until then the constants guarantee the
|
||||
/// hook never fires on production data.
|
||||
fn handle_breaking_schema_bump(
|
||||
_conn: &Connection,
|
||||
_max_applied: i64,
|
||||
_target_version: i64,
|
||||
) -> rusqlite::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn read_conn_sees_committed_writes_from_write_conn() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
store
|
||||
.with_conn("misc", |c| {
|
||||
c.execute(
|
||||
"INSERT INTO sync_state (server_id, library_scope, sync_phase) \
|
||||
VALUES ('s1', '', 'ready')",
|
||||
[],
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
let phase: String = store
|
||||
.with_read_conn(|c| {
|
||||
c.query_row(
|
||||
"SELECT sync_phase FROM sync_state WHERE server_id = 's1'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(phase, "ready");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn open_in_memory_creates_all_expected_tables() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let tables = store
|
||||
.with_conn("misc", |c| {
|
||||
let mut stmt =
|
||||
c.prepare("SELECT name FROM sqlite_master WHERE type='table' ORDER BY name")?;
|
||||
let rows: rusqlite::Result<Vec<String>> =
|
||||
stmt.query_map([], |r| r.get::<_, String>(0))?.collect();
|
||||
rows
|
||||
})
|
||||
.unwrap();
|
||||
|
||||
for expected in [
|
||||
"album",
|
||||
"artist",
|
||||
"canonical_enrichment_link",
|
||||
"canonical_identity",
|
||||
"canonical_track",
|
||||
"schema_migrations",
|
||||
"sync_state",
|
||||
"track",
|
||||
"track_artifact",
|
||||
"track_canonical_link",
|
||||
"track_extension",
|
||||
"track_fact",
|
||||
"track_id_history",
|
||||
"track_offline",
|
||||
] {
|
||||
assert!(
|
||||
tables.iter().any(|t| t == expected),
|
||||
"missing table `{expected}` — got {tables:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn schema_migrations_records_head_version() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let versions: Vec<i64> = store
|
||||
.with_conn("misc", |c| {
|
||||
let mut stmt =
|
||||
c.prepare("SELECT version FROM schema_migrations ORDER BY version")?;
|
||||
let rows: rusqlite::Result<Vec<i64>> =
|
||||
stmt.query_map([], |r| r.get(0))?.collect();
|
||||
rows
|
||||
})
|
||||
.unwrap();
|
||||
// Embedded migrations are numbered 1..=head, all applied on a fresh DB.
|
||||
let expected: Vec<i64> = (1..=LIBRARY_DB_SCHEMA_VERSION).collect();
|
||||
assert_eq!(versions, expected);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn run_migrations_is_idempotent_across_reopens() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let outcome = store
|
||||
.with_conn("migrate", run_migrations)
|
||||
.expect("second migration pass must be a no-op");
|
||||
assert_eq!(outcome, MigrationOutcome::Applied);
|
||||
let count: i64 = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row("SELECT COUNT(*) FROM schema_migrations", [], |r| r.get(0))
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
count, LIBRARY_DB_SCHEMA_VERSION,
|
||||
"one schema_migrations row per embedded migration, no duplicates"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fts_virtual_table_exists() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let count: i64 = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT COUNT(*) FROM sqlite_master WHERE name='track_fts'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(count, 1);
|
||||
}
|
||||
|
||||
// ── PR-1b: edge-case tests via the test-only `run_migrations_with` ─────
|
||||
|
||||
/// `ALTER TABLE artist ADD COLUMN bio TEXT;` — minimal additive fixture,
|
||||
/// nullable column with no default. Mirrors the §5.7 additive-first rule.
|
||||
/// Numbered above the real embedded head so it stacks on a migrated DB.
|
||||
const FIXTURE_ADD_BIO: &str = "ALTER TABLE artist ADD COLUMN bio TEXT;";
|
||||
const FIXTURE_ADD_BIO_VERSION: i64 = LIBRARY_DB_SCHEMA_VERSION + 1;
|
||||
|
||||
fn no_op_hook(_c: &Connection, _from: i64, _to: i64) -> rusqlite::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn always_fail_hook(_c: &Connection, _from: i64, _to: i64) -> rusqlite::Result<()> {
|
||||
panic!("breaking-bump hook must NOT fire in this test");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn additive_migration_preserves_existing_data() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
store
|
||||
.with_conn("misc", |c| {
|
||||
c.execute(
|
||||
"INSERT INTO artist (server_id, id, name, synced_at) \
|
||||
VALUES ('s1', 'a1', 'Existing Artist', 1)",
|
||||
[],
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
|
||||
let outcome = store
|
||||
.with_conn("misc", |c| {
|
||||
run_migrations_with(
|
||||
c,
|
||||
&[(1, INITIAL_SQL), (FIXTURE_ADD_BIO_VERSION, FIXTURE_ADD_BIO)],
|
||||
LIBRARY_DB_MIN_COMPATIBLE_VERSION,
|
||||
always_fail_hook,
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(outcome, MigrationOutcome::Applied);
|
||||
|
||||
let (name, bio): (String, Option<String>) = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT name, bio FROM artist WHERE id = 'a1'",
|
||||
[],
|
||||
|r| Ok((r.get(0)?, r.get(1)?)),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(name, "Existing Artist");
|
||||
assert!(bio.is_none());
|
||||
|
||||
let versions: Vec<i64> = store
|
||||
.with_conn("misc", |c| {
|
||||
let mut stmt =
|
||||
c.prepare("SELECT version FROM schema_migrations ORDER BY version")?;
|
||||
let rows: rusqlite::Result<Vec<i64>> =
|
||||
stmt.query_map([], |r| r.get(0))?.collect();
|
||||
rows
|
||||
})
|
||||
.unwrap();
|
||||
// Real embedded migrations (1..=head) plus the additive fixture.
|
||||
let mut expected: Vec<i64> = (1..=LIBRARY_DB_SCHEMA_VERSION).collect();
|
||||
expected.push(FIXTURE_ADD_BIO_VERSION);
|
||||
assert_eq!(versions, expected);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn runner_sorts_unsorted_migration_slice_before_applying() {
|
||||
// If a future contributor lists migrations out of order in the
|
||||
// source slice, the runner must still apply them ascending.
|
||||
let conn = Connection::open_in_memory().unwrap();
|
||||
conn.pragma_update(None, "foreign_keys", "ON").unwrap();
|
||||
|
||||
let outcome = run_migrations_with(
|
||||
&conn,
|
||||
&[(2, FIXTURE_ADD_BIO), (1, INITIAL_SQL)],
|
||||
LIBRARY_DB_MIN_COMPATIBLE_VERSION,
|
||||
always_fail_hook,
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(outcome, MigrationOutcome::Applied);
|
||||
|
||||
let versions: Vec<i64> = {
|
||||
let mut stmt = conn
|
||||
.prepare("SELECT version FROM schema_migrations ORDER BY applied_at, version")
|
||||
.unwrap();
|
||||
let rows: rusqlite::Result<Vec<i64>> =
|
||||
stmt.query_map([], |r| r.get(0)).unwrap().collect();
|
||||
rows.unwrap()
|
||||
};
|
||||
assert_eq!(versions, vec![1, 2]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn breaking_bump_hook_fires_when_db_below_min_compatible() {
|
||||
// Simulate a future code release where MIN_COMPATIBLE was bumped past
|
||||
// the version the DB currently carries (the real embedded head).
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let outcome = store
|
||||
.with_conn("misc", |c| {
|
||||
run_migrations_with(
|
||||
c,
|
||||
MIGRATIONS,
|
||||
LIBRARY_DB_SCHEMA_VERSION + 1, // bumped past current applied
|
||||
no_op_hook,
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(outcome, MigrationOutcome::BreakingBump);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn breaking_bump_hook_does_not_fire_on_fresh_db() {
|
||||
let conn = Connection::open_in_memory().unwrap();
|
||||
conn.pragma_update(None, "foreign_keys", "ON").unwrap();
|
||||
let outcome = run_migrations_with(
|
||||
&conn,
|
||||
MIGRATIONS,
|
||||
// Even a wildly future min_compatible must not trip on a fresh DB:
|
||||
// no rows in schema_migrations means "nothing to migrate from".
|
||||
999,
|
||||
always_fail_hook,
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(outcome, MigrationOutcome::Applied);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,151 @@
|
||||
//! C12 — HTTP backoff for ingest batches (spec §6.8).
|
||||
//!
|
||||
//! The runner does not advance its cursor on transient failure; it
|
||||
//! sleeps via `Backoff::next_delay`, retries the same batch, and
|
||||
//! resets the counter on success. The cap is 120 s — the user is
|
||||
//! waiting for sync to finish, longer hangs become indistinguishable
|
||||
//! from "stuck" without progress events to reassure them.
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
/// Exponential schedule per §6.8: `2s → 4s → 8s → … cap 120s`. The
|
||||
/// caller adds jitter on top to avoid herd-sync against a recovering
|
||||
/// upstream proxy.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Backoff {
|
||||
attempt: u32,
|
||||
base: Duration,
|
||||
cap: Duration,
|
||||
}
|
||||
|
||||
impl Default for Backoff {
|
||||
fn default() -> Self {
|
||||
Self::new(Duration::from_secs(2), Duration::from_secs(120))
|
||||
}
|
||||
}
|
||||
|
||||
impl Backoff {
|
||||
pub fn new(base: Duration, cap: Duration) -> Self {
|
||||
Self { attempt: 0, base, cap }
|
||||
}
|
||||
|
||||
/// Reset after a successful batch — the next failure starts at
|
||||
/// the base delay again.
|
||||
pub fn reset(&mut self) {
|
||||
self.attempt = 0;
|
||||
}
|
||||
|
||||
/// Compute the next sleep duration and bump the attempt counter.
|
||||
/// Doubles each call (2 → 4 → 8 → …) and clamps to `cap`. Caller
|
||||
/// adds jitter via `with_jitter` if desired.
|
||||
pub fn next_delay(&mut self) -> Duration {
|
||||
let n = self.attempt;
|
||||
self.attempt = self.attempt.saturating_add(1);
|
||||
let scaled = self.base.saturating_mul(1u32 << n.min(31));
|
||||
scaled.min(self.cap)
|
||||
}
|
||||
|
||||
pub fn attempt(&self) -> u32 {
|
||||
self.attempt
|
||||
}
|
||||
}
|
||||
|
||||
/// Salt for production jitter: attempt plus sub-second clock noise so
|
||||
/// concurrent retries don't share the same jitter slot.
|
||||
pub fn jitter_salt(attempt: u32) -> u64 {
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
let nanos = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.map(|d| d.subsec_nanos() as u64)
|
||||
.unwrap_or(0);
|
||||
u64::from(attempt).saturating_add(nanos)
|
||||
}
|
||||
|
||||
/// Add ±25% jitter to a planned sleep — deterministic per `salt` so
|
||||
/// tests can pin the result without pulling in `rand`. Production
|
||||
/// code passes `jitter_salt(attempt)` as the salt; tests pass a fixed
|
||||
/// value to assert the formula.
|
||||
pub fn with_jitter(base: Duration, salt: u64) -> Duration {
|
||||
let millis = base.as_millis().min(u128::from(u64::MAX)) as u64;
|
||||
if millis == 0 {
|
||||
return base;
|
||||
}
|
||||
let span = millis / 2; // ±25% = total span of 50% of base
|
||||
if span == 0 {
|
||||
return base;
|
||||
}
|
||||
// map salt into [-span/2, +span/2]
|
||||
let pct = (salt % span) as i64 - (span as i64 / 2);
|
||||
let jittered = (millis as i64).saturating_add(pct).max(1) as u64;
|
||||
Duration::from_millis(jittered)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn next_delay_doubles_until_cap() {
|
||||
let mut b = Backoff::default();
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(2));
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(4));
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(8));
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(16));
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(32));
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(64));
|
||||
// Next would be 128 — clamps to 120 cap.
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(120));
|
||||
// Stays at cap from here on.
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(120));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reset_brings_schedule_back_to_base() {
|
||||
let mut b = Backoff::default();
|
||||
b.next_delay();
|
||||
b.next_delay();
|
||||
assert!(b.attempt() > 0);
|
||||
b.reset();
|
||||
assert_eq!(b.attempt(), 0);
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(2));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn next_delay_handles_custom_base_and_cap() {
|
||||
let mut b = Backoff::new(Duration::from_millis(50), Duration::from_secs(1));
|
||||
assert_eq!(b.next_delay(), Duration::from_millis(50));
|
||||
assert_eq!(b.next_delay(), Duration::from_millis(100));
|
||||
assert_eq!(b.next_delay(), Duration::from_millis(200));
|
||||
assert_eq!(b.next_delay(), Duration::from_millis(400));
|
||||
assert_eq!(b.next_delay(), Duration::from_millis(800));
|
||||
// 1600 would exceed 1s cap.
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(1));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn next_delay_does_not_overflow_at_extreme_attempts() {
|
||||
let mut b = Backoff::default();
|
||||
for _ in 0..100 {
|
||||
// Should saturate at the cap, never panic on shift overflow.
|
||||
let _ = b.next_delay();
|
||||
}
|
||||
assert_eq!(b.next_delay(), Duration::from_secs(120));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn jitter_stays_within_plus_minus_half_of_span() {
|
||||
let base = Duration::from_secs(8);
|
||||
let span_ms = base.as_millis() as u64 / 2; // 4000 ms
|
||||
let lo = base.as_millis() as u64 - span_ms / 2; // 6000 ms
|
||||
let hi = base.as_millis() as u64 + span_ms / 2; // 10000 ms
|
||||
for salt in 0u64..1000 {
|
||||
let j = with_jitter(base, salt).as_millis() as u64;
|
||||
assert!(j >= lo && j <= hi, "salt {salt} → {j}ms outside [{lo},{hi}]");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn jitter_with_zero_base_returns_zero() {
|
||||
assert_eq!(with_jitter(Duration::ZERO, 12345), Duration::ZERO);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
//! C11 — bandwidth priority lane (spec §6.2.4).
|
||||
//!
|
||||
//! PR-3d2 ships the data types + parallelism resolver. The signal
|
||||
//! itself is pushed in from the top crate (PR-5 hooks audio engine
|
||||
//! events from `psysonic-audio` into a shared `PlaybackHint` cell);
|
||||
//! the library side just consumes it.
|
||||
|
||||
/// What the player is currently doing — drives crawl parallelism.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
|
||||
pub enum PlaybackHint {
|
||||
/// Nothing playing, queue idle → bulk crawl runs at normal
|
||||
/// parallelism.
|
||||
#[default]
|
||||
Idle,
|
||||
/// Active stream (user listening). Bulk drops to single-request
|
||||
/// crawling and increases inter-request delay.
|
||||
Playing,
|
||||
/// Queue prefetch / waveform analysis hot — priority lane only;
|
||||
/// bulk pauses entirely.
|
||||
PrefetchActive,
|
||||
}
|
||||
|
||||
/// Parallelism budget the bulk crawl uses for this tick. `0` means
|
||||
/// bulk is suspended this tick (PR-3d2 returns the value; the runner
|
||||
/// honoring it is wired in PR-3d2 too via `BackgroundScheduler`).
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct ParallelismBudget {
|
||||
pub max_concurrent: u32,
|
||||
/// Minimum gap between successive bulk requests, in ms.
|
||||
pub min_request_gap_ms: u32,
|
||||
}
|
||||
|
||||
impl ParallelismBudget {
|
||||
pub fn resolve(hint: PlaybackHint) -> Self {
|
||||
match hint {
|
||||
PlaybackHint::Idle => Self {
|
||||
max_concurrent: 4,
|
||||
min_request_gap_ms: 0,
|
||||
},
|
||||
PlaybackHint::Playing => Self {
|
||||
max_concurrent: 1,
|
||||
min_request_gap_ms: 250,
|
||||
},
|
||||
PlaybackHint::PrefetchActive => Self {
|
||||
max_concurrent: 0,
|
||||
min_request_gap_ms: 0,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
pub fn bulk_paused(&self) -> bool {
|
||||
self.max_concurrent == 0
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn idle_resolves_to_normal_parallelism() {
|
||||
let b = ParallelismBudget::resolve(PlaybackHint::Idle);
|
||||
assert_eq!(b.max_concurrent, 4);
|
||||
assert_eq!(b.min_request_gap_ms, 0);
|
||||
assert!(!b.bulk_paused());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn playing_caps_parallelism_to_one_with_inter_request_gap() {
|
||||
let b = ParallelismBudget::resolve(PlaybackHint::Playing);
|
||||
assert_eq!(b.max_concurrent, 1);
|
||||
assert!(b.min_request_gap_ms >= 100, "playing must space requests out");
|
||||
assert!(!b.bulk_paused());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prefetch_active_pauses_bulk_crawl_entirely() {
|
||||
let b = ParallelismBudget::resolve(PlaybackHint::PrefetchActive);
|
||||
assert!(b.bulk_paused());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn playback_hint_default_is_idle() {
|
||||
assert_eq!(PlaybackHint::default(), PlaybackHint::Idle);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
//! C9 — request budget per spec §6.2.5.
|
||||
//!
|
||||
//! Soft caps the scheduler declares before each pass; runners
|
||||
//! consume them to decide when to defer the remainder to the next
|
||||
//! tick. PR-3d2 ships the data type + lookups; PR-5 / the runner
|
||||
//! wires the actual consumption (delta runner can already be capped
|
||||
//! via batch_size — the budget is a higher-level abstraction).
|
||||
|
||||
/// Per-pass HTTP call budgets matching the spec table.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum PassKind {
|
||||
/// `poll_tick (unchanged watermark)` — 1 call (`getArtists` only).
|
||||
PollTick,
|
||||
/// `delta_sync (light)` — small targeted delta, 50 calls.
|
||||
DeltaLight,
|
||||
/// `delta_sync (count mismatch / tombstone)` — 200 calls,
|
||||
/// split across ticks if exhausted.
|
||||
DeltaMismatch,
|
||||
/// Initial sync — unlimited (only user cancel stops it).
|
||||
InitialSync,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct RequestBudget {
|
||||
pub kind: PassKind,
|
||||
/// `None` = unlimited.
|
||||
pub cap: Option<u32>,
|
||||
}
|
||||
|
||||
impl RequestBudget {
|
||||
pub const POLL_TICK_CAP: u32 = 1;
|
||||
pub const DELTA_LIGHT_CAP: u32 = 50;
|
||||
pub const DELTA_MISMATCH_CAP: u32 = 200;
|
||||
|
||||
pub fn for_pass(kind: PassKind) -> Self {
|
||||
let cap = match kind {
|
||||
PassKind::PollTick => Some(Self::POLL_TICK_CAP),
|
||||
PassKind::DeltaLight => Some(Self::DELTA_LIGHT_CAP),
|
||||
PassKind::DeltaMismatch => Some(Self::DELTA_MISMATCH_CAP),
|
||||
PassKind::InitialSync => None,
|
||||
};
|
||||
Self { kind, cap }
|
||||
}
|
||||
|
||||
pub fn is_unlimited(&self) -> bool {
|
||||
self.cap.is_none()
|
||||
}
|
||||
|
||||
/// Returns `true` when `used` requests still fit inside the cap.
|
||||
/// Unlimited budgets always return `true`.
|
||||
pub fn has_room(&self, used: u32) -> bool {
|
||||
match self.cap {
|
||||
None => true,
|
||||
Some(cap) => used < cap,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn poll_tick_cap_is_one_request() {
|
||||
let b = RequestBudget::for_pass(PassKind::PollTick);
|
||||
assert_eq!(b.cap, Some(1));
|
||||
assert!(b.has_room(0));
|
||||
assert!(!b.has_room(1));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delta_light_caps_at_fifty() {
|
||||
let b = RequestBudget::for_pass(PassKind::DeltaLight);
|
||||
assert_eq!(b.cap, Some(50));
|
||||
assert!(b.has_room(49));
|
||||
assert!(!b.has_room(50));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delta_mismatch_caps_at_two_hundred() {
|
||||
let b = RequestBudget::for_pass(PassKind::DeltaMismatch);
|
||||
assert_eq!(b.cap, Some(200));
|
||||
assert!(b.has_room(199));
|
||||
assert!(!b.has_room(200));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn initial_sync_is_unlimited() {
|
||||
let b = RequestBudget::for_pass(PassKind::InitialSync);
|
||||
assert!(b.is_unlimited());
|
||||
assert!(b.has_room(u32::MAX));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,680 @@
|
||||
//! C1 — Capability probe (spec §6.1 / §6.1.1).
|
||||
//!
|
||||
//! Drives the Subsonic client + an optional Navidrome native probe to
|
||||
//! populate `sync_state.capability_flags` before initial sync picks its
|
||||
//! ingest strategy (§6.3). PR-3a only writes flags from the responses;
|
||||
//! interpretation lives in PR-3b's `IngestStrategy` selector.
|
||||
|
||||
use psysonic_integration::navidrome::probe::native_bulk_available;
|
||||
use psysonic_integration::subsonic::{ServerInfo, SubsonicClient, SubsonicError};
|
||||
|
||||
/// Bitfield matching spec §6.1.1. `u32` storage so the `sync_state`
|
||||
/// table can keep it as a single integer column (`capability_flags
|
||||
/// INTEGER NOT NULL DEFAULT 0`).
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
pub struct CapabilityFlags(u32);
|
||||
|
||||
impl CapabilityFlags {
|
||||
/// N1 — Navidrome native `/api/song` paginated ingest.
|
||||
pub const NAVIDROME_NATIVE_BULK: u32 = 0x001;
|
||||
/// S1 — Subsonic `search3` empty-query bulk ingest.
|
||||
pub const SUBSONIC_SEARCH3_BULK: u32 = 0x002;
|
||||
/// `getScanStatus` available (Subsonic 1.15+); cheap-poll tier signal.
|
||||
pub const SCAN_STATUS_AVAILABLE: u32 = 0x004;
|
||||
/// Server advertises OpenSubsonic extensions (`isrc`, `played`,
|
||||
/// `bpm`, contributor arrays, …).
|
||||
pub const OPEN_SUBSONIC: u32 = 0x008;
|
||||
/// Track ids may shift across server re-indexing — sync engine must
|
||||
/// run the `track_id_history` remap pass (§6.9, P33). Always set
|
||||
/// for Navidrome.
|
||||
pub const UNSTABLE_TRACK_IDS: u32 = 0x010;
|
||||
/// S3 — `getIndexes` / `getMusicDirectory` available (file-tree
|
||||
/// fallback when ID3 endpoints are missing entirely).
|
||||
pub const FILE_TREE_BROWSE: u32 = 0x020;
|
||||
|
||||
pub fn new(bits: u32) -> Self {
|
||||
Self(bits)
|
||||
}
|
||||
|
||||
pub fn bits(self) -> u32 {
|
||||
self.0
|
||||
}
|
||||
|
||||
pub fn contains(self, flag: u32) -> bool {
|
||||
self.0 & flag == flag
|
||||
}
|
||||
|
||||
pub fn insert(&mut self, flag: u32) {
|
||||
self.0 |= flag;
|
||||
}
|
||||
|
||||
pub fn remove(&mut self, flag: u32) {
|
||||
self.0 &= !flag;
|
||||
}
|
||||
}
|
||||
|
||||
/// Optional input for `CapabilityProbe::run` — Navidrome native API
|
||||
/// needs its own bearer token (separate from the Subsonic salted-md5
|
||||
/// auth). When `None`, the `NavidromeNativeBulk` bit stays clear and
|
||||
/// sync falls back to Subsonic strategies.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct NavidromeProbeCredentials {
|
||||
pub server_url: String,
|
||||
pub bearer_token: String,
|
||||
}
|
||||
|
||||
/// Outcome of the capability probe — both the bitfield (stored in
|
||||
/// `sync_state.capability_flags`) and the raw `ServerInfo` envelope
|
||||
/// metadata (callers may want to log `serverVersion` etc.).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CapabilityProbeResult {
|
||||
pub flags: CapabilityFlags,
|
||||
pub server_info: ServerInfo,
|
||||
/// Server-reported track count from `getScanStatus.count`, when the
|
||||
/// server exposes it. `None` when `getScanStatus` is unavailable or
|
||||
/// reports no count. Persisted as the `server_track_count` watermark so
|
||||
/// the strategy selector can route large catalogs to S1 at IS-1 without
|
||||
/// first hitting N1's deep-offset wall (R7-15 Q4).
|
||||
pub server_track_count: Option<i64>,
|
||||
}
|
||||
|
||||
/// Run `CapabilityProbe::run` and persist the resulting flags +
|
||||
/// transition `sync_phase` from whatever it was to `probing` →
|
||||
/// `idle` (caller is responsible for advancing to `initial_sync` /
|
||||
/// `ready` once the appropriate runner starts).
|
||||
///
|
||||
/// PR-3d wires this in front of every initial / delta run so the
|
||||
/// stored `capability_flags` always reflects the current server.
|
||||
/// Returns the freshly resolved `(flags, server_info)` so callers
|
||||
/// can pick their `IngestStrategy` without re-reading SQLite.
|
||||
pub async fn probe_and_persist(
|
||||
store: &crate::store::LibraryStore,
|
||||
subsonic: &psysonic_integration::subsonic::SubsonicClient,
|
||||
navidrome: Option<&NavidromeProbeCredentials>,
|
||||
server_id: &str,
|
||||
library_scope: &str,
|
||||
) -> Result<CapabilityProbeResult, psysonic_integration::subsonic::SubsonicError> {
|
||||
let sync_state = crate::repos::SyncStateRepository::new(store);
|
||||
sync_state
|
||||
.ensure(server_id, library_scope)
|
||||
.map_err(psysonic_integration::subsonic::SubsonicError::Transport)?;
|
||||
let phase_before = sync_state
|
||||
.get_sync_phase(server_id, library_scope)
|
||||
.map_err(psysonic_integration::subsonic::SubsonicError::Transport)?;
|
||||
sync_state
|
||||
.set_sync_phase(server_id, library_scope, "probing")
|
||||
.map_err(psysonic_integration::subsonic::SubsonicError::Transport)?;
|
||||
|
||||
let existing_flags = sync_state
|
||||
.get_capability_flags(server_id, library_scope)
|
||||
.map_err(psysonic_integration::subsonic::SubsonicError::Transport)?
|
||||
.unwrap_or(0);
|
||||
|
||||
let mut result = CapabilityProbe::run(subsonic, navidrome).await?;
|
||||
|
||||
// R7-15 Q3: a probe run without a Navidrome bearer can't test N1, so it
|
||||
// must not drop a previously-learned NavidromeNativeBulk capability — the
|
||||
// server still supports `/api/song`; only the token is missing this bind.
|
||||
// Token availability gates actual N1 use per run (see library_sync_start).
|
||||
if navidrome.is_none()
|
||||
&& existing_flags & CapabilityFlags::NAVIDROME_NATIVE_BULK != 0
|
||||
{
|
||||
result.flags.insert(CapabilityFlags::NAVIDROME_NATIVE_BULK);
|
||||
}
|
||||
|
||||
sync_state
|
||||
.set_capability_flags(server_id, library_scope, result.flags.bits())
|
||||
.map_err(psysonic_integration::subsonic::SubsonicError::Transport)?;
|
||||
// Refresh the track-count watermark only when the probe learned one — a
|
||||
// missing `getScanStatus.count` must not clobber a count from a prior run.
|
||||
if let Some(count) = result.server_track_count {
|
||||
sync_state
|
||||
.set_server_track_count(server_id, library_scope, count)
|
||||
.map_err(psysonic_integration::subsonic::SubsonicError::Transport)?;
|
||||
}
|
||||
sync_state
|
||||
.set_sync_phase(
|
||||
server_id,
|
||||
library_scope,
|
||||
match phase_before.as_deref() {
|
||||
// Re-bind on app restart must not clobber a finished index —
|
||||
// callers gate local search on `ready` (§9.3 / P8).
|
||||
Some("ready") => "ready",
|
||||
Some("initial_sync") => "initial_sync",
|
||||
Some("error") => "error",
|
||||
_ => {
|
||||
if sync_state
|
||||
.has_last_full_sync_at(server_id, library_scope)
|
||||
.map_err(psysonic_integration::subsonic::SubsonicError::Transport)?
|
||||
{
|
||||
"ready"
|
||||
} else {
|
||||
"idle"
|
||||
}
|
||||
}
|
||||
},
|
||||
)
|
||||
.map_err(psysonic_integration::subsonic::SubsonicError::Transport)?;
|
||||
|
||||
Ok(result)
|
||||
}
|
||||
|
||||
pub struct CapabilityProbe;
|
||||
|
||||
impl CapabilityProbe {
|
||||
/// Run the §6.1 probe chain. Returns the resolved flags plus the
|
||||
/// envelope metadata captured from the Subsonic ping.
|
||||
///
|
||||
/// The Subsonic ping is the only failure-blocking probe — if it
|
||||
/// returns `Err`, the server is unreachable / wrong creds / wrong
|
||||
/// URL, and no other capability can be determined. Every other
|
||||
/// probe is best-effort: it sets its flag on success and leaves it
|
||||
/// clear on any error.
|
||||
pub async fn run(
|
||||
subsonic: &SubsonicClient,
|
||||
navidrome: Option<&NavidromeProbeCredentials>,
|
||||
) -> Result<CapabilityProbeResult, SubsonicError> {
|
||||
let server_info = subsonic.server_info().await?;
|
||||
|
||||
let mut flags = CapabilityFlags::default();
|
||||
|
||||
if server_info.open_subsonic {
|
||||
flags.insert(CapabilityFlags::OPEN_SUBSONIC);
|
||||
}
|
||||
// Navidrome rebuilds its track id space on full re-scan; spec
|
||||
// §6.9 / P33 makes the remap pass mandatory for those servers.
|
||||
if matches!(server_info.server_type.as_deref(), Some("navidrome")) {
|
||||
flags.insert(CapabilityFlags::UNSTABLE_TRACK_IDS);
|
||||
}
|
||||
|
||||
// `search3` with songCount=1 is the cheapest way to confirm the
|
||||
// bulk-ingest endpoint is usable on this server (Navidrome
|
||||
// accepts empty query; some forks reject it).
|
||||
if subsonic.search3("", 1, 0, None).await.is_ok() {
|
||||
flags.insert(CapabilityFlags::SUBSONIC_SEARCH3_BULK);
|
||||
}
|
||||
|
||||
let mut server_track_count = None;
|
||||
if let Ok(scan) = subsonic.get_scan_status().await {
|
||||
flags.insert(CapabilityFlags::SCAN_STATUS_AVAILABLE);
|
||||
// Only a positive count is a usable watermark; a scan in progress
|
||||
// can report 0, which we treat as "unknown" rather than "empty".
|
||||
server_track_count = scan.count.filter(|&c| c > 0);
|
||||
}
|
||||
|
||||
if subsonic.get_indexes(None, None).await.is_ok() {
|
||||
flags.insert(CapabilityFlags::FILE_TREE_BROWSE);
|
||||
}
|
||||
|
||||
if let Some(creds) = navidrome {
|
||||
match native_bulk_available(&creds.server_url, &creds.bearer_token).await {
|
||||
Ok(true) => flags.insert(CapabilityFlags::NAVIDROME_NATIVE_BULK),
|
||||
Ok(false) => {}
|
||||
Err(_) => {
|
||||
// Probe transport failed but Subsonic ping worked —
|
||||
// assume the native endpoint is unavailable for this
|
||||
// setup and let sync fall back to S1/S2.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Ok(CapabilityProbeResult {
|
||||
flags,
|
||||
server_info,
|
||||
server_track_count,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use psysonic_integration::subsonic::{SubsonicClient, SubsonicCredentials};
|
||||
use serde_json::json;
|
||||
use wiremock::matchers::{header, method as wm_method, path as wm_path};
|
||||
use wiremock::{Mock, MockServer, ResponseTemplate};
|
||||
|
||||
// ── CapabilityFlags bitfield ─────────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn capability_flags_contains_respects_individual_bits() {
|
||||
let mut f = CapabilityFlags::default();
|
||||
assert!(!f.contains(CapabilityFlags::OPEN_SUBSONIC));
|
||||
f.insert(CapabilityFlags::OPEN_SUBSONIC);
|
||||
assert!(f.contains(CapabilityFlags::OPEN_SUBSONIC));
|
||||
assert!(!f.contains(CapabilityFlags::NAVIDROME_NATIVE_BULK));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn capability_flags_insert_is_idempotent() {
|
||||
let mut f = CapabilityFlags::default();
|
||||
f.insert(CapabilityFlags::SUBSONIC_SEARCH3_BULK);
|
||||
let after_first = f.bits();
|
||||
f.insert(CapabilityFlags::SUBSONIC_SEARCH3_BULK);
|
||||
assert_eq!(f.bits(), after_first);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn capability_flags_remove_clears_only_the_named_bit() {
|
||||
let mut f = CapabilityFlags::new(
|
||||
CapabilityFlags::OPEN_SUBSONIC | CapabilityFlags::UNSTABLE_TRACK_IDS,
|
||||
);
|
||||
f.remove(CapabilityFlags::OPEN_SUBSONIC);
|
||||
assert!(!f.contains(CapabilityFlags::OPEN_SUBSONIC));
|
||||
assert!(f.contains(CapabilityFlags::UNSTABLE_TRACK_IDS));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn capability_flags_bit_values_match_spec_table() {
|
||||
// Spec §6.1.1 hex values — pin the wire format so future
|
||||
// schema-migration writers don't shift them silently.
|
||||
assert_eq!(CapabilityFlags::NAVIDROME_NATIVE_BULK, 0x001);
|
||||
assert_eq!(CapabilityFlags::SUBSONIC_SEARCH3_BULK, 0x002);
|
||||
assert_eq!(CapabilityFlags::SCAN_STATUS_AVAILABLE, 0x004);
|
||||
assert_eq!(CapabilityFlags::OPEN_SUBSONIC, 0x008);
|
||||
assert_eq!(CapabilityFlags::UNSTABLE_TRACK_IDS, 0x010);
|
||||
assert_eq!(CapabilityFlags::FILE_TREE_BROWSE, 0x020);
|
||||
}
|
||||
|
||||
// ── CapabilityProbe wiremock harness ─────────────────────────────────
|
||||
|
||||
fn test_subsonic_client(uri: &str) -> SubsonicClient {
|
||||
SubsonicClient::with_static_credentials(
|
||||
uri,
|
||||
SubsonicCredentials::with_static("user", "tok", "salt"),
|
||||
reqwest::Client::new(),
|
||||
)
|
||||
}
|
||||
|
||||
fn ok_envelope(body_key: &str, body: serde_json::Value) -> serde_json::Value {
|
||||
json!({
|
||||
"subsonic-response": {
|
||||
"status": "ok",
|
||||
"version": "1.16.1",
|
||||
body_key: body,
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
async fn mount_subsonic_full_navidrome(server: &MockServer) {
|
||||
// ping → navidrome + openSubsonic
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/ping.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": {
|
||||
"status": "ok",
|
||||
"version": "1.16.1",
|
||||
"type": "navidrome",
|
||||
"serverVersion": "0.55.2",
|
||||
"openSubsonic": true
|
||||
}
|
||||
})))
|
||||
.mount(server)
|
||||
.await;
|
||||
// search3 empty query
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/search3.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(ok_envelope(
|
||||
"searchResult3",
|
||||
json!({ "song": [{ "id": "x", "title": "y" }] }),
|
||||
)))
|
||||
.mount(server)
|
||||
.await;
|
||||
// getScanStatus
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getScanStatus.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(ok_envelope(
|
||||
"scanStatus",
|
||||
json!({ "scanning": false }),
|
||||
)))
|
||||
.mount(server)
|
||||
.await;
|
||||
// getIndexes
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getIndexes.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(ok_envelope(
|
||||
"indexes",
|
||||
json!({ "lastModified": 0, "ignoredArticles": "", "index": [] }),
|
||||
)))
|
||||
.mount(server)
|
||||
.await;
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_sets_all_subsonic_bits_on_a_fully_capable_navidrome_server() {
|
||||
let server = MockServer::start().await;
|
||||
mount_subsonic_full_navidrome(&server).await;
|
||||
|
||||
let result = CapabilityProbe::run(&test_subsonic_client(&server.uri()), None)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(result.flags.contains(CapabilityFlags::SUBSONIC_SEARCH3_BULK));
|
||||
assert!(result.flags.contains(CapabilityFlags::SCAN_STATUS_AVAILABLE));
|
||||
assert!(result.flags.contains(CapabilityFlags::FILE_TREE_BROWSE));
|
||||
assert!(result.flags.contains(CapabilityFlags::OPEN_SUBSONIC));
|
||||
assert!(result.flags.contains(CapabilityFlags::UNSTABLE_TRACK_IDS));
|
||||
// No navidrome probe creds passed → N1 stays clear.
|
||||
assert!(!result.flags.contains(CapabilityFlags::NAVIDROME_NATIVE_BULK));
|
||||
assert_eq!(result.server_info.server_type.as_deref(), Some("navidrome"));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_returns_err_when_subsonic_ping_fails() {
|
||||
let server = MockServer::start().await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/ping.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": {
|
||||
"status": "failed",
|
||||
"error": { "code": 40, "message": "Wrong credentials" }
|
||||
}
|
||||
})))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
let err = CapabilityProbe::run(&test_subsonic_client(&server.uri()), None)
|
||||
.await
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, SubsonicError::Api { code: 40, .. }));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_keeps_optional_bits_clear_when_their_endpoint_fails() {
|
||||
// Minimal Subsonic-like server: ping ok, search3 ok, but
|
||||
// scanStatus + getIndexes 4xx. UnstableTrackIds + OpenSubsonic
|
||||
// stay clear because the ping envelope omits them.
|
||||
let server = MockServer::start().await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/ping.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": { "status": "ok", "version": "1.13" }
|
||||
})))
|
||||
.mount(&server)
|
||||
.await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/search3.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(ok_envelope(
|
||||
"searchResult3",
|
||||
json!({}),
|
||||
)))
|
||||
.mount(&server)
|
||||
.await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getScanStatus.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": {
|
||||
"status": "failed",
|
||||
"error": { "code": 30, "message": "Method not available" }
|
||||
}
|
||||
})))
|
||||
.mount(&server)
|
||||
.await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getIndexes.view"))
|
||||
.respond_with(ResponseTemplate::new(500))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
let result = CapabilityProbe::run(&test_subsonic_client(&server.uri()), None)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(result.flags.contains(CapabilityFlags::SUBSONIC_SEARCH3_BULK));
|
||||
assert!(!result.flags.contains(CapabilityFlags::SCAN_STATUS_AVAILABLE));
|
||||
assert!(!result.flags.contains(CapabilityFlags::FILE_TREE_BROWSE));
|
||||
assert!(!result.flags.contains(CapabilityFlags::OPEN_SUBSONIC));
|
||||
assert!(!result.flags.contains(CapabilityFlags::UNSTABLE_TRACK_IDS));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_sets_navidrome_native_bulk_when_credentials_succeed() {
|
||||
let server = MockServer::start().await;
|
||||
mount_subsonic_full_navidrome(&server).await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/api/song"))
|
||||
.and(header("X-ND-Authorization", "Bearer nd-tok"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!([])))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
let nav = NavidromeProbeCredentials {
|
||||
server_url: server.uri(),
|
||||
bearer_token: "nd-tok".into(),
|
||||
};
|
||||
let result = CapabilityProbe::run(&test_subsonic_client(&server.uri()), Some(&nav))
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(result.flags.contains(CapabilityFlags::NAVIDROME_NATIVE_BULK));
|
||||
}
|
||||
|
||||
// ── probe_and_persist round-trip ──────────────────────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_and_persist_writes_flags_and_resets_phase_to_idle() {
|
||||
use crate::repos::SyncStateRepository;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
let server = MockServer::start().await;
|
||||
mount_subsonic_full_navidrome(&server).await;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let result = super::probe_and_persist(
|
||||
&store,
|
||||
&test_subsonic_client(&server.uri()),
|
||||
None,
|
||||
"s1",
|
||||
"",
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
let flags = sync_state.get_capability_flags("s1", "").unwrap().unwrap();
|
||||
assert_eq!(flags, result.flags.bits());
|
||||
assert!(flags & CapabilityFlags::OPEN_SUBSONIC != 0);
|
||||
assert!(flags & CapabilityFlags::UNSTABLE_TRACK_IDS != 0);
|
||||
|
||||
// Fresh server ends at `idle` so the caller can transition to
|
||||
// `initial_sync` / `ready` based on whether a sync is needed.
|
||||
assert_eq!(
|
||||
sync_state.get_sync_phase("s1", "").unwrap().as_deref(),
|
||||
Some("idle")
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_and_persist_preserves_ready_phase_on_rebind() {
|
||||
use crate::repos::SyncStateRepository;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
let server = MockServer::start().await;
|
||||
mount_subsonic_full_navidrome(&server).await;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
sync_state.ensure("s1", "").unwrap();
|
||||
sync_state.set_sync_phase("s1", "", "ready").unwrap();
|
||||
|
||||
super::probe_and_persist(
|
||||
&store,
|
||||
&test_subsonic_client(&server.uri()),
|
||||
None,
|
||||
"s1",
|
||||
"",
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(
|
||||
sync_state.get_sync_phase("s1", "").unwrap().as_deref(),
|
||||
Some("ready")
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_and_persist_promotes_idle_to_ready_when_full_sync_stamped() {
|
||||
use crate::repos::SyncStateRepository;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
let server = MockServer::start().await;
|
||||
mount_subsonic_full_navidrome(&server).await;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
sync_state.ensure("s1", "").unwrap();
|
||||
sync_state.set_sync_phase("s1", "", "idle").unwrap();
|
||||
sync_state
|
||||
.set_last_full_sync_at("s1", "", 1_716_000_000_000)
|
||||
.unwrap();
|
||||
|
||||
super::probe_and_persist(
|
||||
&store,
|
||||
&test_subsonic_client(&server.uri()),
|
||||
None,
|
||||
"s1",
|
||||
"",
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(
|
||||
sync_state.get_sync_phase("s1", "").unwrap().as_deref(),
|
||||
Some("ready")
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_captures_and_persists_scan_status_track_count() {
|
||||
use crate::repos::SyncStateRepository;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
let server = MockServer::start().await;
|
||||
// ping + search3 + getIndexes from the shared helper, then override
|
||||
// getScanStatus with a populated `count` (large library).
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/ping.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": { "status": "ok", "version": "1.16.1", "type": "navidrome" }
|
||||
})))
|
||||
.mount(&server)
|
||||
.await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/search3.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(ok_envelope(
|
||||
"searchResult3",
|
||||
json!({ "song": [{ "id": "x", "title": "y" }] }),
|
||||
)))
|
||||
.mount(&server)
|
||||
.await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getScanStatus.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(ok_envelope(
|
||||
"scanStatus",
|
||||
json!({ "scanning": false, "count": 170_000 }),
|
||||
)))
|
||||
.mount(&server)
|
||||
.await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getIndexes.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(ok_envelope(
|
||||
"indexes",
|
||||
json!({ "lastModified": 0, "ignoredArticles": "", "index": [] }),
|
||||
)))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let result =
|
||||
super::probe_and_persist(&store, &test_subsonic_client(&server.uri()), None, "s1", "")
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(result.server_track_count, Some(170_000));
|
||||
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
assert_eq!(
|
||||
sync_state.get_server_track_count("s1", "").unwrap(),
|
||||
Some(170_000)
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_preserves_navidrome_native_bulk_when_no_token_supplied() {
|
||||
use crate::repos::SyncStateRepository;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
// A prior bind (with a working bearer) already learned N1.
|
||||
sync_state
|
||||
.set_capability_flags("s1", "", CapabilityFlags::NAVIDROME_NATIVE_BULK)
|
||||
.unwrap();
|
||||
|
||||
let server = MockServer::start().await;
|
||||
mount_subsonic_full_navidrome(&server).await;
|
||||
|
||||
// Re-probe without a Navidrome token (transient /auth/login failure).
|
||||
// R7-15 Q3: the server still supports /api/song — the flag must stay.
|
||||
let result = super::probe_and_persist(
|
||||
&store,
|
||||
&test_subsonic_client(&server.uri()),
|
||||
None,
|
||||
"s1",
|
||||
"",
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(
|
||||
result.flags.contains(CapabilityFlags::NAVIDROME_NATIVE_BULK),
|
||||
"result must keep the previously-learned N1 capability"
|
||||
);
|
||||
let persisted = sync_state.get_capability_flags("s1", "").unwrap().unwrap();
|
||||
assert!(
|
||||
persisted & CapabilityFlags::NAVIDROME_NATIVE_BULK != 0,
|
||||
"persisted flags must keep N1 across a token-less probe"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_does_not_clobber_track_count_when_scan_status_omits_it() {
|
||||
use crate::repos::SyncStateRepository;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
// A prior run already learned the count.
|
||||
sync_state.set_server_track_count("s1", "", 52_000).unwrap();
|
||||
|
||||
let server = MockServer::start().await;
|
||||
mount_subsonic_full_navidrome(&server).await; // scanStatus has no count
|
||||
|
||||
let result =
|
||||
super::probe_and_persist(&store, &test_subsonic_client(&server.uri()), None, "s1", "")
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(result.server_track_count, None);
|
||||
// Watermark from the prior run survives the count-less probe.
|
||||
assert_eq!(
|
||||
sync_state.get_server_track_count("s1", "").unwrap(),
|
||||
Some(52_000)
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn probe_leaves_navidrome_native_bulk_clear_when_endpoint_404s() {
|
||||
let server = MockServer::start().await;
|
||||
mount_subsonic_full_navidrome(&server).await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/api/song"))
|
||||
.respond_with(ResponseTemplate::new(404))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
let nav = NavidromeProbeCredentials {
|
||||
server_url: server.uri(),
|
||||
bearer_token: "nd-tok".into(),
|
||||
};
|
||||
let result = CapabilityProbe::run(&test_subsonic_client(&server.uri()), Some(&nav))
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(!result.flags.contains(CapabilityFlags::NAVIDROME_NATIVE_BULK));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,157 @@
|
||||
//! Cursor shape persisted in `sync_state.initial_sync_cursor_json`.
|
||||
//! Resume after a restart, kill, or app crash deserializes this value
|
||||
//! and tells the runner where to pick up. Strategy is recorded so a
|
||||
//! cap-flag change between runs is detected and the stale cursor is
|
||||
//! reset — the runner restarts ingest under the newly-selected strategy
|
||||
//! rather than resuming the wrong loop.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use super::strategy::IngestStrategy;
|
||||
|
||||
/// Top-level cursor. `phase` advances `Ingest → ArtistPass → Watermarks
|
||||
/// → Done`; each phase only reads the fields it owns.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
|
||||
pub struct InitialSyncCursor {
|
||||
/// Ingest strategy in flight — stored as the tag string so the JSON
|
||||
/// is human-readable on disk.
|
||||
pub strategy: String,
|
||||
pub phase: CursorPhase,
|
||||
/// Scope this run operates on (Navidrome `library_id` / Subsonic
|
||||
/// `musicFolderId`). `None` means "all libraries on this server".
|
||||
#[serde(default)]
|
||||
pub library_scope: Option<String>,
|
||||
/// Tracks ingested across the entire run so far — informational,
|
||||
/// matches the §6 progress event payload.
|
||||
#[serde(default)]
|
||||
pub ingested_count: u32,
|
||||
/// Per-strategy offset state. Discriminated by `strategy` so a
|
||||
/// future field add doesn't break old cursors.
|
||||
#[serde(default)]
|
||||
pub strategy_state: StrategyState,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum CursorPhase {
|
||||
Ingest,
|
||||
ArtistPass,
|
||||
Watermarks,
|
||||
Done,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, Default)]
|
||||
#[serde(tag = "kind", rename_all = "snake_case")]
|
||||
pub enum StrategyState {
|
||||
/// N1 / S1 — single linear offset.
|
||||
LinearOffset { offset: u32 },
|
||||
/// S2 — outer album-list offset plus optional in-flight album we
|
||||
/// were halfway through when interrupted.
|
||||
AlbumCrawl {
|
||||
album_offset: u32,
|
||||
#[serde(default)]
|
||||
current_album_id: Option<String>,
|
||||
},
|
||||
/// Fresh cursor with no progress yet.
|
||||
#[default]
|
||||
Empty,
|
||||
}
|
||||
|
||||
impl InitialSyncCursor {
|
||||
pub fn fresh(strategy: IngestStrategy, library_scope: Option<String>) -> Self {
|
||||
let strategy_state = match strategy {
|
||||
IngestStrategy::N1 | IngestStrategy::S1 => StrategyState::LinearOffset { offset: 0 },
|
||||
IngestStrategy::S2 => StrategyState::AlbumCrawl {
|
||||
album_offset: 0,
|
||||
current_album_id: None,
|
||||
},
|
||||
IngestStrategy::S3 => StrategyState::Empty,
|
||||
};
|
||||
Self {
|
||||
strategy: strategy.as_tag().to_string(),
|
||||
phase: CursorPhase::Ingest,
|
||||
library_scope,
|
||||
ingested_count: 0,
|
||||
strategy_state,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn strategy_tag(&self) -> &str {
|
||||
&self.strategy
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn fresh_n1_starts_at_offset_zero() {
|
||||
let c = InitialSyncCursor::fresh(IngestStrategy::N1, None);
|
||||
assert_eq!(c.phase, CursorPhase::Ingest);
|
||||
assert_eq!(c.ingested_count, 0);
|
||||
match c.strategy_state {
|
||||
StrategyState::LinearOffset { offset } => assert_eq!(offset, 0),
|
||||
other => panic!("expected LinearOffset, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fresh_s2_uses_album_crawl_state() {
|
||||
let c = InitialSyncCursor::fresh(IngestStrategy::S2, Some("lib-1".into()));
|
||||
assert_eq!(c.library_scope.as_deref(), Some("lib-1"));
|
||||
match c.strategy_state {
|
||||
StrategyState::AlbumCrawl { album_offset, current_album_id } => {
|
||||
assert_eq!(album_offset, 0);
|
||||
assert!(current_album_id.is_none());
|
||||
}
|
||||
other => panic!("expected AlbumCrawl, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cursor_roundtrips_through_json() {
|
||||
// The cursor lives as TEXT in `sync_state.initial_sync_cursor_json`;
|
||||
// serde must keep the round trip stable for resume to work.
|
||||
let c = InitialSyncCursor {
|
||||
strategy: "n1".into(),
|
||||
phase: CursorPhase::Ingest,
|
||||
library_scope: Some("lib-1".into()),
|
||||
ingested_count: 2500,
|
||||
strategy_state: StrategyState::LinearOffset { offset: 2500 },
|
||||
};
|
||||
let json = serde_json::to_value(&c).unwrap();
|
||||
let back: InitialSyncCursor = serde_json::from_value(json.clone()).unwrap();
|
||||
assert_eq!(c, back);
|
||||
|
||||
// strategy_state internally tagged with `"kind"`.
|
||||
assert_eq!(
|
||||
json.get("strategy_state").and_then(|s| s.get("kind")),
|
||||
Some(&json!("linear_offset"))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cursor_deserialize_tolerates_omitted_defaults() {
|
||||
// Minimal cursor produced by a much older client must still
|
||||
// deserialize as a "fresh ingest" state.
|
||||
let raw = json!({
|
||||
"strategy": "s1",
|
||||
"phase": "ingest"
|
||||
});
|
||||
let c: InitialSyncCursor = serde_json::from_value(raw).unwrap();
|
||||
assert_eq!(c.ingested_count, 0);
|
||||
assert_eq!(c.library_scope, None);
|
||||
assert!(matches!(c.strategy_state, StrategyState::Empty));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_object_does_not_parse_as_cursor() {
|
||||
// sync_state.initial_sync_cursor_json default is `'{}'`; that
|
||||
// value is not a valid cursor (no strategy / phase). Runner
|
||||
// must treat it as "no cursor → start fresh".
|
||||
let raw = json!({});
|
||||
assert!(serde_json::from_value::<InitialSyncCursor>(raw).is_err());
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,149 @@
|
||||
//! Errors surfaced by the sync orchestrator. Designed so callers
|
||||
//! (PR-5 Tauri command surface) can pattern-match on the variant for
|
||||
//! UI affordances — `Cancelled` is silent, `Transport`/`HttpStatus`
|
||||
//! retry, `Subsonic { code: 70, .. }` flows into the tombstone path,
|
||||
//! and `StrategyUnsupported` surfaces a Settings hint.
|
||||
|
||||
use std::fmt;
|
||||
|
||||
use psysonic_integration::subsonic::SubsonicError;
|
||||
|
||||
#[derive(Debug)]
|
||||
pub enum SyncError {
|
||||
/// Network / TLS / DNS failure surfaced by the Subsonic or
|
||||
/// Navidrome HTTP client. Retryable per §6.8.
|
||||
Transport(String),
|
||||
|
||||
/// Subsonic-level error after the envelope parsed cleanly. The
|
||||
/// dedicated `NotFound` (error code 70) is kept inline rather
|
||||
/// than collapsed into a string so the tombstone path can match.
|
||||
Subsonic { code: i32, message: String },
|
||||
|
||||
/// Subsonic returned error code 70 — track missing from the
|
||||
/// server. Tombstone reconciler matches on this variant directly.
|
||||
NotFound,
|
||||
|
||||
/// Navidrome native REST returned a non-success status; carries
|
||||
/// the flattened HTTP error string from `nd_err`.
|
||||
Navidrome(String),
|
||||
|
||||
/// Persistence layer (SQLite) failure.
|
||||
Storage(String),
|
||||
|
||||
/// Strategy is enumerated but not implemented for v1
|
||||
/// (currently only `S3`).
|
||||
StrategyUnsupported {
|
||||
strategy: &'static str,
|
||||
},
|
||||
|
||||
/// Cancellation token tripped — caller asked us to abort.
|
||||
/// Cursor stays where it was so the next run resumes the batch.
|
||||
Cancelled,
|
||||
|
||||
/// Cursor JSON in `sync_state` is from an incompatible strategy
|
||||
/// (e.g. switching from N1 to S2 mid-sync). Caller should clear
|
||||
/// the cursor and restart initial sync.
|
||||
CursorIncompatible {
|
||||
expected: &'static str,
|
||||
actual: String,
|
||||
},
|
||||
}
|
||||
|
||||
impl fmt::Display for SyncError {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
match self {
|
||||
Self::Transport(m) => write!(f, "sync transport: {m}"),
|
||||
Self::Subsonic { code, message } => write!(f, "subsonic {code}: {message}"),
|
||||
Self::NotFound => write!(f, "subsonic: not found (code 70)"),
|
||||
Self::Navidrome(m) => write!(f, "navidrome: {m}"),
|
||||
Self::Storage(m) => write!(f, "storage: {m}"),
|
||||
Self::StrategyUnsupported { strategy } => {
|
||||
write!(f, "ingest strategy not supported: {strategy}")
|
||||
}
|
||||
Self::Cancelled => write!(f, "sync cancelled"),
|
||||
Self::CursorIncompatible { expected, actual } => write!(
|
||||
f,
|
||||
"sync cursor strategy mismatch: cursor says `{actual}`, runner is `{expected}`"
|
||||
),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for SyncError {}
|
||||
|
||||
impl SyncError {
|
||||
/// Parsed HTTP status when this is a Navidrome native REST failure
|
||||
/// shaped like `HTTP 500` or `HTTP 500: body`.
|
||||
pub fn navidrome_http_status(&self) -> Option<u16> {
|
||||
let SyncError::Navidrome(msg) = self else {
|
||||
return None;
|
||||
};
|
||||
let rest = msg.strip_prefix("HTTP ")?.split(':').next()?.trim();
|
||||
rest.split_whitespace().next()?.parse().ok()
|
||||
}
|
||||
}
|
||||
|
||||
impl From<SubsonicError> for SyncError {
|
||||
fn from(e: SubsonicError) -> Self {
|
||||
match e {
|
||||
SubsonicError::Transport(m) => Self::Transport(m),
|
||||
SubsonicError::HttpStatus(s) => Self::Transport(format!("http {s}")),
|
||||
SubsonicError::Api { code, message } => Self::Subsonic { code, message },
|
||||
SubsonicError::NotFound => Self::NotFound,
|
||||
SubsonicError::Decode(m) => Self::Transport(format!("decode: {m}")),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn subsonic_not_found_maps_to_sync_not_found() {
|
||||
let e: SyncError = SubsonicError::NotFound.into();
|
||||
assert!(matches!(e, SyncError::NotFound));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn subsonic_api_error_carries_code_through() {
|
||||
let e: SyncError = SubsonicError::Api { code: 40, message: "bad creds".into() }.into();
|
||||
match e {
|
||||
SyncError::Subsonic { code, message } => {
|
||||
assert_eq!(code, 40);
|
||||
assert!(message.contains("bad creds"));
|
||||
}
|
||||
other => panic!("expected Subsonic, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn http_status_collapses_into_transport() {
|
||||
let e: SyncError = SubsonicError::HttpStatus(reqwest::StatusCode::SERVICE_UNAVAILABLE).into();
|
||||
assert!(matches!(e, SyncError::Transport(ref m) if m.contains("503")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn navidrome_http_status_parses_status_line() {
|
||||
let e = SyncError::Navidrome("HTTP 500".into());
|
||||
assert_eq!(e.navidrome_http_status(), Some(500));
|
||||
let with_body = SyncError::Navidrome("HTTP 503: upstream timeout".into());
|
||||
assert_eq!(with_body.navidrome_http_status(), Some(503));
|
||||
let with_reason = SyncError::Navidrome("HTTP 500 Internal Server Error".into());
|
||||
assert_eq!(with_reason.navidrome_http_status(), Some(500));
|
||||
assert_eq!(
|
||||
SyncError::Transport("http 500".into()).navidrome_http_status(),
|
||||
None
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cursor_incompatible_renders_both_strategies() {
|
||||
let s = SyncError::CursorIncompatible {
|
||||
expected: "n1",
|
||||
actual: "s2".into(),
|
||||
}
|
||||
.to_string();
|
||||
assert!(s.contains("n1") && s.contains("s2"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,226 @@
|
||||
//! Concurrent fetch helpers for initial-sync ingest (C11 parallelism budget).
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::future::Future;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use psysonic_integration::subsonic::{Album, SubsonicClient};
|
||||
use serde_json::Value;
|
||||
use tokio::sync::Semaphore;
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use super::backoff::{jitter_salt, with_jitter, Backoff};
|
||||
use super::bandwidth::ParallelismBudget;
|
||||
use super::error::SyncError;
|
||||
|
||||
const MAX_ATTEMPTS_PER_BATCH: u32 = 5;
|
||||
|
||||
pub fn check_cancel_flag(cancel: &Option<Arc<std::sync::atomic::AtomicBool>>) -> Result<(), SyncError> {
|
||||
if cancel.as_ref().is_some_and(|f| f.load(Ordering::SeqCst)) {
|
||||
return Err(SyncError::Cancelled);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn is_retryable(e: &SyncError) -> bool {
|
||||
matches!(e, SyncError::Transport(_) | SyncError::Navidrome(_))
|
||||
}
|
||||
|
||||
/// How many linear pages to keep in flight (`1` = sequential).
|
||||
pub fn linear_prefetch_depth(budget: &ParallelismBudget) -> usize {
|
||||
if budget.bulk_paused() {
|
||||
return 1;
|
||||
}
|
||||
budget.max_concurrent.max(1) as usize
|
||||
}
|
||||
|
||||
pub async fn sleep_request_gap(budget: &ParallelismBudget, sleep_enabled: bool) {
|
||||
if sleep_enabled && budget.min_request_gap_ms > 0 {
|
||||
tokio::time::sleep(Duration::from_millis(budget.min_request_gap_ms as u64)).await;
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn wait_while_bulk_paused(
|
||||
budget: &ParallelismBudget,
|
||||
sleep_enabled: bool,
|
||||
mut check_cancel: impl FnMut() -> Result<(), SyncError>,
|
||||
) -> Result<(), SyncError> {
|
||||
while budget.bulk_paused() {
|
||||
check_cancel()?;
|
||||
if sleep_enabled {
|
||||
tokio::time::sleep(Duration::from_millis(500)).await;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Standalone retry loop for spawned prefetch tasks (no `InitialSyncRunner` ref).
|
||||
pub async fn retry_fetch<T, F, Fut, E>(
|
||||
sleep_enabled: bool,
|
||||
mut check_cancel: impl FnMut() -> Result<(), SyncError>,
|
||||
mut build: F,
|
||||
map_err: impl Fn(E) -> SyncError,
|
||||
) -> Result<T, SyncError>
|
||||
where
|
||||
F: FnMut() -> Fut,
|
||||
Fut: Future<Output = Result<T, E>>,
|
||||
{
|
||||
let mut backoff = Backoff::default();
|
||||
let mut attempt = 0u32;
|
||||
loop {
|
||||
check_cancel()?;
|
||||
attempt += 1;
|
||||
match build().await {
|
||||
Ok(v) => return Ok(v),
|
||||
Err(e) => {
|
||||
let mapped = map_err(e);
|
||||
if !is_retryable(&mapped) || attempt >= MAX_ATTEMPTS_PER_BATCH {
|
||||
return Err(mapped);
|
||||
}
|
||||
let delay = backoff.next_delay();
|
||||
let jittered = with_jitter(delay, jitter_salt(attempt));
|
||||
if sleep_enabled && !jittered.is_zero() {
|
||||
tokio::time::sleep(jittered).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Ordered in-flight page queue for N1/S1 linear ingest.
|
||||
pub struct LinearPrefetchQueue<T> {
|
||||
depth: usize,
|
||||
batch_size: u32,
|
||||
next_enqueue_offset: u32,
|
||||
exhausted: bool,
|
||||
inflight: BTreeMap<u32, JoinHandle<Result<T, SyncError>>>,
|
||||
}
|
||||
|
||||
impl<T: Send + 'static> LinearPrefetchQueue<T> {
|
||||
pub fn new(budget: &ParallelismBudget, batch_size: u32, start_offset: u32) -> Self {
|
||||
Self {
|
||||
depth: linear_prefetch_depth(budget),
|
||||
batch_size,
|
||||
next_enqueue_offset: start_offset,
|
||||
exhausted: false,
|
||||
inflight: BTreeMap::new(),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn mark_exhausted(&mut self) {
|
||||
self.exhausted = true;
|
||||
}
|
||||
|
||||
pub fn pump<F>(
|
||||
&mut self,
|
||||
mut check_cancel: impl FnMut() -> Result<(), SyncError>,
|
||||
mut spawn: F,
|
||||
) -> Result<(), SyncError>
|
||||
where
|
||||
F: FnMut(u32) -> JoinHandle<Result<T, SyncError>>,
|
||||
{
|
||||
while !self.exhausted && self.inflight.len() < self.depth {
|
||||
check_cancel()?;
|
||||
let offset = self.next_enqueue_offset;
|
||||
self.next_enqueue_offset = offset.saturating_add(self.batch_size);
|
||||
let handle = spawn(offset);
|
||||
self.inflight.insert(offset, handle);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub async fn take_at(
|
||||
&mut self,
|
||||
offset: u32,
|
||||
mut check_cancel: impl FnMut() -> Result<(), SyncError>,
|
||||
) -> Result<Option<T>, SyncError> {
|
||||
check_cancel()?;
|
||||
let Some(handle) = self.inflight.remove(&offset) else {
|
||||
return Ok(None);
|
||||
};
|
||||
let value = handle
|
||||
.await
|
||||
.map_err(|e| SyncError::Transport(format!("prefetch join: {e}")))??;
|
||||
Ok(Some(value))
|
||||
}
|
||||
}
|
||||
|
||||
/// Fetch `getAlbum` bodies for a page with at most `budget.max_concurrent`
|
||||
/// requests in flight. Results preserve input order.
|
||||
#[derive(Clone)]
|
||||
pub struct ParallelAlbumFetchOpts {
|
||||
pub budget: ParallelismBudget,
|
||||
pub sleep_enabled: bool,
|
||||
pub cancel: Option<Arc<AtomicBool>>,
|
||||
}
|
||||
|
||||
pub async fn fetch_albums_parallel(
|
||||
subsonic: &SubsonicClient,
|
||||
album_ids: &[String],
|
||||
opts: ParallelAlbumFetchOpts,
|
||||
) -> Result<Vec<(Album, Value)>, SyncError> {
|
||||
if album_ids.is_empty() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
wait_while_bulk_paused(&opts.budget, opts.sleep_enabled, || check_cancel_flag(&opts.cancel)).await?;
|
||||
let max = opts.budget.max_concurrent.max(1) as usize;
|
||||
let client = subsonic.clone();
|
||||
let sem = Arc::new(Semaphore::new(max));
|
||||
let mut handles: Vec<JoinHandle<Result<(Album, Value), SyncError>>> =
|
||||
Vec::with_capacity(album_ids.len());
|
||||
|
||||
for id in album_ids {
|
||||
check_cancel_flag(&opts.cancel)?;
|
||||
sleep_request_gap(&opts.budget, opts.sleep_enabled).await;
|
||||
let permit = sem
|
||||
.clone()
|
||||
.acquire_owned()
|
||||
.await
|
||||
.map_err(|_| SyncError::Transport("parallel fetch semaphore closed".into()))?;
|
||||
let client = client.clone();
|
||||
let id = id.clone();
|
||||
let cancel = opts.cancel.clone();
|
||||
let sleep_enabled = opts.sleep_enabled;
|
||||
handles.push(tokio::spawn(async move {
|
||||
let _permit = permit;
|
||||
retry_fetch(
|
||||
sleep_enabled,
|
||||
|| check_cancel_flag(&cancel),
|
||||
|| async {
|
||||
client
|
||||
.get_album_with_raw(&id)
|
||||
.await
|
||||
.map_err(SyncError::from)
|
||||
},
|
||||
|e| e,
|
||||
)
|
||||
.await
|
||||
}));
|
||||
}
|
||||
|
||||
let mut out = Vec::with_capacity(handles.len());
|
||||
for handle in handles {
|
||||
check_cancel_flag(&opts.cancel)?;
|
||||
out.push(
|
||||
handle
|
||||
.await
|
||||
.map_err(|e| SyncError::Transport(format!("parallel album fetch join: {e}")))??
|
||||
);
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn linear_prefetch_depth_respects_budget() {
|
||||
let idle = ParallelismBudget::resolve(super::super::bandwidth::PlaybackHint::Idle);
|
||||
assert_eq!(linear_prefetch_depth(&idle), 4);
|
||||
let playing = ParallelismBudget::resolve(super::super::bandwidth::PlaybackHint::Playing);
|
||||
assert_eq!(linear_prefetch_depth(&playing), 1);
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,303 @@
|
||||
//! Subsonic / Navidrome song JSON → `TrackRow`. PR-3b's ingest paths
|
||||
//! all feed the same upsert API, so the projection happens here once.
|
||||
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::repos::TrackRow;
|
||||
use psysonic_integration::subsonic::Song;
|
||||
|
||||
/// Project a Subsonic `Song` plus its raw JSON sub-tree into a
|
||||
/// `TrackRow`. `raw_value` is what `track.raw_json` stores verbatim so
|
||||
/// OpenSubsonic extensions survive (spec §5.1 / ADR-7).
|
||||
pub fn subsonic_song_to_track_row(
|
||||
server_id: &str,
|
||||
song: &Song,
|
||||
raw_value: &Value,
|
||||
synced_at: i64,
|
||||
library_id_fallback: Option<&str>,
|
||||
) -> TrackRow {
|
||||
TrackRow {
|
||||
server_id: server_id.to_string(),
|
||||
id: song.id.clone(),
|
||||
title: song.title.clone(),
|
||||
title_sort: None,
|
||||
artist: song.artist.clone(),
|
||||
artist_id: song.artist_id.clone(),
|
||||
album: song.album.clone().unwrap_or_default(),
|
||||
album_id: song.album_id.clone(),
|
||||
album_artist: song.album_artist.clone(),
|
||||
duration_sec: song.duration.unwrap_or(0),
|
||||
track_number: song.track_number,
|
||||
disc_number: song.disc_number,
|
||||
year: song.year,
|
||||
genre: song.genre.clone(),
|
||||
suffix: song.suffix.clone(),
|
||||
bit_rate: song.bit_rate,
|
||||
size_bytes: song.size,
|
||||
cover_art_id: song.cover_art.clone(),
|
||||
starred_at: parse_iso_ms(song.starred.as_deref()),
|
||||
user_rating: song.user_rating,
|
||||
play_count: song.play_count,
|
||||
played_at: parse_iso_ms(song.played.as_deref()),
|
||||
server_path: song.path.clone(),
|
||||
library_id: song.library_id.clone().or_else(|| library_id_fallback.map(String::from)),
|
||||
isrc: song.isrc.clone(),
|
||||
mbid_recording: song.mbid_recording.clone(),
|
||||
bpm: song.bpm,
|
||||
replay_gain_track_db: raw_value
|
||||
.get("replayGain")
|
||||
.and_then(|rg| rg.get("trackGain"))
|
||||
.and_then(|v| v.as_f64()),
|
||||
replay_gain_album_db: raw_value
|
||||
.get("replayGain")
|
||||
.and_then(|rg| rg.get("albumGain"))
|
||||
.and_then(|v| v.as_f64()),
|
||||
content_hash: None,
|
||||
server_updated_at: None,
|
||||
server_created_at: None,
|
||||
deleted: false,
|
||||
synced_at,
|
||||
raw_json: raw_value.to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Project a Navidrome `/api/song` row (native REST shape) into a
|
||||
/// `TrackRow`. Field names mostly overlap with Subsonic but use
|
||||
/// snake_case JSON aliases — we read fields by `get(name)` rather
|
||||
/// than reusing the Subsonic `Song` deserializer so a server-side
|
||||
/// rename doesn't silently zero out hot columns.
|
||||
pub fn navidrome_song_to_track_row(
|
||||
server_id: &str,
|
||||
raw: &Value,
|
||||
synced_at: i64,
|
||||
library_id_fallback: Option<&str>,
|
||||
) -> Option<TrackRow> {
|
||||
let id = raw.get("id").and_then(|v| v.as_str())?.to_string();
|
||||
let title = raw
|
||||
.get("title")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or_default()
|
||||
.to_string();
|
||||
let server_updated_at = raw
|
||||
.get("updatedAt")
|
||||
.and_then(|v| v.as_str())
|
||||
.and_then(parse_iso_ms_str);
|
||||
let library_id = json_string_field(raw, "libraryId")
|
||||
.or_else(|| json_string_field(raw, "library_id"))
|
||||
.or_else(|| json_string_field(raw, "musicFolderId"))
|
||||
.or_else(|| library_id_fallback.map(String::from));
|
||||
Some(TrackRow {
|
||||
server_id: server_id.to_string(),
|
||||
id,
|
||||
title,
|
||||
title_sort: string_field(raw, "sortTitle").or_else(|| string_field(raw, "orderTitle")),
|
||||
artist: string_field(raw, "artist"),
|
||||
artist_id: string_field(raw, "artistId"),
|
||||
album: string_field(raw, "album").unwrap_or_default(),
|
||||
album_id: string_field(raw, "albumId"),
|
||||
album_artist: string_field(raw, "albumArtist"),
|
||||
duration_sec: raw.get("duration").and_then(|v| v.as_i64()).unwrap_or(0),
|
||||
track_number: raw.get("trackNumber").and_then(|v| v.as_i64()),
|
||||
disc_number: raw.get("discNumber").and_then(|v| v.as_i64()),
|
||||
year: raw.get("year").and_then(|v| v.as_i64()),
|
||||
genre: string_field(raw, "genre"),
|
||||
suffix: string_field(raw, "suffix"),
|
||||
bit_rate: raw.get("bitRate").and_then(|v| v.as_i64()),
|
||||
size_bytes: raw.get("size").and_then(|v| v.as_i64()),
|
||||
cover_art_id: string_field(raw, "coverArtId").or_else(|| string_field(raw, "coverArt")),
|
||||
starred_at: raw.get("starredAt").and_then(|v| v.as_str()).and_then(parse_iso_ms_str),
|
||||
user_rating: raw.get("rating").and_then(|v| v.as_i64()),
|
||||
play_count: raw.get("playCount").and_then(|v| v.as_i64()),
|
||||
played_at: raw.get("playedAt").and_then(|v| v.as_str()).and_then(parse_iso_ms_str),
|
||||
server_path: string_field(raw, "path"),
|
||||
library_id,
|
||||
isrc: string_field(raw, "isrc"),
|
||||
mbid_recording: string_field(raw, "mbzTrackId").or_else(|| string_field(raw, "musicBrainzId")),
|
||||
bpm: raw.get("bpm").and_then(|v| v.as_i64()),
|
||||
replay_gain_track_db: raw.get("rgTrackGain").and_then(|v| v.as_f64()),
|
||||
replay_gain_album_db: raw.get("rgAlbumGain").and_then(|v| v.as_f64()),
|
||||
content_hash: None,
|
||||
server_updated_at,
|
||||
server_created_at: raw
|
||||
.get("createdAt")
|
||||
.and_then(|v| v.as_str())
|
||||
.and_then(parse_iso_ms_str),
|
||||
deleted: false,
|
||||
synced_at,
|
||||
raw_json: raw.to_string(),
|
||||
})
|
||||
}
|
||||
|
||||
fn json_string_field(raw: &Value, key: &str) -> Option<String> {
|
||||
match raw.get(key)? {
|
||||
Value::String(s) => Some(s.clone()),
|
||||
Value::Number(n) => Some(n.to_string()),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
fn string_field(raw: &Value, key: &str) -> Option<String> {
|
||||
json_string_field(raw, key)
|
||||
}
|
||||
|
||||
fn parse_iso_ms(s: Option<&str>) -> Option<i64> {
|
||||
s.and_then(parse_iso_ms_str)
|
||||
}
|
||||
|
||||
/// Lightweight ISO-8601 → epoch-ms parser. Supports the Navidrome /
|
||||
/// OpenSubsonic shape (`2024-06-01T12:00:00Z` or
|
||||
/// `2024-06-01T12:00:00.123+02:00`). Falls back to `None` on parse
|
||||
/// failure — sync code never panics on a bad timestamp.
|
||||
fn parse_iso_ms_str(s: &str) -> Option<i64> {
|
||||
// Strip fractional + timezone before doing the manual parse —
|
||||
// SQLite stores starred_at / played_at as integer ms, so we only
|
||||
// need second precision rounded up from the offset.
|
||||
let trimmed = s.trim();
|
||||
if trimmed.is_empty() {
|
||||
return None;
|
||||
}
|
||||
// Accept either `Z`, `+HH:MM`, or no suffix. Reduce to a flat
|
||||
// `YYYY-MM-DDTHH:MM:SS` core for parsing — server-side timestamps
|
||||
// are already in UTC for Navidrome, and we don't track timezone
|
||||
// in the schema column.
|
||||
let core = trimmed
|
||||
.find(|c: char| c == '.' || c == 'Z' || c == '+' || (c == '-' && trimmed.find('T').is_some_and(|t| trimmed[t..].contains(c))))
|
||||
.map(|i| &trimmed[..i])
|
||||
.unwrap_or(trimmed);
|
||||
let mut parts = core.split(['T', '-', ':']);
|
||||
let year: i64 = parts.next()?.parse().ok()?;
|
||||
let month: i64 = parts.next()?.parse().ok()?;
|
||||
let day: i64 = parts.next()?.parse().ok()?;
|
||||
let hour: i64 = parts.next().unwrap_or("0").parse().ok()?;
|
||||
let minute: i64 = parts.next().unwrap_or("0").parse().ok()?;
|
||||
let second: i64 = parts.next().unwrap_or("0").parse().ok()?;
|
||||
if !(1970..=2100).contains(&year)
|
||||
|| !(1..=12).contains(&month)
|
||||
|| !(1..=31).contains(&day)
|
||||
|| !(0..=23).contains(&hour)
|
||||
|| !(0..=59).contains(&minute)
|
||||
|| !(0..=60).contains(&second)
|
||||
{
|
||||
return None;
|
||||
}
|
||||
// Days since 1970-01-01 — Howard Hinnant's civil_from_days inverse.
|
||||
let y = if month <= 2 { year - 1 } else { year };
|
||||
let era = y.div_euclid(400);
|
||||
let yoe = y - era * 400; // [0, 399]
|
||||
let m = if month > 2 { month - 3 } else { month + 9 };
|
||||
let doy = (153 * m + 2) / 5 + day - 1;
|
||||
let doe = yoe * 365 + yoe / 4 - yoe / 100 + doy;
|
||||
let days = era * 146_097 + doe - 719_468;
|
||||
let seconds = days * 86_400 + hour * 3600 + minute * 60 + second;
|
||||
Some(seconds.saturating_mul(1000))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn parse_iso_handles_zulu_suffix() {
|
||||
// 2024-01-01T00:00:00Z = 1704067200000 ms.
|
||||
let ms = parse_iso_ms_str("2024-01-01T00:00:00Z").unwrap();
|
||||
assert_eq!(ms, 1_704_067_200_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_iso_handles_fractional_and_offset() {
|
||||
let ms = parse_iso_ms_str("2024-01-01T00:00:00.123+02:00").unwrap();
|
||||
// Truncated before offset → same epoch-second as Zulu of the
|
||||
// wall-clock value. Good enough for the schema's integer-ms
|
||||
// column.
|
||||
assert_eq!(ms, 1_704_067_200_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_iso_rejects_garbage() {
|
||||
assert!(parse_iso_ms_str("").is_none());
|
||||
assert!(parse_iso_ms_str("not-a-date").is_none());
|
||||
assert!(parse_iso_ms_str("9999-99-99").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn subsonic_song_maps_hot_columns_and_keeps_raw_json() {
|
||||
let raw = json!({
|
||||
"id": "tr_1",
|
||||
"title": "Hello",
|
||||
"artist": "World",
|
||||
"albumId": "al_1",
|
||||
"duration": 240,
|
||||
"track": 3,
|
||||
"year": 2024,
|
||||
"musicBrainzId": "mb-1",
|
||||
"replayGain": { "trackGain": -1.2, "albumGain": -0.8 }
|
||||
});
|
||||
let song: Song = serde_json::from_value(raw.clone()).unwrap();
|
||||
let row = subsonic_song_to_track_row("s1", &song, &raw, 1_000, Some("lib-fb"));
|
||||
assert_eq!(row.id, "tr_1");
|
||||
assert_eq!(row.album_id.as_deref(), Some("al_1"));
|
||||
assert_eq!(row.duration_sec, 240);
|
||||
assert_eq!(row.mbid_recording.as_deref(), Some("mb-1"));
|
||||
assert_eq!(row.replay_gain_track_db, Some(-1.2));
|
||||
assert_eq!(row.replay_gain_album_db, Some(-0.8));
|
||||
// Fallback library_id kicks in when the song didn't ship one.
|
||||
assert_eq!(row.library_id.as_deref(), Some("lib-fb"));
|
||||
assert!(row.raw_json.contains("replayGain"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn navidrome_song_maps_native_field_shape() {
|
||||
let raw = json!({
|
||||
"id": "tr_1",
|
||||
"title": "Hello",
|
||||
"artist": "World",
|
||||
"artistId": "ar_1",
|
||||
"album": "An Album",
|
||||
"albumId": "al_1",
|
||||
"albumArtist": "World",
|
||||
"duration": 240,
|
||||
"trackNumber": 3,
|
||||
"discNumber": 1,
|
||||
"year": 2024,
|
||||
"genre": "Ambient",
|
||||
"suffix": "flac",
|
||||
"bitRate": 1000,
|
||||
"size": 32_000_000_i64,
|
||||
"path": "World/An Album/03.flac",
|
||||
"libraryId": "1",
|
||||
"isrc": "USRC17607839",
|
||||
"mbzTrackId": "mb-1",
|
||||
"bpm": 128,
|
||||
"rgTrackGain": -1.2,
|
||||
"rgAlbumGain": -0.8,
|
||||
"createdAt": "2024-01-01T00:00:00Z",
|
||||
"updatedAt": "2024-06-01T00:00:00Z"
|
||||
});
|
||||
let row = navidrome_song_to_track_row("s1", &raw, 9_999, None).unwrap();
|
||||
assert_eq!(row.id, "tr_1");
|
||||
assert_eq!(row.track_number, Some(3));
|
||||
assert_eq!(row.isrc.as_deref(), Some("USRC17607839"));
|
||||
assert_eq!(row.mbid_recording.as_deref(), Some("mb-1"));
|
||||
assert_eq!(row.replay_gain_track_db, Some(-1.2));
|
||||
assert_eq!(row.library_id.as_deref(), Some("1"));
|
||||
assert!(row.server_updated_at.unwrap_or(0) > 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn navidrome_song_maps_numeric_library_id() {
|
||||
let raw = json!({
|
||||
"id": "tr_1",
|
||||
"title": "Hello",
|
||||
"libraryId": 3
|
||||
});
|
||||
let row = navidrome_song_to_track_row("s1", &raw, 1, None).unwrap();
|
||||
assert_eq!(row.library_id.as_deref(), Some("3"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn navidrome_song_skips_rows_without_id() {
|
||||
let row = navidrome_song_to_track_row("s1", &json!({"title": "no id"}), 1, None);
|
||||
assert!(row.is_none());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
//! Sync orchestrator.
|
||||
//!
|
||||
//! PR-3a landed the foundation (`CapabilityProbe` C1 +
|
||||
//! `SyncStateRepository` C7); PR-3b adds the initial-sync runner,
|
||||
//! ingest-strategy selection, backoff, and the §6.9 id remap path.
|
||||
//! `DeltaSyncRunner` / background scheduler / Tauri surface follow in
|
||||
//! PR-3c / PR-3d / PR-5.
|
||||
|
||||
pub mod backoff;
|
||||
pub mod bandwidth;
|
||||
pub mod budget;
|
||||
pub mod capability;
|
||||
pub mod cursor;
|
||||
pub mod delta;
|
||||
pub mod error;
|
||||
pub mod ingest_parallel;
|
||||
pub mod initial;
|
||||
pub mod mapping;
|
||||
pub mod poll_stats;
|
||||
pub mod progress;
|
||||
pub mod scheduler;
|
||||
pub mod strategy;
|
||||
pub mod supervisor;
|
||||
pub mod tombstone;
|
||||
|
||||
pub use backoff::{with_jitter, Backoff};
|
||||
pub use bandwidth::{ParallelismBudget, PlaybackHint};
|
||||
pub use budget::{PassKind, RequestBudget};
|
||||
pub use capability::{CapabilityFlags, CapabilityProbe, NavidromeProbeCredentials};
|
||||
pub use cursor::{CursorPhase, InitialSyncCursor, StrategyState};
|
||||
pub use delta::{DeltaSyncReport, DeltaSyncRunner};
|
||||
pub use error::SyncError;
|
||||
pub use initial::{InitialSyncReport, InitialSyncRunner};
|
||||
pub use mapping::{navidrome_song_to_track_row, subsonic_song_to_track_row};
|
||||
pub use poll_stats::{classify_tier, next_interval_ms, LibraryTier, PollStats};
|
||||
pub use progress::{ChannelProgress, NoopProgress, Progress, ProgressEvent};
|
||||
pub use scheduler::{BackgroundScheduler, SchedulerTickReport, DEFAULT_TOMBSTONE_THRESHOLD_PCT};
|
||||
pub use strategy::IngestStrategy;
|
||||
pub use supervisor::SyncSupervisor;
|
||||
pub use tombstone::{should_auto_reconcile, TombstoneReconciler, TombstoneReport};
|
||||
@@ -0,0 +1,250 @@
|
||||
//! C10 — adaptive poll interval (spec §6.2.2).
|
||||
//!
|
||||
//! Rolling EWMA over the last delta pass's HTTP response stats plus
|
||||
//! the artist-count signal classifies the server into a `LibraryTier`,
|
||||
//! which then drives the next poll interval. Persisted in
|
||||
//! `sync_state.poll_stats_json` so the scheduler picks up where it
|
||||
//! left off across restarts.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// EWMA smoothing factor — higher values weight recent samples more.
|
||||
/// 0.3 matches the §6.2.2 "rolling EWMA" target without over-reacting
|
||||
/// to a single slow response.
|
||||
pub const EWMA_ALPHA: f64 = 0.3;
|
||||
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum LibraryTier {
|
||||
/// `<2k` artists.
|
||||
Small,
|
||||
/// `2k-15k` artists.
|
||||
Medium,
|
||||
/// `>15k` artists OR `ewma_bytes > 2 MB`.
|
||||
Huge,
|
||||
#[default]
|
||||
Unknown,
|
||||
}
|
||||
|
||||
impl LibraryTier {
|
||||
pub fn as_tag(self) -> &'static str {
|
||||
match self {
|
||||
Self::Small => "small",
|
||||
Self::Medium => "medium",
|
||||
Self::Huge => "huge",
|
||||
Self::Unknown => "unknown",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Serialize, Deserialize)]
|
||||
pub struct PollStats {
|
||||
/// Last successful `getArtists.index` cardinality, or
|
||||
/// `getScanStatus.count` estimate. Used as the primary tier
|
||||
/// signal.
|
||||
#[serde(default)]
|
||||
pub artist_count: u64,
|
||||
/// EWMA of the most recent poll response sizes, in bytes.
|
||||
/// Decompressed (post-gzip) per §2.2.1.
|
||||
#[serde(default)]
|
||||
pub ewma_bytes: f64,
|
||||
/// EWMA of the most recent poll wall-clock durations, in ms.
|
||||
#[serde(default)]
|
||||
pub ewma_duration_ms: f64,
|
||||
/// Resolved tier from the inputs above.
|
||||
#[serde(default)]
|
||||
pub library_tier: LibraryTier,
|
||||
}
|
||||
|
||||
impl PollStats {
|
||||
/// Fold a new sample into the EWMAs. First sample seeds the
|
||||
/// averages directly so a single fresh poll yields a meaningful
|
||||
/// tier classification.
|
||||
pub fn observe(&mut self, bytes: u64, duration_ms: u64) {
|
||||
let b = bytes as f64;
|
||||
let d = duration_ms as f64;
|
||||
self.ewma_bytes = if self.ewma_bytes == 0.0 {
|
||||
b
|
||||
} else {
|
||||
EWMA_ALPHA * b + (1.0 - EWMA_ALPHA) * self.ewma_bytes
|
||||
};
|
||||
self.ewma_duration_ms = if self.ewma_duration_ms == 0.0 {
|
||||
d
|
||||
} else {
|
||||
EWMA_ALPHA * d + (1.0 - EWMA_ALPHA) * self.ewma_duration_ms
|
||||
};
|
||||
}
|
||||
|
||||
/// Update `artist_count` and recompute the resolved tier.
|
||||
pub fn set_artist_count(&mut self, count: u64) {
|
||||
self.artist_count = count;
|
||||
self.library_tier = classify_tier(self.artist_count, self.ewma_bytes);
|
||||
}
|
||||
|
||||
/// Force a tier re-classification — call after a fresh
|
||||
/// `observe()` pair if a borderline `ewma_bytes` may have tipped
|
||||
/// the threshold.
|
||||
pub fn reclassify(&mut self) {
|
||||
self.library_tier = classify_tier(self.artist_count, self.ewma_bytes);
|
||||
}
|
||||
}
|
||||
|
||||
/// §6.2.2 classification table.
|
||||
pub fn classify_tier(artist_count: u64, ewma_bytes: f64) -> LibraryTier {
|
||||
if artist_count == 0 && ewma_bytes == 0.0 {
|
||||
return LibraryTier::Unknown;
|
||||
}
|
||||
if artist_count > 15_000 || ewma_bytes > 2_000_000.0 {
|
||||
return LibraryTier::Huge;
|
||||
}
|
||||
if artist_count >= 2_000 {
|
||||
return LibraryTier::Medium;
|
||||
}
|
||||
LibraryTier::Small
|
||||
}
|
||||
|
||||
/// Spec §6.2.2 formula. Returns the delta in milliseconds — caller
|
||||
/// stamps `next_poll_at = now + this`. All arithmetic in `f64` so the
|
||||
/// `load_factor` clamp produces a smooth curve.
|
||||
pub fn next_interval_ms(stats: &PollStats) -> u64 {
|
||||
let base_ms: u64 = match stats.library_tier {
|
||||
LibraryTier::Huge => 15 * 60 * 1000,
|
||||
LibraryTier::Medium => 10 * 60 * 1000,
|
||||
LibraryTier::Small => 5 * 60 * 1000,
|
||||
// No data yet — start short so the first real tick lands
|
||||
// quickly and re-classifies.
|
||||
LibraryTier::Unknown => 60 * 1000,
|
||||
};
|
||||
let load_factor = if stats.ewma_duration_ms <= 0.0 {
|
||||
1.0
|
||||
} else {
|
||||
(stats.ewma_duration_ms / 3000.0).clamp(1.0, 10.0)
|
||||
};
|
||||
let artist_factor: f64 = if matches!(stats.library_tier, LibraryTier::Huge) {
|
||||
3.0
|
||||
} else {
|
||||
1.0
|
||||
};
|
||||
let scaled = (base_ms as f64) * load_factor * artist_factor;
|
||||
scaled.min(u64::MAX as f64) as u64
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn classify_returns_unknown_with_zero_signals() {
|
||||
assert_eq!(classify_tier(0, 0.0), LibraryTier::Unknown);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn classify_small_under_2k_artists() {
|
||||
assert_eq!(classify_tier(500, 1000.0), LibraryTier::Small);
|
||||
assert_eq!(classify_tier(1_999, 1000.0), LibraryTier::Small);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn classify_medium_in_range() {
|
||||
assert_eq!(classify_tier(2_000, 1000.0), LibraryTier::Medium);
|
||||
assert_eq!(classify_tier(15_000, 1000.0), LibraryTier::Medium);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn classify_huge_above_15k_artists() {
|
||||
assert_eq!(classify_tier(15_001, 1000.0), LibraryTier::Huge);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn classify_huge_when_ewma_bytes_exceed_2mb_even_with_few_artists() {
|
||||
// Slow-network signal — the spec's `ewma_bytes > 2MB` override.
|
||||
assert_eq!(classify_tier(500, 2_500_000.0), LibraryTier::Huge);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn observe_first_sample_seeds_ewmas_directly() {
|
||||
let mut s = PollStats::default();
|
||||
s.observe(100_000, 1500);
|
||||
assert_eq!(s.ewma_bytes, 100_000.0);
|
||||
assert_eq!(s.ewma_duration_ms, 1500.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn observe_subsequent_samples_apply_alpha_smoothing() {
|
||||
let mut s = PollStats::default();
|
||||
s.observe(100_000, 1000);
|
||||
s.observe(200_000, 2000);
|
||||
// 0.3 * 200_000 + 0.7 * 100_000 = 60_000 + 70_000 = 130_000
|
||||
assert!((s.ewma_bytes - 130_000.0).abs() < 0.001);
|
||||
assert!((s.ewma_duration_ms - 1300.0).abs() < 0.001);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn set_artist_count_triggers_reclassification() {
|
||||
let mut s = PollStats::default();
|
||||
s.observe(50_000, 1500);
|
||||
s.set_artist_count(5_000);
|
||||
assert_eq!(s.library_tier, LibraryTier::Medium);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn next_interval_unknown_tier_starts_short() {
|
||||
let s = PollStats::default();
|
||||
assert_eq!(next_interval_ms(&s), 60_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn next_interval_small_base_5min_at_idle_load() {
|
||||
let mut s = PollStats::default();
|
||||
s.set_artist_count(1000);
|
||||
// load_factor clamps at 1.0 when ewma_duration_ms = 0.
|
||||
assert_eq!(next_interval_ms(&s), 5 * 60_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn next_interval_huge_uses_artist_factor_3x() {
|
||||
let mut s = PollStats::default();
|
||||
s.set_artist_count(20_000);
|
||||
// 15 min * 3 = 45 min on idle load.
|
||||
assert_eq!(next_interval_ms(&s), 45 * 60_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn next_interval_load_factor_stretches_with_slow_network() {
|
||||
let mut s = PollStats::default();
|
||||
s.set_artist_count(1000);
|
||||
s.observe(50_000, 9_000); // 3× the 3000ms target
|
||||
// 5 min * 3 (load_factor) = 15 min.
|
||||
let ms = next_interval_ms(&s);
|
||||
assert!(
|
||||
(14 * 60_000..=16 * 60_000).contains(&ms),
|
||||
"expected ~15min, got {ms}ms"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn next_interval_load_factor_clamped_at_10x() {
|
||||
let mut s = PollStats::default();
|
||||
s.set_artist_count(1000);
|
||||
s.observe(50_000, 60_000); // 20× target → clamps at 10
|
||||
assert_eq!(next_interval_ms(&s), 10 * 5 * 60_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn poll_stats_round_trips_through_json() {
|
||||
let mut s = PollStats::default();
|
||||
s.observe(123_456, 789);
|
||||
s.set_artist_count(3_500);
|
||||
let json = serde_json::to_value(s).unwrap();
|
||||
let back: PollStats = serde_json::from_value(json).unwrap();
|
||||
assert_eq!(back, s);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn poll_stats_deserialize_tolerates_missing_fields() {
|
||||
// Stored default is `'{}'` per spec §5.1; runner must accept it
|
||||
// as a fresh stats object.
|
||||
let s: PollStats = serde_json::from_str("{}").unwrap();
|
||||
assert_eq!(s, PollStats::default());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,329 @@
|
||||
//! C6 — progress channel for the sync runners (spec §6 emit limit
|
||||
//! `≤2 events/s`). The runners call `Progress::emit` at phase
|
||||
//! transitions and per-batch checkpoints; the supervisor wraps an
|
||||
//! `mpsc::UnboundedSender` so the top crate (PR-5) can forward events
|
||||
//! to Tauri's emit surface.
|
||||
//!
|
||||
//! Non-terminal events are rate-limited. `IngestPage` updates are
|
||||
//! **coalesced** (latest `ingested_total` wins) so fast S1/N1 batches
|
||||
//! do not leave the UI stuck on a stale count between throttled emits.
|
||||
|
||||
use std::sync::Mutex;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::repos::RemapEntry;
|
||||
|
||||
/// Per-batch ingest timings (DevTools + terminal diagnosis).
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct IngestBatchMetrics {
|
||||
pub offset: u32,
|
||||
pub strategy: String,
|
||||
pub fetch_ms: u32,
|
||||
pub write_ms: u32,
|
||||
pub lock_wait_ms: u32,
|
||||
pub sql_exec_ms: u32,
|
||||
pub persist_ms: u32,
|
||||
pub row_count: u32,
|
||||
pub bulk_ingest_active: bool,
|
||||
}
|
||||
|
||||
/// Lean event union — server_id / library_scope context lives on the
|
||||
/// channel side (one supervisor = one scope). Top-crate code wraps
|
||||
/// these into Tauri events with their own envelope.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum ProgressEvent {
|
||||
PhaseChanged { phase: String },
|
||||
IngestPage {
|
||||
ingested_total: u32,
|
||||
batch_count: u32,
|
||||
metrics: Option<IngestBatchMetrics>,
|
||||
},
|
||||
Remapped { entries: Vec<RemapEntry> },
|
||||
Tombstoned { deleted_count: u32, checked_count: u32 },
|
||||
Completed { kind: String },
|
||||
Error { message: String },
|
||||
}
|
||||
|
||||
impl ProgressEvent {
|
||||
/// Terminal events always bypass the throttle so callers never
|
||||
/// miss a "we're done" / "we crashed" signal.
|
||||
pub fn always_emit(&self) -> bool {
|
||||
matches!(self, Self::Completed { .. } | Self::Error { .. })
|
||||
}
|
||||
}
|
||||
|
||||
pub trait Progress: Send + Sync {
|
||||
fn emit(&self, event: ProgressEvent);
|
||||
}
|
||||
|
||||
/// No-op implementation. Used as the default when runners are called
|
||||
/// outside a supervisor (tests, future ad-hoc invocations).
|
||||
pub struct NoopProgress;
|
||||
|
||||
impl Progress for NoopProgress {
|
||||
fn emit(&self, _event: ProgressEvent) {}
|
||||
}
|
||||
|
||||
/// `Progress` impl that forwards through a tokio mpsc channel,
|
||||
/// throttling non-terminal events to the configured `min_interval`.
|
||||
pub struct ChannelProgress {
|
||||
sender: tokio::sync::mpsc::UnboundedSender<ProgressEvent>,
|
||||
min_interval: Duration,
|
||||
last_emit: Mutex<Option<Instant>>,
|
||||
/// Latest ingest checkpoint held back while the throttle gate is closed.
|
||||
pending_ingest: Mutex<Option<(u32, u32, Option<IngestBatchMetrics>)>>,
|
||||
}
|
||||
|
||||
impl ChannelProgress {
|
||||
/// 500 ms gate ≈ 2 events/s per spec §6.
|
||||
pub const DEFAULT_INTERVAL: Duration = Duration::from_millis(500);
|
||||
|
||||
pub fn new(sender: tokio::sync::mpsc::UnboundedSender<ProgressEvent>) -> Self {
|
||||
Self::with_interval(sender, Self::DEFAULT_INTERVAL)
|
||||
}
|
||||
|
||||
pub fn with_interval(
|
||||
sender: tokio::sync::mpsc::UnboundedSender<ProgressEvent>,
|
||||
min_interval: Duration,
|
||||
) -> Self {
|
||||
Self {
|
||||
sender,
|
||||
min_interval,
|
||||
last_emit: Mutex::new(None),
|
||||
pending_ingest: Mutex::new(None),
|
||||
}
|
||||
}
|
||||
|
||||
fn flush_pending_ingest(&self) {
|
||||
let pending = self
|
||||
.pending_ingest
|
||||
.lock()
|
||||
.expect("progress pending lock poisoned")
|
||||
.take();
|
||||
if let Some((ingested_total, batch_count, metrics)) = pending {
|
||||
let _ = self.sender.send(ProgressEvent::IngestPage {
|
||||
ingested_total,
|
||||
batch_count,
|
||||
metrics,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
fn throttle_open(&self) -> bool {
|
||||
if self.min_interval.is_zero() {
|
||||
return true;
|
||||
}
|
||||
let last = self.last_emit.lock().expect("progress lock poisoned");
|
||||
match *last {
|
||||
None => true,
|
||||
Some(prev) if prev.elapsed() >= self.min_interval => true,
|
||||
Some(_) => false,
|
||||
}
|
||||
}
|
||||
|
||||
fn mark_emitted(&self) {
|
||||
if self.min_interval.is_zero() {
|
||||
return;
|
||||
}
|
||||
*self
|
||||
.last_emit
|
||||
.lock()
|
||||
.expect("progress lock poisoned") = Some(Instant::now());
|
||||
}
|
||||
}
|
||||
|
||||
impl Progress for ChannelProgress {
|
||||
fn emit(&self, event: ProgressEvent) {
|
||||
if event.always_emit() {
|
||||
self.flush_pending_ingest();
|
||||
let _ = self.sender.send(event);
|
||||
return;
|
||||
}
|
||||
|
||||
if let ProgressEvent::IngestPage {
|
||||
ingested_total,
|
||||
batch_count,
|
||||
metrics,
|
||||
} = event
|
||||
{
|
||||
let gate_open = self.throttle_open();
|
||||
{
|
||||
let mut pending = self
|
||||
.pending_ingest
|
||||
.lock()
|
||||
.expect("progress pending lock poisoned");
|
||||
if gate_open {
|
||||
if let Some((total, batch, m)) = pending.take() {
|
||||
let _ = self.sender.send(ProgressEvent::IngestPage {
|
||||
ingested_total: total,
|
||||
batch_count: batch,
|
||||
metrics: m,
|
||||
});
|
||||
}
|
||||
}
|
||||
*pending = Some((ingested_total, batch_count, metrics));
|
||||
}
|
||||
if gate_open {
|
||||
self.mark_emitted();
|
||||
self.flush_pending_ingest();
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if !self.min_interval.is_zero() && !self.throttle_open() {
|
||||
return;
|
||||
}
|
||||
self.flush_pending_ingest();
|
||||
self.mark_emitted();
|
||||
let _ = self.sender.send(event);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::thread;
|
||||
use std::time::Duration;
|
||||
use tokio::sync::mpsc;
|
||||
|
||||
#[test]
|
||||
fn noop_progress_swallows_events_without_panicking() {
|
||||
let p = NoopProgress;
|
||||
p.emit(ProgressEvent::PhaseChanged { phase: "ingest".into() });
|
||||
p.emit(ProgressEvent::Completed { kind: "initial_sync".into() });
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn zero_interval_channel_emits_every_event() {
|
||||
let (tx, mut rx) = mpsc::unbounded_channel();
|
||||
let p = ChannelProgress::with_interval(tx, Duration::ZERO);
|
||||
for i in 0..10 {
|
||||
p.emit(ProgressEvent::IngestPage {
|
||||
ingested_total: i,
|
||||
batch_count: 1,
|
||||
metrics: None,
|
||||
});
|
||||
}
|
||||
let mut received = 0;
|
||||
while rx.try_recv().is_ok() {
|
||||
received += 1;
|
||||
}
|
||||
assert_eq!(received, 10, "ZERO interval must not drop anything");
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn terminal_events_bypass_throttle() {
|
||||
let (tx, mut rx) = mpsc::unbounded_channel();
|
||||
let p = ChannelProgress::with_interval(tx, Duration::from_secs(60));
|
||||
p.emit(ProgressEvent::IngestPage {
|
||||
ingested_total: 999,
|
||||
batch_count: 3,
|
||||
metrics: None,
|
||||
});
|
||||
p.emit(ProgressEvent::Completed { kind: "delta_sync".into() });
|
||||
p.emit(ProgressEvent::Error { message: "boom".into() });
|
||||
assert!(matches!(
|
||||
rx.try_recv(),
|
||||
Ok(ProgressEvent::IngestPage {
|
||||
ingested_total: 999,
|
||||
..
|
||||
})
|
||||
));
|
||||
assert!(matches!(
|
||||
rx.try_recv(),
|
||||
Ok(ProgressEvent::Completed { .. })
|
||||
));
|
||||
assert!(matches!(rx.try_recv(), Ok(ProgressEvent::Error { .. })));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn non_terminal_events_collapse_under_throttle() {
|
||||
let (tx, mut rx) = mpsc::unbounded_channel();
|
||||
let p = ChannelProgress::with_interval(tx, Duration::from_millis(100));
|
||||
p.emit(ProgressEvent::PhaseChanged { phase: "a".into() });
|
||||
p.emit(ProgressEvent::PhaseChanged { phase: "b".into() });
|
||||
assert!(matches!(
|
||||
rx.try_recv(),
|
||||
Ok(ProgressEvent::PhaseChanged { ref phase }) if phase == "a"
|
||||
));
|
||||
assert!(rx.try_recv().is_err(), "second emit must have been dropped");
|
||||
|
||||
thread::sleep(Duration::from_millis(120));
|
||||
p.emit(ProgressEvent::PhaseChanged { phase: "c".into() });
|
||||
assert!(matches!(
|
||||
rx.try_recv(),
|
||||
Ok(ProgressEvent::PhaseChanged { ref phase }) if phase == "c"
|
||||
));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn ingest_pages_coalesce_to_latest_within_throttle_window() {
|
||||
let (tx, mut rx) = mpsc::unbounded_channel();
|
||||
let p = ChannelProgress::with_interval(tx, Duration::from_millis(100));
|
||||
p.emit(ProgressEvent::IngestPage {
|
||||
ingested_total: 500,
|
||||
batch_count: 1,
|
||||
metrics: None,
|
||||
});
|
||||
p.emit(ProgressEvent::IngestPage {
|
||||
ingested_total: 2500,
|
||||
batch_count: 5,
|
||||
metrics: None,
|
||||
});
|
||||
p.emit(ProgressEvent::IngestPage {
|
||||
ingested_total: 5000,
|
||||
batch_count: 10,
|
||||
metrics: None,
|
||||
});
|
||||
assert!(matches!(
|
||||
rx.try_recv(),
|
||||
Ok(ProgressEvent::IngestPage {
|
||||
ingested_total: 500,
|
||||
..
|
||||
})
|
||||
));
|
||||
assert!(rx.try_recv().is_err(), "bursts must coalesce, not stack");
|
||||
|
||||
thread::sleep(Duration::from_millis(120));
|
||||
p.emit(ProgressEvent::IngestPage {
|
||||
ingested_total: 5500,
|
||||
batch_count: 11,
|
||||
metrics: None,
|
||||
});
|
||||
assert!(
|
||||
matches!(
|
||||
rx.try_recv(),
|
||||
Ok(ProgressEvent::IngestPage {
|
||||
ingested_total: 5000,
|
||||
..
|
||||
})
|
||||
),
|
||||
"latest pending count must flush when the gate opens"
|
||||
);
|
||||
assert!(matches!(
|
||||
rx.try_recv(),
|
||||
Ok(ProgressEvent::IngestPage {
|
||||
ingested_total: 5500,
|
||||
..
|
||||
})
|
||||
));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn closed_receiver_does_not_panic_the_sender() {
|
||||
let (tx, rx) = mpsc::unbounded_channel();
|
||||
let p = ChannelProgress::with_interval(tx, Duration::ZERO);
|
||||
drop(rx);
|
||||
p.emit(ProgressEvent::PhaseChanged { phase: "x".into() });
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn always_emit_true_for_terminal_events() {
|
||||
assert!(ProgressEvent::Completed { kind: "k".into() }.always_emit());
|
||||
assert!(ProgressEvent::Error { message: "m".into() }.always_emit());
|
||||
assert!(!ProgressEvent::PhaseChanged { phase: "p".into() }.always_emit());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,619 @@
|
||||
//! C8 — background scheduler (spec §6.2).
|
||||
//!
|
||||
//! Tick-based: the top crate (PR-5) drives the actual timer; PR-3d2
|
||||
//! ships the logic that decides "is it time?", picks the budget +
|
||||
//! tombstone trigger, runs the DeltaSyncRunner, and writes back the
|
||||
//! adaptive interval.
|
||||
//!
|
||||
//! Owns no tokio task itself — keeps testability high and lets the
|
||||
//! caller decide spawn behaviour (Supervisor or inline).
|
||||
|
||||
use std::sync::atomic::AtomicBool;
|
||||
use std::sync::Arc;
|
||||
|
||||
use psysonic_integration::subsonic::SubsonicClient;
|
||||
|
||||
use super::bandwidth::{ParallelismBudget, PlaybackHint};
|
||||
use super::budget::{PassKind, RequestBudget};
|
||||
use super::capability::{CapabilityFlags, NavidromeProbeCredentials};
|
||||
use super::delta::{DeltaSyncReport, DeltaSyncRunner};
|
||||
use super::error::SyncError;
|
||||
use super::poll_stats::{next_interval_ms, PollStats};
|
||||
use super::progress::{NoopProgress, Progress};
|
||||
use super::tombstone::should_auto_reconcile;
|
||||
use crate::repos::SyncStateRepository;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
/// Default Mode B threshold per §6.7 (5 % gap before auto reconcile).
|
||||
pub const DEFAULT_TOMBSTONE_THRESHOLD_PCT: u32 = 5;
|
||||
|
||||
/// Outcome of one scheduler tick — what happened plus the resolved
|
||||
/// `next_poll_at` so the caller can re-schedule its timer.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct SchedulerTickReport {
|
||||
pub skipped_not_due: bool,
|
||||
pub skipped_bulk_paused: bool,
|
||||
/// Delta/tombstone pass deferred while initial sync or capability probe
|
||||
/// holds `sync_phase`, IS-3 bulk ingest is active, or a foreground sync
|
||||
/// job (`LibraryRuntime::current_job`) is running for this server.
|
||||
pub skipped_sync_pass_active: bool,
|
||||
pub delta: Option<DeltaSyncReport>,
|
||||
pub next_poll_at_ms: i64,
|
||||
}
|
||||
|
||||
pub struct BackgroundScheduler<'a> {
|
||||
store: &'a LibraryStore,
|
||||
subsonic: &'a SubsonicClient,
|
||||
navidrome: Option<NavidromeProbeCredentials>,
|
||||
server_id: String,
|
||||
library_scope: String,
|
||||
capability_flags: CapabilityFlags,
|
||||
playback_hint: PlaybackHint,
|
||||
cancel: Option<Arc<AtomicBool>>,
|
||||
progress: Arc<dyn Progress + Send + Sync>,
|
||||
tombstone_threshold_pct: u32,
|
||||
sleep_enabled: bool,
|
||||
/// When true, a user-triggered sync job (delta / verify / full resync)
|
||||
/// already owns this server — skip the background delta pass.
|
||||
foreground_sync_job_active: bool,
|
||||
}
|
||||
|
||||
impl<'a> BackgroundScheduler<'a> {
|
||||
pub fn new(
|
||||
store: &'a LibraryStore,
|
||||
subsonic: &'a SubsonicClient,
|
||||
server_id: impl Into<String>,
|
||||
library_scope: impl Into<String>,
|
||||
capability_flags: CapabilityFlags,
|
||||
) -> Self {
|
||||
Self {
|
||||
store,
|
||||
subsonic,
|
||||
navidrome: None,
|
||||
server_id: server_id.into(),
|
||||
library_scope: library_scope.into(),
|
||||
capability_flags,
|
||||
playback_hint: PlaybackHint::Idle,
|
||||
cancel: None,
|
||||
progress: Arc::new(NoopProgress),
|
||||
tombstone_threshold_pct: DEFAULT_TOMBSTONE_THRESHOLD_PCT,
|
||||
sleep_enabled: true,
|
||||
foreground_sync_job_active: false,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn with_navidrome_credentials(mut self, creds: NavidromeProbeCredentials) -> Self {
|
||||
self.navidrome = Some(creds);
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_playback_hint(mut self, hint: PlaybackHint) -> Self {
|
||||
self.playback_hint = hint;
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_cancellation(mut self, flag: Arc<AtomicBool>) -> Self {
|
||||
self.cancel = Some(flag);
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_progress(mut self, progress: Arc<dyn Progress + Send + Sync>) -> Self {
|
||||
self.progress = progress;
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_tombstone_threshold_pct(mut self, pct: u32) -> Self {
|
||||
self.tombstone_threshold_pct = pct;
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_sleep_disabled(mut self) -> Self {
|
||||
self.sleep_enabled = false;
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_foreground_sync_job_active(mut self, active: bool) -> Self {
|
||||
self.foreground_sync_job_active = active;
|
||||
self
|
||||
}
|
||||
|
||||
/// `true` when `next_poll_at` has passed (or no value yet). Caller
|
||||
/// short-circuits its timer when this returns `false`.
|
||||
pub fn is_due(&self, now_ms: i64) -> Result<bool, SyncError> {
|
||||
let sync_state = SyncStateRepository::new(self.store);
|
||||
let next = sync_state
|
||||
.get_next_poll_at(&self.server_id, &self.library_scope)
|
||||
.map_err(SyncError::Storage)?;
|
||||
Ok(next.map(|n| now_ms >= n).unwrap_or(true))
|
||||
}
|
||||
|
||||
/// Resolve the parallelism budget for the current playback state.
|
||||
/// Bulk-paused state means the scheduler skips the tick entirely
|
||||
/// and just re-schedules.
|
||||
pub fn parallelism_budget(&self) -> ParallelismBudget {
|
||||
ParallelismBudget::resolve(self.playback_hint)
|
||||
}
|
||||
|
||||
/// Run one tick — runs a delta sync if due and bulk isn't paused
|
||||
/// by the playback signal, then writes the new `next_poll_at`.
|
||||
pub async fn tick(&self, now_ms: i64) -> Result<SchedulerTickReport, SyncError> {
|
||||
let sync_state = SyncStateRepository::new(self.store);
|
||||
sync_state
|
||||
.ensure(&self.server_id, &self.library_scope)
|
||||
.map_err(SyncError::Storage)?;
|
||||
|
||||
let mut report = SchedulerTickReport {
|
||||
skipped_not_due: false,
|
||||
skipped_bulk_paused: false,
|
||||
skipped_sync_pass_active: false,
|
||||
delta: None,
|
||||
next_poll_at_ms: now_ms,
|
||||
};
|
||||
|
||||
if self.sync_pass_active(&sync_state)? {
|
||||
report.skipped_sync_pass_active = true;
|
||||
report.next_poll_at_ms = now_ms + 30_000;
|
||||
sync_state
|
||||
.set_next_poll_at(&self.server_id, &self.library_scope, report.next_poll_at_ms)
|
||||
.map_err(SyncError::Storage)?;
|
||||
crate::app_eprintln!(
|
||||
"[library-sync] scheduler tick skipped: sync pass active (phase={:?}, bulk={})",
|
||||
sync_state
|
||||
.get_sync_phase(&self.server_id, &self.library_scope)
|
||||
.ok()
|
||||
.flatten(),
|
||||
self.store.bulk_ingest_active()
|
||||
);
|
||||
return Ok(report);
|
||||
}
|
||||
|
||||
if !self.is_due(now_ms)? {
|
||||
report.skipped_not_due = true;
|
||||
let stats = self.load_poll_stats(&sync_state)?;
|
||||
report.next_poll_at_ms = now_ms + next_interval_ms(&stats) as i64;
|
||||
return Ok(report);
|
||||
}
|
||||
|
||||
let parallelism = self.parallelism_budget();
|
||||
if parallelism.bulk_paused() {
|
||||
// §6.2.4 PrefetchActive — skip this tick entirely, re-poll
|
||||
// soon so we can catch the prefetch finishing.
|
||||
report.skipped_bulk_paused = true;
|
||||
report.next_poll_at_ms = now_ms + 30_000; // ~30s short retry
|
||||
sync_state
|
||||
.set_next_poll_at(&self.server_id, &self.library_scope, report.next_poll_at_ms)
|
||||
.map_err(SyncError::Storage)?;
|
||||
return Ok(report);
|
||||
}
|
||||
|
||||
// Decide budget + tombstone trigger.
|
||||
let mut tombstone_budget: u32 = 0;
|
||||
if let (Some(local), Some(server)) = (
|
||||
sync_state
|
||||
.get_local_track_count(&self.server_id, &self.library_scope)
|
||||
.map_err(SyncError::Storage)?,
|
||||
sync_state
|
||||
.get_server_track_count(&self.server_id, &self.library_scope)
|
||||
.map_err(SyncError::Storage)?,
|
||||
) {
|
||||
let (local_u, server_u) = (local.max(0) as u32, server.max(0) as u32);
|
||||
if should_auto_reconcile(local_u, server_u, self.tombstone_threshold_pct) {
|
||||
tombstone_budget = RequestBudget::DELTA_MISMATCH_CAP;
|
||||
}
|
||||
}
|
||||
let _pass_budget = if tombstone_budget > 0 {
|
||||
RequestBudget::for_pass(PassKind::DeltaMismatch)
|
||||
} else {
|
||||
RequestBudget::for_pass(PassKind::DeltaLight)
|
||||
};
|
||||
// PR-3d2 doesn't enforce pass_budget against the runner yet —
|
||||
// delta runner is already small (1 probe + ≤8 album-list
|
||||
// pages); the budget value is recorded so PR-5 can surface it
|
||||
// in Settings. Wire actual cap in the runner when DS-7
|
||||
// starred delta or other request-heavy paths land.
|
||||
|
||||
// Run the delta pass.
|
||||
let mut runner = DeltaSyncRunner::new(
|
||||
self.store,
|
||||
self.subsonic,
|
||||
&self.server_id,
|
||||
&self.library_scope,
|
||||
self.capability_flags,
|
||||
)
|
||||
.with_progress(Arc::clone(&self.progress));
|
||||
if let Some(creds) = &self.navidrome {
|
||||
runner = runner.with_navidrome_credentials(creds.clone());
|
||||
}
|
||||
if let Some(flag) = &self.cancel {
|
||||
runner = runner.with_cancellation(Arc::clone(flag));
|
||||
}
|
||||
if !self.sleep_enabled {
|
||||
runner = runner.with_sleep_disabled();
|
||||
}
|
||||
if tombstone_budget > 0 {
|
||||
runner = runner.with_tombstone_budget(tombstone_budget);
|
||||
}
|
||||
let delta_report = runner.run().await?;
|
||||
|
||||
// Update poll_stats: nothing measured per-request yet in
|
||||
// PR-3d2 (PR-5 will plumb byte/duration via a custom HTTP
|
||||
// wrapper). For now the tier signal updates from artist_count
|
||||
// when the next probe lands; we just persist the artist_count
|
||||
// we know from the local DB so the tier classifier has data.
|
||||
let mut stats = self.load_poll_stats(&sync_state)?;
|
||||
if delta_report.changed_count > 0 {
|
||||
// Re-stamp the local count snapshot so the next tick's
|
||||
// threshold check has fresh data.
|
||||
if let Ok(local) = self.count_local_tracks() {
|
||||
sync_state
|
||||
.set_local_track_count(&self.server_id, &self.library_scope, local)
|
||||
.map_err(SyncError::Storage)?;
|
||||
}
|
||||
}
|
||||
stats.reclassify();
|
||||
sync_state
|
||||
.set_library_tier(
|
||||
&self.server_id,
|
||||
&self.library_scope,
|
||||
stats.library_tier.as_tag(),
|
||||
)
|
||||
.map_err(SyncError::Storage)?;
|
||||
sync_state
|
||||
.set_poll_stats_json(
|
||||
&self.server_id,
|
||||
&self.library_scope,
|
||||
&serde_json::to_value(stats).unwrap_or_default(),
|
||||
)
|
||||
.map_err(SyncError::Storage)?;
|
||||
|
||||
report.next_poll_at_ms = now_ms + next_interval_ms(&stats) as i64;
|
||||
sync_state
|
||||
.set_next_poll_at(&self.server_id, &self.library_scope, report.next_poll_at_ms)
|
||||
.map_err(SyncError::Storage)?;
|
||||
|
||||
report.delta = Some(delta_report);
|
||||
Ok(report)
|
||||
}
|
||||
|
||||
fn load_poll_stats(
|
||||
&self,
|
||||
sync_state: &SyncStateRepository<'_>,
|
||||
) -> Result<PollStats, SyncError> {
|
||||
let raw = sync_state
|
||||
.get_poll_stats_json(&self.server_id, &self.library_scope)
|
||||
.map_err(SyncError::Storage)?;
|
||||
match raw {
|
||||
None => Ok(PollStats::default()),
|
||||
Some(v) => serde_json::from_value(v).map_err(|e| SyncError::Storage(e.to_string())),
|
||||
}
|
||||
}
|
||||
|
||||
/// True while initial sync, capability probe, IS-3 bulk ingest, or a
|
||||
/// foreground sync job for this server is in flight — background delta
|
||||
/// must not compete for HTTP budget or tombstone probes.
|
||||
fn sync_pass_active(&self, sync_state: &SyncStateRepository<'_>) -> Result<bool, SyncError> {
|
||||
if self.foreground_sync_job_active {
|
||||
return Ok(true);
|
||||
}
|
||||
if self.store.bulk_ingest_active() {
|
||||
return Ok(true);
|
||||
}
|
||||
let phase = sync_state
|
||||
.get_sync_phase(&self.server_id, &self.library_scope)
|
||||
.map_err(SyncError::Storage)?;
|
||||
Ok(matches!(
|
||||
phase.as_deref(),
|
||||
Some("initial_sync") | Some("probing")
|
||||
))
|
||||
}
|
||||
|
||||
fn count_local_tracks(&self) -> Result<i64, SyncError> {
|
||||
self.store
|
||||
.with_conn("scheduler.count_local_tracks", |c| {
|
||||
c.query_row(
|
||||
"SELECT COUNT(*) FROM track WHERE server_id = ?1 AND deleted = 0",
|
||||
rusqlite::params![self.server_id],
|
||||
|row| row.get(0),
|
||||
)
|
||||
})
|
||||
.map_err(SyncError::Storage)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use psysonic_integration::subsonic::{SubsonicClient, SubsonicCredentials};
|
||||
use serde_json::json;
|
||||
use wiremock::matchers::{method as wm_method, path as wm_path};
|
||||
use wiremock::{Mock, MockServer, ResponseTemplate};
|
||||
|
||||
fn test_subsonic(uri: &str) -> SubsonicClient {
|
||||
SubsonicClient::with_static_credentials(
|
||||
uri,
|
||||
SubsonicCredentials::with_static("user", "tok", "salt"),
|
||||
reqwest::Client::new(),
|
||||
)
|
||||
}
|
||||
|
||||
fn flags(bits: u32) -> CapabilityFlags {
|
||||
CapabilityFlags::new(bits)
|
||||
}
|
||||
|
||||
async fn empty_probe_and_albumlist(server: &MockServer, last_modified: i64) {
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getArtists.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": {
|
||||
"status": "ok",
|
||||
"artists": {
|
||||
"lastModified": last_modified,
|
||||
"ignoredArticles": "",
|
||||
"index": []
|
||||
}
|
||||
}
|
||||
})))
|
||||
.mount(server)
|
||||
.await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getAlbumList2.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": {
|
||||
"status": "ok",
|
||||
"albumList2": { "album": [] }
|
||||
}
|
||||
})))
|
||||
.mount(server)
|
||||
.await;
|
||||
}
|
||||
|
||||
// ── is_due ────────────────────────────────────────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn is_due_returns_true_when_no_schedule_yet() {
|
||||
let server = MockServer::start().await;
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let sched = BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
);
|
||||
assert!(sched.is_due(0).unwrap());
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn is_due_false_when_next_poll_in_future() {
|
||||
let server = MockServer::start().await;
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
sync_state.ensure("s1", "").unwrap();
|
||||
sync_state.set_next_poll_at("s1", "", 5_000_000).unwrap();
|
||||
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let sched = BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
);
|
||||
assert!(!sched.is_due(1_000_000).unwrap());
|
||||
assert!(sched.is_due(5_000_001).unwrap());
|
||||
}
|
||||
|
||||
// ── tick skips when not due ──────────────────────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn tick_skips_while_initial_sync_phase_active() {
|
||||
let server = MockServer::start().await;
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
sync_state.ensure("s1", "").unwrap();
|
||||
sync_state
|
||||
.set_sync_phase("s1", "", "initial_sync")
|
||||
.unwrap();
|
||||
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
)
|
||||
.with_sleep_disabled()
|
||||
.tick(0)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert!(report.skipped_sync_pass_active);
|
||||
assert!(report.delta.is_none());
|
||||
assert_eq!(report.next_poll_at_ms, 30_000);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn tick_skips_when_foreground_sync_job_active() {
|
||||
let server = MockServer::start().await;
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
sync_state.ensure("s1", "").unwrap();
|
||||
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
)
|
||||
.with_sleep_disabled()
|
||||
.with_foreground_sync_job_active(true)
|
||||
.tick(0)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert!(report.skipped_sync_pass_active);
|
||||
assert!(report.delta.is_none());
|
||||
assert_eq!(report.next_poll_at_ms, 30_000);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn tick_skips_when_not_due_and_reports_next_poll() {
|
||||
let server = MockServer::start().await;
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
sync_state.ensure("s1", "").unwrap();
|
||||
sync_state.set_next_poll_at("s1", "", 1_000_000_000).unwrap();
|
||||
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
)
|
||||
.with_sleep_disabled()
|
||||
.tick(500)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert!(report.skipped_not_due);
|
||||
assert!(report.delta.is_none());
|
||||
assert!(report.next_poll_at_ms > 500);
|
||||
}
|
||||
|
||||
// ── tick pauses when PrefetchActive ──────────────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn tick_pauses_when_playback_hint_is_prefetch_active() {
|
||||
let server = MockServer::start().await;
|
||||
let store = LibraryStore::open_in_memory();
|
||||
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
)
|
||||
.with_playback_hint(PlaybackHint::PrefetchActive)
|
||||
.with_sleep_disabled()
|
||||
.tick(0)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert!(report.skipped_bulk_paused);
|
||||
assert!(report.delta.is_none());
|
||||
// Re-scheduled soon (≤ 60s after now) so we catch the
|
||||
// prefetch finishing.
|
||||
assert!(report.next_poll_at_ms > 0);
|
||||
assert!(report.next_poll_at_ms <= 60_000);
|
||||
}
|
||||
|
||||
// ── tick runs delta and stamps next_poll_at ──────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn tick_runs_delta_and_persists_next_poll_at() {
|
||||
let server = MockServer::start().await;
|
||||
empty_probe_and_albumlist(&server, 1_716_840_000_000).await;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
)
|
||||
.with_sleep_disabled()
|
||||
.tick(1_000)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert!(!report.skipped_not_due);
|
||||
assert!(!report.skipped_bulk_paused);
|
||||
assert!(report.delta.is_some());
|
||||
let next = SyncStateRepository::new(&store)
|
||||
.get_next_poll_at("s1", "")
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
assert_eq!(next, report.next_poll_at_ms);
|
||||
assert!(next > 1_000);
|
||||
}
|
||||
|
||||
// ── auto-tombstone trigger ──────────────────────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn tick_auto_tombstones_when_count_gap_exceeds_threshold() {
|
||||
let server = MockServer::start().await;
|
||||
empty_probe_and_albumlist(&server, 1_716_840_000_000).await;
|
||||
// Tombstone probe — empty store has nothing to probe, so we
|
||||
// only need to know the runner *would* have called getSong if
|
||||
// there were rows. For this test it's enough that no panic
|
||||
// occurs and the delta report's tombstone counters are zero.
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let sync_state = SyncStateRepository::new(&store);
|
||||
sync_state.ensure("s1", "").unwrap();
|
||||
// 110 local vs 100 server → 10 % gap, threshold 5 % default.
|
||||
sync_state.set_local_track_count("s1", "", 110).unwrap();
|
||||
sync_state.set_server_track_count("s1", "", 100).unwrap();
|
||||
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
)
|
||||
.with_sleep_disabled()
|
||||
.tick(0)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let delta = report.delta.expect("delta ran");
|
||||
// Tombstone budget was set (200), but no local tracks exist →
|
||||
// nothing to probe, both counters stay at 0. The important
|
||||
// signal is that the runner accepted the trigger.
|
||||
assert_eq!(delta.tombstones_checked, 0);
|
||||
assert_eq!(delta.tombstones_deleted, 0);
|
||||
}
|
||||
|
||||
// ── PollStats persistence round trip ────────────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn poll_stats_persist_round_trip_through_tick() {
|
||||
let server = MockServer::start().await;
|
||||
empty_probe_and_albumlist(&server, 1_716_840_000_000).await;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
BackgroundScheduler::new(
|
||||
&store,
|
||||
&subsonic,
|
||||
"s1",
|
||||
"",
|
||||
flags(CapabilityFlags::SUBSONIC_SEARCH3_BULK),
|
||||
)
|
||||
.with_sleep_disabled()
|
||||
.tick(0)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let stored = SyncStateRepository::new(&store)
|
||||
.get_poll_stats_json("s1", "")
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
// tier is recorded — runner reclassifies even with no
|
||||
// observations yet, so this is "unknown" on a fresh store.
|
||||
let stats: PollStats = serde_json::from_value(stored).unwrap();
|
||||
assert_eq!(stats.library_tier.as_tag(), "unknown");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,216 @@
|
||||
//! C2 ingest strategy selection. Per the PR-3 kickoff answer (workdocs
|
||||
//! `2026-05-19-pr3-kickoff.md` Q3) the choice is made once at initial
|
||||
//! sync start from the probed capability flags; the runner does not
|
||||
//! auto-switch on transient failure (C12 retries the same batch).
|
||||
|
||||
use super::capability::CapabilityFlags;
|
||||
|
||||
/// Server track count above which N1 is skipped in favour of S1 at initial
|
||||
/// sync start (R7-15 Q4). N1's native `/api/song` deep-offset 500 wall makes
|
||||
/// it unable to finish very large catalogs; S1 (`search3`) does not hit it.
|
||||
/// Tunable constant — the live wall was observed past ~50k.
|
||||
pub const LARGE_LIBRARY_THRESHOLD: i64 = 40_000;
|
||||
|
||||
/// Spec §6.3 IS-3 strategies. Names match §6.1.1 capability bits where
|
||||
/// applicable; S2 has no flag of its own — it's the universal
|
||||
/// album-crawl fallback assumed available whenever the Subsonic ping
|
||||
/// succeeds.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
|
||||
pub enum IngestStrategy {
|
||||
/// N1 — Navidrome native `GET /api/song` paginated. Cheapest at
|
||||
/// 500k; requires `NavidromeNativeBulk` flag set by the probe.
|
||||
N1,
|
||||
/// S1 — Subsonic `search3` empty query, songOffset paged. Requires
|
||||
/// `SubsonicSearch3Bulk`.
|
||||
S1,
|
||||
/// S2 — `getAlbumList2` + `getAlbum` per album. Universal Subsonic
|
||||
/// fallback — assumed available whenever the ping returns ok.
|
||||
S2,
|
||||
/// S3 — `getIndexes` + `getMusicDirectory` recursive file-tree
|
||||
/// crawl. Last resort; PR-3b does not auto-select it.
|
||||
S3,
|
||||
}
|
||||
|
||||
impl IngestStrategy {
|
||||
/// Pick the cheapest strategy supported by `flags`. Per kickoff Q3:
|
||||
/// `N1 → S1 → S2`. S3 is enumerated for completeness but never
|
||||
/// auto-selected — when neither N1 nor S1 is available, S2 is
|
||||
/// always tried first because every Subsonic-compliant server
|
||||
/// exposes `getAlbumList2` + `getAlbum`.
|
||||
pub fn select_from_flags(flags: CapabilityFlags) -> Self {
|
||||
if flags.contains(CapabilityFlags::NAVIDROME_NATIVE_BULK) {
|
||||
Self::N1
|
||||
} else if flags.contains(CapabilityFlags::SUBSONIC_SEARCH3_BULK) {
|
||||
Self::S1
|
||||
} else {
|
||||
Self::S2
|
||||
}
|
||||
}
|
||||
|
||||
/// Pick the initial-sync strategy with the large-library policy on top
|
||||
/// of `select_from_flags` (R7-15 Q4). A large catalog — or one already
|
||||
/// flagged `n1_bulk_unreliable` after an N1 deep-offset 500 — must avoid
|
||||
/// N1, which cannot finish past the wall. Prefer S1 (`search3`) and fall
|
||||
/// back to S2 (universal album crawl) when search3 bulk is unavailable.
|
||||
///
|
||||
/// `server_track_count` is best-effort at IS-1 (probe `getScanStatus`
|
||||
/// count or a prior watermark); `None` means unknown, in which case only
|
||||
/// the `n1_bulk_unreliable` flag forces the non-N1 path (a first run with
|
||||
/// no count still tries the cheapest strategy and learns from a 500).
|
||||
pub fn select_initial_strategy(
|
||||
flags: CapabilityFlags,
|
||||
server_track_count: Option<i64>,
|
||||
n1_bulk_unreliable: bool,
|
||||
) -> Self {
|
||||
let is_large = server_track_count
|
||||
.map(|c| c > LARGE_LIBRARY_THRESHOLD)
|
||||
.unwrap_or(false);
|
||||
if n1_bulk_unreliable || is_large {
|
||||
if flags.contains(CapabilityFlags::SUBSONIC_SEARCH3_BULK) {
|
||||
return Self::S1;
|
||||
}
|
||||
return Self::S2;
|
||||
}
|
||||
Self::select_from_flags(flags)
|
||||
}
|
||||
|
||||
/// String tag stored in `initial_sync_cursor_json` so the runner
|
||||
/// can resume after restart without re-running capability probe.
|
||||
pub fn as_tag(self) -> &'static str {
|
||||
match self {
|
||||
Self::N1 => "n1",
|
||||
Self::S1 => "s1",
|
||||
Self::S2 => "s2",
|
||||
Self::S3 => "s3",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn from_tag(tag: &str) -> Option<Self> {
|
||||
match tag {
|
||||
"n1" => Some(Self::N1),
|
||||
"s1" => Some(Self::S1),
|
||||
"s2" => Some(Self::S2),
|
||||
"s3" => Some(Self::S3),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn select_prefers_navidrome_native_when_both_n1_and_s1_present() {
|
||||
// At 500k, N1 cuts request count by 10× vs S1 — selector must
|
||||
// pick it whenever the flag is set, regardless of S1.
|
||||
let flags = CapabilityFlags::new(
|
||||
CapabilityFlags::NAVIDROME_NATIVE_BULK | CapabilityFlags::SUBSONIC_SEARCH3_BULK,
|
||||
);
|
||||
assert_eq!(IngestStrategy::select_from_flags(flags), IngestStrategy::N1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn select_falls_back_to_s1_without_n1() {
|
||||
let flags = CapabilityFlags::new(CapabilityFlags::SUBSONIC_SEARCH3_BULK);
|
||||
assert_eq!(IngestStrategy::select_from_flags(flags), IngestStrategy::S1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn select_falls_back_to_s2_when_no_bulk_flag_set() {
|
||||
// Generic Subsonic server without search3 bulk → universal
|
||||
// album crawl. S3 is not auto-selected even with FileTreeBrowse.
|
||||
let flags = CapabilityFlags::new(CapabilityFlags::FILE_TREE_BROWSE);
|
||||
assert_eq!(IngestStrategy::select_from_flags(flags), IngestStrategy::S2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn select_falls_back_to_s2_with_no_flags() {
|
||||
// Default-flag (`0x000`) — fresh DB before probe runs, or a
|
||||
// truly minimal Subsonic implementation. Still resolves to a
|
||||
// strategy; runner surfaces errors if S2 endpoints then fail.
|
||||
assert_eq!(
|
||||
IngestStrategy::select_from_flags(CapabilityFlags::default()),
|
||||
IngestStrategy::S2
|
||||
);
|
||||
}
|
||||
|
||||
// ── select_initial_strategy (R7-15 Q4 large-library policy) ──────────
|
||||
|
||||
fn navidrome_full() -> CapabilityFlags {
|
||||
CapabilityFlags::new(
|
||||
CapabilityFlags::NAVIDROME_NATIVE_BULK | CapabilityFlags::SUBSONIC_SEARCH3_BULK,
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn initial_strategy_small_library_keeps_cheapest_n1() {
|
||||
// Below threshold, not flagged unreliable → unchanged N1-first chain.
|
||||
let s = IngestStrategy::select_initial_strategy(navidrome_full(), Some(1_000), false);
|
||||
assert_eq!(s, IngestStrategy::N1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn initial_strategy_large_library_avoids_n1_for_s1() {
|
||||
// Over threshold → S1 even though N1 is advertised (deep-offset wall).
|
||||
let s = IngestStrategy::select_initial_strategy(navidrome_full(), Some(170_000), false);
|
||||
assert_eq!(s, IngestStrategy::S1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn initial_strategy_unreliable_flag_avoids_n1_regardless_of_size() {
|
||||
// Learned `n1_bulk_unreliable` forces the non-N1 path even when the
|
||||
// count is small/unknown (covers R7-15 "unknown → large if N1 failed").
|
||||
let small =
|
||||
IngestStrategy::select_initial_strategy(navidrome_full(), Some(500), true);
|
||||
assert_eq!(small, IngestStrategy::S1);
|
||||
let unknown = IngestStrategy::select_initial_strategy(navidrome_full(), None, true);
|
||||
assert_eq!(unknown, IngestStrategy::S1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn initial_strategy_large_without_search3_falls_back_to_s2() {
|
||||
// Avoid-N1 path but no search3 bulk → universal album crawl, not N1.
|
||||
let flags = CapabilityFlags::new(CapabilityFlags::NAVIDROME_NATIVE_BULK);
|
||||
let s = IngestStrategy::select_initial_strategy(flags, Some(170_000), false);
|
||||
assert_eq!(s, IngestStrategy::S2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn initial_strategy_unknown_count_uses_cheapest_when_not_flagged() {
|
||||
// First run, no count yet, N1 never failed → try cheapest (N1); the
|
||||
// mid-run N1→S1 fallback (R7-15 Q5) handles the wall if hit.
|
||||
let s = IngestStrategy::select_initial_strategy(navidrome_full(), None, false);
|
||||
assert_eq!(s, IngestStrategy::N1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn initial_strategy_threshold_is_strictly_greater_than() {
|
||||
// Exactly at the threshold is not "large"; one above is.
|
||||
let at = IngestStrategy::select_initial_strategy(
|
||||
navidrome_full(),
|
||||
Some(LARGE_LIBRARY_THRESHOLD),
|
||||
false,
|
||||
);
|
||||
assert_eq!(at, IngestStrategy::N1);
|
||||
let over = IngestStrategy::select_initial_strategy(
|
||||
navidrome_full(),
|
||||
Some(LARGE_LIBRARY_THRESHOLD + 1),
|
||||
false,
|
||||
);
|
||||
assert_eq!(over, IngestStrategy::S1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tag_roundtrip_is_stable_for_cursor_persistence() {
|
||||
for s in [
|
||||
IngestStrategy::N1,
|
||||
IngestStrategy::S1,
|
||||
IngestStrategy::S2,
|
||||
IngestStrategy::S3,
|
||||
] {
|
||||
assert_eq!(IngestStrategy::from_tag(s.as_tag()), Some(s));
|
||||
}
|
||||
assert_eq!(IngestStrategy::from_tag("unknown"), None);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,166 @@
|
||||
//! C5 — `SyncSupervisor`. Wraps the spawn / cancel / join lifecycle
|
||||
//! the top crate (PR-5) will use to run an `InitialSyncRunner` or
|
||||
//! `DeltaSyncRunner` from a Tauri command. The supervisor owns the
|
||||
//! cancellation `AtomicBool` and the `mpsc` receiver carrying
|
||||
//! progress events.
|
||||
//!
|
||||
//! Stays pure library — no Tauri imports here; PR-5 hooks the
|
||||
//! receiver to `AppHandle::emit`.
|
||||
|
||||
use std::sync::atomic::AtomicBool;
|
||||
use std::sync::atomic::Ordering;
|
||||
use std::sync::Arc;
|
||||
|
||||
use super::error::SyncError;
|
||||
use super::progress::{ChannelProgress, Progress, ProgressEvent};
|
||||
|
||||
pub struct SyncSupervisor {
|
||||
cancel: Arc<AtomicBool>,
|
||||
handle: Option<tokio::task::JoinHandle<Result<(), SyncError>>>,
|
||||
progress_rx: Option<tokio::sync::mpsc::UnboundedReceiver<ProgressEvent>>,
|
||||
}
|
||||
|
||||
impl SyncSupervisor {
|
||||
/// Spawn a sync workload. `task` receives the cancellation flag
|
||||
/// and a `Progress` handle backed by an internal mpsc channel.
|
||||
/// Caller drives the receiver via `progress_receiver()` and waits
|
||||
/// for completion via `join().await`.
|
||||
pub fn spawn<F, Fut>(task: F) -> Self
|
||||
where
|
||||
F: FnOnce(Arc<AtomicBool>, Arc<dyn Progress + Send + Sync>) -> Fut + Send + 'static,
|
||||
Fut: std::future::Future<Output = Result<(), SyncError>> + Send + 'static,
|
||||
{
|
||||
Self::spawn_with_interval(task, ChannelProgress::DEFAULT_INTERVAL)
|
||||
}
|
||||
|
||||
/// Variant that overrides the throttle interval — tests pass
|
||||
/// `Duration::ZERO` so every event observed in
|
||||
/// `progress_receiver()` matches the runner's emit order.
|
||||
pub fn spawn_with_interval<F, Fut>(task: F, throttle: std::time::Duration) -> Self
|
||||
where
|
||||
F: FnOnce(Arc<AtomicBool>, Arc<dyn Progress + Send + Sync>) -> Fut + Send + 'static,
|
||||
Fut: std::future::Future<Output = Result<(), SyncError>> + Send + 'static,
|
||||
{
|
||||
let cancel = Arc::new(AtomicBool::new(false));
|
||||
let (tx, rx) = tokio::sync::mpsc::unbounded_channel();
|
||||
let progress: Arc<dyn Progress + Send + Sync> =
|
||||
Arc::new(ChannelProgress::with_interval(tx, throttle));
|
||||
let cancel_clone = Arc::clone(&cancel);
|
||||
let handle = tokio::task::spawn(async move { task(cancel_clone, progress).await });
|
||||
Self {
|
||||
cancel,
|
||||
handle: Some(handle),
|
||||
progress_rx: Some(rx),
|
||||
}
|
||||
}
|
||||
|
||||
/// Trip the cancellation flag. Runners check it between batches
|
||||
/// and bail out with `SyncError::Cancelled` on the next iteration.
|
||||
pub fn cancel(&self) {
|
||||
self.cancel.store(true, Ordering::SeqCst);
|
||||
}
|
||||
|
||||
pub fn is_cancelled(&self) -> bool {
|
||||
self.cancel.load(Ordering::SeqCst)
|
||||
}
|
||||
|
||||
/// Take the progress receiver. Single consumer only — the
|
||||
/// `Option::take` semantics mean later calls return `None`.
|
||||
pub fn progress_receiver(
|
||||
&mut self,
|
||||
) -> Option<tokio::sync::mpsc::UnboundedReceiver<ProgressEvent>> {
|
||||
self.progress_rx.take()
|
||||
}
|
||||
|
||||
/// Wait for the spawned task. Returns the task's `Result` —
|
||||
/// `Ok(Ok(()))` on clean exit, `Ok(Err(SyncError))` on a
|
||||
/// runner-level failure, `Err(JoinError)` only if the task
|
||||
/// panicked, in which case we surface that as `SyncError::Storage`
|
||||
/// so callers never need to know about tokio internals.
|
||||
pub async fn join(mut self) -> Result<(), SyncError> {
|
||||
let handle = self
|
||||
.handle
|
||||
.take()
|
||||
.ok_or_else(|| SyncError::Storage("supervisor already joined".into()))?;
|
||||
match handle.await {
|
||||
Ok(inner) => inner,
|
||||
Err(join_err) => Err(SyncError::Storage(format!(
|
||||
"sync task panicked: {join_err}"
|
||||
))),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::time::Duration;
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn supervisor_runs_task_to_completion_and_emits_progress() {
|
||||
let mut sup = SyncSupervisor::spawn_with_interval(
|
||||
|_cancel, progress| async move {
|
||||
progress.emit(ProgressEvent::PhaseChanged { phase: "ingest".into() });
|
||||
progress.emit(ProgressEvent::Completed {
|
||||
kind: "initial_sync".into(),
|
||||
});
|
||||
Ok(())
|
||||
},
|
||||
Duration::ZERO,
|
||||
);
|
||||
let mut rx = sup.progress_receiver().expect("receiver still in place");
|
||||
let result = sup.join().await;
|
||||
assert!(result.is_ok());
|
||||
let mut events = Vec::new();
|
||||
while let Ok(ev) = rx.try_recv() {
|
||||
events.push(ev);
|
||||
}
|
||||
assert_eq!(events.len(), 2);
|
||||
assert!(matches!(events[0], ProgressEvent::PhaseChanged { .. }));
|
||||
assert!(matches!(events[1], ProgressEvent::Completed { .. }));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn supervisor_cancel_trips_atomic_flag_before_task_joins() {
|
||||
let started = Arc::new(tokio::sync::Notify::new());
|
||||
let started_clone = Arc::clone(&started);
|
||||
let sup = SyncSupervisor::spawn(move |cancel, _progress| {
|
||||
let started = started_clone;
|
||||
async move {
|
||||
started.notify_one();
|
||||
// Spin until cancelled or 2s timeout.
|
||||
for _ in 0..200 {
|
||||
if cancel.load(Ordering::SeqCst) {
|
||||
return Err(SyncError::Cancelled);
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(10)).await;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
});
|
||||
started.notified().await;
|
||||
sup.cancel();
|
||||
assert!(sup.is_cancelled());
|
||||
let result = sup.join().await;
|
||||
assert!(matches!(result, Err(SyncError::Cancelled)));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn supervisor_propagates_task_panic_as_sync_error() {
|
||||
let sup = SyncSupervisor::spawn(|_cancel, _progress| async move {
|
||||
panic!("simulated task panic");
|
||||
});
|
||||
let result = sup.join().await;
|
||||
assert!(matches!(result, Err(SyncError::Storage(_))));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn progress_receiver_take_returns_none_after_first_call() {
|
||||
let mut sup = SyncSupervisor::spawn(|_cancel, _progress| async move { Ok(()) });
|
||||
let first = sup.progress_receiver();
|
||||
let second = sup.progress_receiver();
|
||||
assert!(first.is_some());
|
||||
assert!(second.is_none());
|
||||
sup.join().await.unwrap();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,447 @@
|
||||
//! C4 — `TombstoneReconciler` (spec §6.7).
|
||||
//!
|
||||
//! Streams a chunk of local track ids, hits `getSong` per id, and
|
||||
//! marks `track.deleted = 1` for every `SubsonicError::NotFound`
|
||||
//! (error code 70). Designed for two callers:
|
||||
//!
|
||||
//! - **Mode A (manual integrity check):** Settings → "Verify library
|
||||
//! integrity" loops `reconcile_chunk(N)` until it returns
|
||||
//! `checked == 0`.
|
||||
//! - **Mode B (auto, threshold-triggered):** the delta scheduler
|
||||
//! tests `should_auto_reconcile` against the count drop, then loops
|
||||
//! `reconcile_chunk(budget)` once per delta tick until the gap
|
||||
//! closes.
|
||||
//!
|
||||
//! Streaming so memory stays bounded at 500k: `LIMIT N ORDER BY
|
||||
//! synced_at ASC` picks the next chunk; PR-3c keeps the loop entirely
|
||||
//! caller-driven so cancellation is checked between chunks.
|
||||
|
||||
use std::sync::atomic::Ordering;
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use psysonic_integration::subsonic::{SubsonicClient, SubsonicError};
|
||||
|
||||
use super::backoff::{jitter_salt, with_jitter, Backoff};
|
||||
use super::error::SyncError;
|
||||
use crate::store::LibraryStore;
|
||||
|
||||
const MAX_ATTEMPTS_PER_BATCH: u32 = 5;
|
||||
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq)]
|
||||
pub struct TombstoneReport {
|
||||
pub checked: u32,
|
||||
pub deleted: u32,
|
||||
}
|
||||
|
||||
pub struct TombstoneReconciler<'a> {
|
||||
store: &'a LibraryStore,
|
||||
subsonic: &'a SubsonicClient,
|
||||
server_id: String,
|
||||
cancel: Option<Arc<std::sync::atomic::AtomicBool>>,
|
||||
sleep_enabled: bool,
|
||||
}
|
||||
|
||||
impl<'a> TombstoneReconciler<'a> {
|
||||
pub fn new(
|
||||
store: &'a LibraryStore,
|
||||
subsonic: &'a SubsonicClient,
|
||||
server_id: impl Into<String>,
|
||||
) -> Self {
|
||||
Self {
|
||||
store,
|
||||
subsonic,
|
||||
server_id: server_id.into(),
|
||||
cancel: None,
|
||||
sleep_enabled: true,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn with_cancellation(mut self, flag: Arc<std::sync::atomic::AtomicBool>) -> Self {
|
||||
self.cancel = Some(flag);
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_sleep_disabled(mut self) -> Self {
|
||||
self.sleep_enabled = false;
|
||||
self
|
||||
}
|
||||
|
||||
/// Process up to `budget` not-yet-checked tracks. Returns counts
|
||||
/// for this call only — caller loops until `checked == 0` to
|
||||
/// complete a Mode A pass, or stops at any budget for Mode B
|
||||
/// sampled passes. Order: oldest `synced_at` first so the most
|
||||
/// stale rows get re-validated soonest.
|
||||
pub async fn reconcile_chunk(&self, budget: u32) -> Result<TombstoneReport, SyncError> {
|
||||
if budget == 0 {
|
||||
return Ok(TombstoneReport::default());
|
||||
}
|
||||
let ids = self.next_candidates(budget)?;
|
||||
let mut report = TombstoneReport::default();
|
||||
for id in ids {
|
||||
self.check_cancellation()?;
|
||||
report.checked = report.checked.saturating_add(1);
|
||||
let outcome = retry_with_backoff(
|
||||
self,
|
||||
|| self.subsonic.get_song(&id),
|
||||
|e: SubsonicError| -> SyncError { e.into() },
|
||||
)
|
||||
.await;
|
||||
match outcome {
|
||||
Ok(_) => {
|
||||
// Still present — stamp `synced_at` so it goes to
|
||||
// the back of the queue and we don't re-probe it
|
||||
// again on the next chunk.
|
||||
self.mark_synced(&id)?;
|
||||
}
|
||||
Err(SyncError::NotFound) => {
|
||||
self.mark_deleted(&id)?;
|
||||
report.deleted = report.deleted.saturating_add(1);
|
||||
}
|
||||
Err(other) => return Err(other),
|
||||
}
|
||||
}
|
||||
Ok(report)
|
||||
}
|
||||
|
||||
fn check_cancellation(&self) -> Result<(), SyncError> {
|
||||
if let Some(flag) = &self.cancel {
|
||||
if flag.load(Ordering::SeqCst) {
|
||||
return Err(SyncError::Cancelled);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn next_candidates(&self, budget: u32) -> Result<Vec<String>, SyncError> {
|
||||
self.store
|
||||
.with_conn("misc", |c| {
|
||||
let mut stmt = c.prepare(
|
||||
"SELECT id FROM track \
|
||||
WHERE server_id = ?1 AND deleted = 0 \
|
||||
ORDER BY synced_at ASC LIMIT ?2",
|
||||
)?;
|
||||
let rows: rusqlite::Result<Vec<String>> = stmt
|
||||
.query_map(rusqlite::params![self.server_id, budget as i64], |r| {
|
||||
r.get::<_, String>(0)
|
||||
})?
|
||||
.collect();
|
||||
rows
|
||||
})
|
||||
.map_err(SyncError::Storage)
|
||||
}
|
||||
|
||||
fn mark_deleted(&self, id: &str) -> Result<(), SyncError> {
|
||||
self.store
|
||||
.with_conn("misc", |c| {
|
||||
c.execute(
|
||||
"UPDATE track SET deleted = 1, synced_at = ?3 \
|
||||
WHERE server_id = ?1 AND id = ?2",
|
||||
rusqlite::params![self.server_id, id, now_unix_ms()],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
.map_err(SyncError::Storage)
|
||||
}
|
||||
|
||||
fn mark_synced(&self, id: &str) -> Result<(), SyncError> {
|
||||
self.store
|
||||
.with_conn("misc", |c| {
|
||||
c.execute(
|
||||
"UPDATE track SET synced_at = ?3 \
|
||||
WHERE server_id = ?1 AND id = ?2",
|
||||
rusqlite::params![self.server_id, id, now_unix_ms()],
|
||||
)?;
|
||||
Ok(())
|
||||
})
|
||||
.map_err(SyncError::Storage)
|
||||
}
|
||||
|
||||
async fn sleep(&self, d: Duration) {
|
||||
if self.sleep_enabled && !d.is_zero() {
|
||||
tokio::time::sleep(d).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// §6.7 Mode B threshold check — returns `true` when the local /
|
||||
/// server count gap exceeds the configured percentage. `server_count
|
||||
/// == 0` is treated as "no signal" → `false` (no spurious reconcile
|
||||
/// on a fresh server response).
|
||||
pub fn should_auto_reconcile(local_count: u32, server_count: u32, threshold_pct: u32) -> bool {
|
||||
if server_count == 0 {
|
||||
return false;
|
||||
}
|
||||
let gap = local_count.saturating_sub(server_count);
|
||||
let ratio_x100 = gap.saturating_mul(100) / server_count;
|
||||
ratio_x100 > threshold_pct
|
||||
}
|
||||
|
||||
fn now_unix_ms() -> i64 {
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.map(|d| d.as_millis().min(i64::MAX as u128) as i64)
|
||||
.unwrap_or(0)
|
||||
}
|
||||
|
||||
async fn retry_with_backoff<'a, F, FFut, T, E>(
|
||||
reconciler: &TombstoneReconciler<'a>,
|
||||
mut build: F,
|
||||
map_err: impl Fn(E) -> SyncError,
|
||||
) -> Result<T, SyncError>
|
||||
where
|
||||
F: FnMut() -> FFut,
|
||||
FFut: std::future::Future<Output = Result<T, E>>,
|
||||
{
|
||||
let mut backoff = Backoff::default();
|
||||
let mut attempt = 0u32;
|
||||
loop {
|
||||
reconciler.check_cancellation()?;
|
||||
attempt += 1;
|
||||
match build().await {
|
||||
Ok(v) => return Ok(v),
|
||||
Err(e) => {
|
||||
let mapped = map_err(e);
|
||||
if !is_retryable(&mapped) || attempt >= MAX_ATTEMPTS_PER_BATCH {
|
||||
return Err(mapped);
|
||||
}
|
||||
let delay = backoff.next_delay();
|
||||
let jittered = with_jitter(delay, jitter_salt(attempt));
|
||||
reconciler.sleep(jittered).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn is_retryable(e: &SyncError) -> bool {
|
||||
matches!(e, SyncError::Transport(_) | SyncError::Navidrome(_))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::repos::{TrackRepository, TrackRow};
|
||||
use psysonic_integration::subsonic::{SubsonicClient, SubsonicCredentials};
|
||||
use serde_json::json;
|
||||
use wiremock::matchers::{method as wm_method, path as wm_path, query_param};
|
||||
use wiremock::{Mock, MockServer, ResponseTemplate};
|
||||
|
||||
fn test_subsonic(uri: &str) -> SubsonicClient {
|
||||
SubsonicClient::with_static_credentials(
|
||||
uri,
|
||||
SubsonicCredentials::with_static("user", "tok", "salt"),
|
||||
reqwest::Client::new(),
|
||||
)
|
||||
}
|
||||
|
||||
fn seed_track(store: &LibraryStore, id: &str, synced_at: i64) {
|
||||
TrackRepository::new(store)
|
||||
.upsert_batch(&[TrackRow {
|
||||
server_id: "s1".into(),
|
||||
id: id.into(),
|
||||
title: id.into(),
|
||||
title_sort: None,
|
||||
artist: None,
|
||||
artist_id: None,
|
||||
album: String::new(),
|
||||
album_id: None,
|
||||
album_artist: None,
|
||||
duration_sec: 0,
|
||||
track_number: None,
|
||||
disc_number: None,
|
||||
year: None,
|
||||
genre: None,
|
||||
suffix: None,
|
||||
bit_rate: None,
|
||||
size_bytes: None,
|
||||
cover_art_id: None,
|
||||
starred_at: None,
|
||||
user_rating: None,
|
||||
play_count: None,
|
||||
played_at: None,
|
||||
server_path: None,
|
||||
library_id: None,
|
||||
isrc: None,
|
||||
mbid_recording: None,
|
||||
bpm: None,
|
||||
replay_gain_track_db: None,
|
||||
replay_gain_album_db: None,
|
||||
content_hash: None,
|
||||
server_updated_at: None,
|
||||
server_created_at: None,
|
||||
deleted: false,
|
||||
synced_at,
|
||||
raw_json: "{}".into(),
|
||||
}])
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
// ── should_auto_reconcile threshold predicate ─────────────────────
|
||||
|
||||
#[test]
|
||||
fn threshold_fires_when_local_outpaces_server_above_pct() {
|
||||
// 110 local vs 100 server → 10% gap > 5% threshold.
|
||||
assert!(should_auto_reconcile(110, 100, 5));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn threshold_stays_silent_within_tolerance() {
|
||||
// 102 local vs 100 server → 2% gap, threshold 5%.
|
||||
assert!(!should_auto_reconcile(102, 100, 5));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn threshold_silent_when_local_is_below_or_equal_server() {
|
||||
assert!(!should_auto_reconcile(100, 100, 0));
|
||||
assert!(!should_auto_reconcile(50, 100, 5));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn threshold_silent_when_server_count_is_zero() {
|
||||
// No signal — never reconcile on a server that's still scanning.
|
||||
assert!(!should_auto_reconcile(1000, 0, 5));
|
||||
}
|
||||
|
||||
// ── reconcile_chunk marks deleted on code 70 ─────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn reconcile_chunk_marks_deleted_for_code_70() {
|
||||
let server = MockServer::start().await;
|
||||
// tr_a → still present, tr_b → 404 via code 70.
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getSong.view"))
|
||||
.and(query_param("id", "tr_a"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": {
|
||||
"status": "ok",
|
||||
"song": { "id": "tr_a", "title": "Still here" }
|
||||
}
|
||||
})))
|
||||
.mount(&server)
|
||||
.await;
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getSong.view"))
|
||||
.and(query_param("id", "tr_b"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": {
|
||||
"status": "failed",
|
||||
"error": { "code": 70, "message": "Song not found" }
|
||||
}
|
||||
})))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "tr_a", 1);
|
||||
seed_track(&store, "tr_b", 2);
|
||||
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = TombstoneReconciler::new(&store, &subsonic, "s1")
|
||||
.with_sleep_disabled()
|
||||
.reconcile_chunk(10)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(report.checked, 2);
|
||||
assert_eq!(report.deleted, 1);
|
||||
|
||||
// tr_b is marked deleted; tr_a stays live but its synced_at is
|
||||
// refreshed (so it doesn't get re-picked immediately).
|
||||
let (a_deleted, b_deleted): (i64, i64) = store
|
||||
.with_conn("misc", |c| {
|
||||
let a: i64 = c.query_row(
|
||||
"SELECT deleted FROM track WHERE id='tr_a'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)?;
|
||||
let b: i64 = c.query_row(
|
||||
"SELECT deleted FROM track WHERE id='tr_b'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)?;
|
||||
Ok((a, b))
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(a_deleted, 0);
|
||||
assert_eq!(b_deleted, 1);
|
||||
}
|
||||
|
||||
// ── reconcile_chunk respects budget and ordering ─────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn reconcile_chunk_processes_oldest_first_up_to_budget() {
|
||||
let server = MockServer::start().await;
|
||||
// Any id → ok envelope.
|
||||
Mock::given(wm_method("GET"))
|
||||
.and(wm_path("/rest/getSong.view"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"subsonic-response": {
|
||||
"status": "ok",
|
||||
"song": { "id": "any", "title": "t" }
|
||||
}
|
||||
})))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
let store = LibraryStore::open_in_memory();
|
||||
// Seed three tracks with distinct synced_at values; oldest first.
|
||||
seed_track(&store, "tr_oldest", 100);
|
||||
seed_track(&store, "tr_middle", 200);
|
||||
seed_track(&store, "tr_newest", 300);
|
||||
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = TombstoneReconciler::new(&store, &subsonic, "s1")
|
||||
.with_sleep_disabled()
|
||||
.reconcile_chunk(2)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(report.checked, 2);
|
||||
|
||||
// After the chunk: the two checked rows have a refreshed
|
||||
// synced_at; the un-checked tr_newest still sits at 300.
|
||||
let untouched: i64 = store
|
||||
.with_conn("misc", |c| c.query_row(
|
||||
"SELECT synced_at FROM track WHERE id='tr_newest'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
))
|
||||
.unwrap();
|
||||
assert_eq!(untouched, 300, "tr_newest must not be probed within budget=2");
|
||||
}
|
||||
|
||||
// ── reconcile_chunk: empty store ───────────────────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn reconcile_chunk_returns_zero_counts_on_empty_store() {
|
||||
let server = MockServer::start().await;
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let report = TombstoneReconciler::new(&store, &subsonic, "s1")
|
||||
.with_sleep_disabled()
|
||||
.reconcile_chunk(50)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(report.checked, 0);
|
||||
assert_eq!(report.deleted, 0);
|
||||
}
|
||||
|
||||
// ── reconcile_chunk: cancellation ─────────────────────────────────
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn reconcile_chunk_returns_cancelled_when_flag_tripped() {
|
||||
let server = MockServer::start().await;
|
||||
let store = LibraryStore::open_in_memory();
|
||||
seed_track(&store, "tr_x", 1);
|
||||
|
||||
let flag = Arc::new(std::sync::atomic::AtomicBool::new(true));
|
||||
let subsonic = test_subsonic(&server.uri());
|
||||
let err = TombstoneReconciler::new(&store, &subsonic, "s1")
|
||||
.with_cancellation(flag)
|
||||
.with_sleep_disabled()
|
||||
.reconcile_chunk(10)
|
||||
.await
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, SyncError::Cancelled));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,156 @@
|
||||
//! FTS5 maintenance for bulk initial-sync ingest.
|
||||
//!
|
||||
//! Per-row FTS triggers dominate N1/S1 write time at scale. IS-3 suspends
|
||||
//! them, loads tracks without FTS maintenance, then rebuilds from the
|
||||
//! external-content table once before restoring triggers.
|
||||
|
||||
use rusqlite::Connection;
|
||||
|
||||
const RESTORE_TRACK_FTS_TRIGGERS: &str = r#"
|
||||
CREATE TRIGGER IF NOT EXISTS track_ai AFTER INSERT ON track BEGIN
|
||||
INSERT INTO track_fts(rowid, title, artist, album, album_artist, genre)
|
||||
VALUES (new.rowid, new.title, new.artist, new.album, new.album_artist, new.genre);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS track_ad AFTER DELETE ON track BEGIN
|
||||
INSERT INTO track_fts(track_fts, rowid, title, artist, album, album_artist, genre)
|
||||
VALUES ('delete', old.rowid, old.title, old.artist, old.album, old.album_artist, old.genre);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS track_au AFTER UPDATE ON track BEGIN
|
||||
INSERT INTO track_fts(track_fts, rowid, title, artist, album, album_artist, genre)
|
||||
VALUES ('delete', old.rowid, old.title, old.artist, old.album, old.album_artist, old.genre);
|
||||
INSERT INTO track_fts(rowid, title, artist, album, album_artist, genre)
|
||||
VALUES (new.rowid, new.title, new.artist, new.album, new.album_artist, new.genre);
|
||||
END;
|
||||
"#;
|
||||
|
||||
/// Drop FTS sync triggers so bulk upserts skip per-row FTS maintenance.
|
||||
pub fn suspend_track_fts_triggers(conn: &Connection) -> rusqlite::Result<()> {
|
||||
conn.execute_batch(
|
||||
"DROP TRIGGER IF EXISTS track_ai;
|
||||
DROP TRIGGER IF EXISTS track_ad;
|
||||
DROP TRIGGER IF EXISTS track_au;",
|
||||
)
|
||||
}
|
||||
|
||||
/// Rebuild `track_fts` from the `track` content table (fts5 external content).
|
||||
pub fn rebuild_track_fts_from_content(conn: &Connection) -> rusqlite::Result<()> {
|
||||
conn.execute("INSERT INTO track_fts(track_fts) VALUES('rebuild')", [])?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Restore FTS sync triggers after bulk ingest + rebuild.
|
||||
pub fn restore_track_fts_triggers(conn: &Connection) -> rusqlite::Result<()> {
|
||||
conn.execute_batch(RESTORE_TRACK_FTS_TRIGGERS)
|
||||
}
|
||||
|
||||
/// Normalize trigger DDL for equality checks — strips `IF NOT EXISTS`
|
||||
/// and collapses whitespace so migration vs restore sources compare cleanly.
|
||||
#[cfg(test)]
|
||||
pub(crate) fn normalize_trigger_ddl(sql: &str) -> String {
|
||||
sql.replace("IF NOT EXISTS", "")
|
||||
.split_whitespace()
|
||||
.collect::<Vec<_>>()
|
||||
.join(" ")
|
||||
.to_lowercase()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::store::LibraryStore;
|
||||
use std::collections::HashMap;
|
||||
|
||||
const TRACK_FTS_TRIGGER_NAMES: [&str; 3] = ["track_ai", "track_ad", "track_au"];
|
||||
|
||||
fn fetch_track_fts_trigger_ddl(conn: &Connection, name: &str) -> String {
|
||||
conn.query_row(
|
||||
"SELECT sql FROM sqlite_master WHERE type = 'trigger' AND name = ?1",
|
||||
[name],
|
||||
|r| r.get(0),
|
||||
)
|
||||
.unwrap_or_else(|e| panic!("missing trigger `{name}`: {e}"))
|
||||
}
|
||||
|
||||
fn migration_track_fts_trigger_bodies(
|
||||
conn: &Connection,
|
||||
) -> rusqlite::Result<HashMap<String, String>> {
|
||||
Ok(TRACK_FTS_TRIGGER_NAMES
|
||||
.iter()
|
||||
.map(|name| {
|
||||
let ddl = fetch_track_fts_trigger_ddl(conn, name);
|
||||
(name.to_string(), normalize_trigger_ddl(&ddl))
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn restore_track_fts_triggers_match_migration_bodies() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
let baseline = store
|
||||
.with_conn("misc", migration_track_fts_trigger_bodies)
|
||||
.unwrap();
|
||||
|
||||
store
|
||||
.with_conn_mut("misc", |conn| {
|
||||
suspend_track_fts_triggers(conn)?;
|
||||
restore_track_fts_triggers(conn)?;
|
||||
for name in TRACK_FTS_TRIGGER_NAMES {
|
||||
let after = normalize_trigger_ddl(&fetch_track_fts_trigger_ddl(conn, name));
|
||||
assert_eq!(
|
||||
after,
|
||||
baseline[name],
|
||||
"trigger `{name}` body drifted after suspend/restore"
|
||||
);
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bulk_ingest_suspend_rebuild_restores_fts_search() {
|
||||
let store = LibraryStore::open_in_memory();
|
||||
store
|
||||
.with_conn_mut("misc", |conn| suspend_track_fts_triggers(conn))
|
||||
.unwrap();
|
||||
store
|
||||
.with_conn_mut("misc", |conn| {
|
||||
conn.execute(
|
||||
"INSERT INTO track (server_id, id, title, album, duration_sec, deleted, synced_at, raw_json) \
|
||||
VALUES ('s1', 't1', 'Bulk Title', 'Album', 1, 0, 1, '{}')",
|
||||
[],
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
let mid: i64 = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT COUNT(*) FROM track_fts WHERE track_fts MATCH 'Bulk'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(mid, 0, "FTS must not update while triggers are suspended");
|
||||
|
||||
store
|
||||
.with_conn_mut("misc", |conn| {
|
||||
rebuild_track_fts_from_content(conn)?;
|
||||
restore_track_fts_triggers(conn)
|
||||
})
|
||||
.unwrap();
|
||||
|
||||
let after: i64 = store
|
||||
.with_conn("misc", |c| {
|
||||
c.query_row(
|
||||
"SELECT COUNT(*) FROM track_fts WHERE track_fts MATCH 'Bulk'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(after, 1);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user