mirror of
https://github.com/kilyabin/psysonic.git
synced 2026-07-21 22:15:40 +00:00
fix(cover-backfill): kill idle CPU spin and offline-cache menu re-walks (#943)
* fix(cover-backfill): snapshot-diff worklist and live-tunable parallelism Aggressive cover backfill pegged one tokio worker at ~100% on large, fully-synced libraries while the download queues stayed empty. - Take two snapshots once per pass — the DB catalog (single GROUP BY) and the on-disk cover bucket (one directory walk) — and download the set-difference. No per-row `stat` syscalls and no re-scan loop; the empty cache case (heavy backfill) costs zero per-item disk hits. - Replace the front-loaded enumeration with a producer/consumer pipeline: the producer streams the catalog in chunks and feeds misses into a bounded channel; a fixed consumer pool keeps the download/encode pools saturated. - Make cover backfill parallelism runtime-tunable from the Performance Probe (threads slider + "Run full pass now"); HTTP download and CPU encode semaphores resize live. Not surfaced in app settings. - Add a "nothing changed" idle gate (catalog signature) so a settled pass is not re-run on every library:sync-idle, mirroring the analysis worker. - Cancel promptly on switch to lazy: consumers bail on enabled/focus change and the producer feeds via try_send so a full channel cannot deadlock. - Drop the per-item recursive disk walk from the ensure hot path. * fix(cover-backfill): cheap idle gate, settle on 404s, transient retries Follow-up to the snapshot-diff backfill: stop the periodic CPU spikes and the 89%-plateau wake storm on libraries whose covers can never reach 100%. - Idle gate is now disk-free: compare only the catalog COUNT(DISTINCT) instead of walking ~all cover dirs on every sync-idle. "Did the server change?" never touches the filesystem. Clear-cache commands re-arm the gate (rearm_idle_gate) since a clear leaves the catalog total unchanged, and the settings UI wakes the active server after a clear. - Settle the gate on any completed pass regardless of pending: remaining items are unfetchable-for-now (404), so the wake/sync-idle storm stops once the fetchable set is exhausted. - Stop auto-clearing .fetch-failed markers every pass (it defeated the 30-min backoff and re-attempted 404s forever). The manual "Run full pass now" sends force=true to clear them and retry; wake/sync-idle/configure stay opportunistic. - Rate-limit sync-idle passes (60s cooldown) as defence against chatty syncs. - Retry cover downloads up to 3x with backoff on transient failures (5xx / 429 / network), but never on a real 4xx so missing covers don't hammer the server. * fix(cover-cache): stop re-walking cover dirs from offline & cache menu The settings cover-cache section polled disk usage + progress every 15s for every server, each call doing a full recursive walk of the per-server cover directory. On a fully populated cache this caused periodic CPU spikes whenever that menu was open. - mod.rs: add a 10s TTL memo around the per-server cover dir walk (cached_dir_usage_for_server), shared by cover_cache_stats_server and library_cover_progress; invalidate on clear (per-server and clear-all). - CoverCacheStrategySection: recompute on entry only; rely on the cover:library-progress and cover:cache-cleared events for live updates; drop the per-cover cover:tier-ready refresh storm; turn the 15s loop into a 5-minute safety net. * fix(cover-backfill): keep emitting progress during the whole pass The producer finishes enumerating the worklist long before the consumer pool finishes downloading it, so progress was only emitted while feeding the channel — the "offline & cache" menu and overlay then froze through the entire drain phase. Replace the per-chunk emit with a 3s progress ticker that runs for the lifetime of the pass and is aborted once the consumers drain (final accurate emit still happens at settle). * docs(changelog): record cover-backfill idle-CPU fix (PR #943) Add [1.47.0] Fixed + Changed entries and a settingsCredits line for the cover-backfill idle CPU / offline & cache menu work.
This commit is contained in:
@@ -359,6 +359,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
* Frontend and Rust `coverage` jobs no longer carry `continue-on-error`; listed hot-path files must stay at ≥70% line coverage or the PR fails.
|
||||
|
||||
### Cover backfill — live-tunable parallelism and pipeline
|
||||
|
||||
**By [@cucadmuh](https://github.com/cucadmuh), PR [#943](https://github.com/Psychotoxical/psysonic/pull/943)**
|
||||
|
||||
* Cover backfill runs through a producer/consumer pipeline (bounded channel + fixed consumer pool) that stays saturated and bails promptly on a switch to **lazy** instead of draining the whole backlog.
|
||||
* **Performance Probe** gains a runtime cover-thread control (`library_cover_backfill_set_parallel`) that resizes the HTTP/encode pools live; "Run full pass now" forces a pass and clears fetch-failed backoff.
|
||||
* Clearing the active server's cover cache re-arms the idle gate and wakes the worker, and in-pass progress is emitted on a ticker so the offline & cache view keeps counting through the whole scan.
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -645,7 +653,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
* Pausing a track and resuming later inflated the listening time in **Statistics → Player stats** — the whole paused span was billed as if the track had been playing.
|
||||
* Root cause: the session's tick baseline froze on pause, so the first progress tick after resume measured against the pre-pause timestamp. It now settles the played segment on pause and rebaselines on resume.
|
||||
|
||||
### Cover backfill — idle CPU spin and offline & cache menu spikes
|
||||
|
||||
**By [@cucadmuh](https://github.com/cucadmuh), PR [#943](https://github.com/Psychotoxical/psysonic/pull/943)**
|
||||
|
||||
* **Aggressive** cover backfill could pin a `tokio-runtime-worker` near 100% CPU when effectively idle: it had no "nothing changed, don't rescan" gate. Added a cheap disk-free idle signature (`COUNT(DISTINCT)` covers) with a `sync-idle` cooldown; the gate settles on a completed pass even when some covers are unfetchable (404), so libraries that never reach 100% no longer trigger a wake storm.
|
||||
* Worklist is now built from a single DB `GROUP BY` plus one cover-dir snapshot and diffed in memory — no per-row `stat`, no per-batch rescan loop — so increasing parallelism actually saturates the pipeline.
|
||||
* **Settings → offline & cache** caused periodic CPU spikes: the section re-walked each server's full cover directory every 15s. The per-server walk is now memoized with a short TTL and reused by the stats/progress commands; the menu recomputes on entry and via progress/cache-cleared events, with a 5-minute safety poll instead of a tight 15s loop.
|
||||
* Transient download failures (network / 5xx / 429) retry up to 3× with exponential backoff; permanent 4xx settle without re-scanning.
|
||||
|
||||
## [1.46.0] - 2026-05-18
|
||||
|
||||
|
||||
Reference in New Issue
Block a user