fix(cover-backfill): kill idle CPU spin and offline-cache menu re-walks (#943)

* fix(cover-backfill): snapshot-diff worklist and live-tunable parallelism

Aggressive cover backfill pegged one tokio worker at ~100% on large,
fully-synced libraries while the download queues stayed empty.

- Take two snapshots once per pass — the DB catalog (single GROUP BY) and
  the on-disk cover bucket (one directory walk) — and download the
  set-difference. No per-row `stat` syscalls and no re-scan loop; the empty
  cache case (heavy backfill) costs zero per-item disk hits.
- Replace the front-loaded enumeration with a producer/consumer pipeline:
  the producer streams the catalog in chunks and feeds misses into a bounded
  channel; a fixed consumer pool keeps the download/encode pools saturated.
- Make cover backfill parallelism runtime-tunable from the Performance Probe
  (threads slider + "Run full pass now"); HTTP download and CPU encode
  semaphores resize live. Not surfaced in app settings.
- Add a "nothing changed" idle gate (catalog signature) so a settled pass is
  not re-run on every library:sync-idle, mirroring the analysis worker.
- Cancel promptly on switch to lazy: consumers bail on enabled/focus change
  and the producer feeds via try_send so a full channel cannot deadlock.
- Drop the per-item recursive disk walk from the ensure hot path.

* fix(cover-backfill): cheap idle gate, settle on 404s, transient retries

Follow-up to the snapshot-diff backfill: stop the periodic CPU spikes and the
89%-plateau wake storm on libraries whose covers can never reach 100%.

- Idle gate is now disk-free: compare only the catalog COUNT(DISTINCT) instead
  of walking ~all cover dirs on every sync-idle. "Did the server change?" never
  touches the filesystem. Clear-cache commands re-arm the gate (rearm_idle_gate)
  since a clear leaves the catalog total unchanged, and the settings UI wakes the
  active server after a clear.
- Settle the gate on any completed pass regardless of pending: remaining items
  are unfetchable-for-now (404), so the wake/sync-idle storm stops once the
  fetchable set is exhausted.
- Stop auto-clearing .fetch-failed markers every pass (it defeated the 30-min
  backoff and re-attempted 404s forever). The manual "Run full pass now" sends
  force=true to clear them and retry; wake/sync-idle/configure stay opportunistic.
- Rate-limit sync-idle passes (60s cooldown) as defence against chatty syncs.
- Retry cover downloads up to 3x with backoff on transient failures (5xx / 429 /
  network), but never on a real 4xx so missing covers don't hammer the server.

* fix(cover-cache): stop re-walking cover dirs from offline & cache menu

The settings cover-cache section polled disk usage + progress every 15s for
every server, each call doing a full recursive walk of the per-server cover
directory. On a fully populated cache this caused periodic CPU spikes whenever
that menu was open.

- mod.rs: add a 10s TTL memo around the per-server cover dir walk
  (cached_dir_usage_for_server), shared by cover_cache_stats_server and
  library_cover_progress; invalidate on clear (per-server and clear-all).
- CoverCacheStrategySection: recompute on entry only; rely on the
  cover:library-progress and cover:cache-cleared events for live updates;
  drop the per-cover cover:tier-ready refresh storm; turn the 15s loop into a
  5-minute safety net.

* fix(cover-backfill): keep emitting progress during the whole pass

The producer finishes enumerating the worklist long before the consumer pool
finishes downloading it, so progress was only emitted while feeding the channel
— the "offline & cache" menu and overlay then froze through the entire drain
phase. Replace the per-chunk emit with a 3s progress ticker that runs for the
lifetime of the pass and is aborted once the consumers drain (final accurate
emit still happens at settle).

* docs(changelog): record cover-backfill idle-CPU fix (PR #943)

Add [1.47.0] Fixed + Changed entries and a settingsCredits line for the
cover-backfill idle CPU / offline & cache menu work.
This commit is contained in:
cucadmuh
2026-06-02 04:56:34 +03:00
committed by GitHub
parent 5e977cfd49
commit a63ba3c9cb
12 changed files with 832 additions and 138 deletions
+14 -3
View File
@@ -223,9 +223,15 @@ export async function libraryCoverBackfillPulse(): Promise<CoverBackfillPulseRes
return invoke<CoverBackfillPulseResult>('library_cover_backfill_pulse');
}
/** Start one full-catalog pass on the native runtime (works when the window is inactive). */
export async function libraryCoverBackfillRunFullPass(): Promise<{ started: boolean }> {
return invoke<{ started: boolean }>('library_cover_backfill_run_full_pass');
/**
* Start one full-catalog pass on the native runtime (works when the window is inactive).
* `force` bypasses the idle gate and clears the fetch-failed backoff so previously
* unfetchable (404) covers are retried — used by the manual "Run full pass now".
*/
export async function libraryCoverBackfillRunFullPass(
force = false,
): Promise<{ started: boolean }> {
return invoke<{ started: boolean }>('library_cover_backfill_run_full_pass', { force });
}
export async function libraryCoverBackfillResetCursor(): Promise<void> {
@@ -237,6 +243,11 @@ export async function libraryCoverBackfillSetUiPriority(hold: boolean): Promise<
return invoke('library_cover_backfill_set_ui_priority', { hold });
}
/** Perf-probe only: retune cover backfill threads (download + encode). Returns the clamped value applied. */
export async function libraryCoverBackfillSetParallel(threads: number): Promise<number> {
return invoke<number>('library_cover_backfill_set_parallel', { threads });
}
export async function libraryCoverClearFetchFailures(serverIndexKey: string): Promise<number> {
return invoke<number>('library_cover_clear_fetch_failures', { serverIndexKey });
}
@@ -79,9 +79,14 @@ export default function CoverCacheStrategySection() {
);
}, [servers, refreshRow]);
// Recompute on entry only. Live updates during an active backfill arrive via the
// `cover:library-progress` event (carries done/total/pending/bytes/entryCount), and
// clearing the cache emits `cover:cache-cleared`. A slow 5-minute tick is just a
// safety net for changes made outside this view (e.g. browsing-time caching); it is
// not needed for correctness, so we avoid re-walking the cover dirs in a tight loop.
useEffect(() => {
refreshAll();
const id = window.setInterval(refreshAll, 15_000);
const id = window.setInterval(refreshAll, 300_000);
return () => window.clearInterval(id);
}, [refreshAll]);
@@ -128,9 +133,6 @@ export default function CoverCacheStrategySection() {
refreshAll();
}
}));
unsubs.push(await listen('cover:tier-ready', () => {
refreshAll();
}));
})();
return () => {
for (const u of unsubs) u();
@@ -173,6 +175,9 @@ export default function CoverCacheStrategySection() {
await coverCacheClearServer(clearTarget.indexKey);
clearDiskSrcCacheForServer(clearTarget.indexKey);
await refreshRow(clearTarget.serverId, clearTarget.indexKey);
if (clearTarget.serverId === activeServerId) {
wakeLibraryCoverBackfill();
}
showToast(t('settings.coverCacheStrategyClearSuccess'), 4000, 'success');
} catch {
showToast(t('settings.coverCacheStrategyClearError'), 5000, 'error');
@@ -0,0 +1,95 @@
import { useEffect, useRef, useState } from 'react';
import {
coverGetPipelineQueueStats,
libraryCoverBackfillResetCursor,
libraryCoverBackfillRunFullPass,
libraryCoverBackfillSetParallel,
} from '../../../api/coverCache';
const COVER_THREADS_MIN = 1;
const COVER_THREADS_MAX = 16;
/**
* Perf-probe-only knob for cover backfill concurrency (download + encode pools
* move together). Deliberately not surfaced in app Settings — it is a live
* diagnostics/experiment control. The value is process-local and resets to the
* backend default on app restart.
*/
export default function PerfCoverThreadsControl() {
const [threads, setThreads] = useState<number | null>(null);
const pendingRef = useRef(false);
useEffect(() => {
let cancelled = false;
void coverGetPipelineQueueStats()
.then(stats => {
if (!cancelled) setThreads(stats.libraryBackfillHttpMax || COVER_THREADS_MIN);
})
.catch(() => {
if (!cancelled) setThreads(COVER_THREADS_MIN);
});
return () => {
cancelled = true;
};
}, []);
const apply = (next: number) => {
setThreads(next);
if (pendingRef.current) return;
pendingRef.current = true;
void libraryCoverBackfillSetParallel(next)
.then(applied => setThreads(applied))
.catch(() => {})
.finally(() => {
pendingRef.current = false;
});
};
const value = threads ?? COVER_THREADS_MIN;
const [running, setRunning] = useState(false);
const runPass = () => {
if (running) return;
setRunning(true);
void libraryCoverBackfillResetCursor()
.then(() => libraryCoverBackfillRunFullPass(true))
.catch(() => {})
.finally(() => setRunning(false));
};
return (
<section className="perf-live-poll" aria-label="Cover backfill threads">
<div className="perf-live-poll__title">Cover backfill</div>
<label className="perf-live-poll__row">
<span className="perf-live-poll__label">
Threads
{' '}
<span className="perf-live-poll__value">{value}</span>
</span>
<input
type="range"
min={COVER_THREADS_MIN}
max={COVER_THREADS_MAX}
step={1}
value={value}
disabled={threads === null}
onChange={e => apply(Number(e.target.value))}
/>
</label>
<button
type="button"
className="perf-live-poll__run"
onClick={runPass}
disabled={running}
>
{running ? 'Starting…' : 'Run full pass now'}
</button>
<p className="perf-live-poll__hint">
Download + encode concurrency for background cover warm-up. Higher = faster
backfill but more CPU/network while it runs. Resets to default on restart.
lib /N in the cover overlay only fills when covers are actually missing
clear a servers cover cache first, then run a pass.
</p>
</section>
);
}
@@ -12,6 +12,7 @@ import PerfProbeMetricCard, { PerfProbeMetricSection } from './PerfProbeMetricCa
import PerfOverlayAppearanceControls from './PerfOverlayAppearanceControls';
import PerfOverlayModeControls from './PerfOverlayModeControls';
import PerfLivePollControls from './PerfLivePollControls';
import PerfCoverThreadsControl from './PerfCoverThreadsControl';
function memoryBarPct(rssKb: number, maxKb: number): number {
if (maxKb <= 0) return 0;
@@ -60,6 +61,7 @@ export default function SidebarPerfProbeMonitorTab() {
<PerfOverlayModeControls />
<PerfOverlayAppearanceControls />
<PerfLivePollControls />
<PerfCoverThreadsControl />
<PerfProbeMetricSection title="Pipeline overlays" hint="Rust / UI queues">
<PerfProbeMetricCard
label="FPS"
+1
View File
@@ -144,6 +144,7 @@ const CONTRIBUTOR_ENTRIES = [
'Genres: local index genre browse with Subsonic fallback, aligned counts cache, scroll restore, hold-to-shuffle play (PR #937)',
'Live Search: scoped browse on Artists, Albums, New Releases, Tracks, and Composers — header badge, ghost restore, album title FTS, session stash (PR #938)',
'Performance: idle Rust CPU — backfill coordinator park, probe overlay stability, throttled idle polls, lazy cover prefetch restore (PR #939)',
'Cover backfill: disk-free idle gate, snapshot-diff worklist, live-tunable parallelism, transient-error retries, and memoized offline & cache stats to stop idle CPU spin (PR #943)',
],
},
{
+21
View File
@@ -206,6 +206,27 @@
color: var(--text-primary);
}
.perf-live-poll__run {
margin-top: 10px;
align-self: flex-start;
padding: 4px 10px;
font-size: 11px;
border-radius: 6px;
border: 1px solid var(--border-subtle, color-mix(in srgb, var(--text-muted) 30%, transparent));
background: color-mix(in srgb, var(--text-muted) 12%, transparent);
color: var(--text-primary);
cursor: pointer;
}
.perf-live-poll__run:disabled {
opacity: 0.6;
cursor: default;
}
.perf-live-poll__run:not(:disabled):hover {
background: color-mix(in srgb, var(--text-muted) 20%, transparent);
}
.perf-live-poll__hint {
margin: 6px 0 0;
font-size: 10px;