mirror of
https://github.com/Psychotoxical/psysonic.git
synced 2026-07-21 23:05:46 +00:00
feat(analysis): ship index-key rebuild, strategy controls, and playback/queue pipeline updates (#864)
* feat(analysis): align index settings and per-server strategies Rebuild the local index UX to live under Servers with per-server analytics strategies, and scope analysis queue hints/pruning by playback server so priorities stay isolated across profiles. * feat(analysis): add progress tracking and server analysis deletion functionality Introduce new interfaces for tracking library analysis progress and reporting on server analysis deletions. Implement functions to retrieve analysis progress for a server and to delete all analysis data for a specified server, enhancing the analytics strategy section with real-time progress updates and management capabilities. Update relevant components and localization files to support these features. * feat(server): implement server index key migration and enhance server ID resolution Add functionality to migrate server index keys from legacy IDs to new URL-based keys, improving server ID resolution across the application. Introduce new types and commands for handling server key migrations in both analysis and library contexts. Update relevant functions to utilize the new server ID resolution logic, ensuring consistency and accuracy in server-related operations. * refactor(library): simplify server ID handling in sync progress and idle subscriptions Refactor the library sync progress and idle subscription functions to directly use the payload's server ID without additional mapping. Update related components to resolve server IDs using a new utility function, ensuring consistent server ID resolution across the application. This change enhances code clarity and maintains functionality. * refactor(analytics): rename advanced strategy to aggressive and update descriptions Refactor the AnalyticsStrategySection component to rename the 'advanced' strategy to 'aggressive' for clarity. Update related localization strings to reflect this change, enhancing the user experience by providing clearer descriptions of the analytics strategies. Additionally, remove unused strategy description functions to streamline the code. * fix(audio): update server ID handling in audio progress functions Refactor the audio progress handling to utilize the new `getPlaybackIndexKey` function for server ID resolution. This change ensures that the correct analysis server ID is used when processing audio progress, enhancing the accuracy of playback operations. Additionally, a minor update was made to the analysis cache to include a checkpoint after seeding from bytes. Update the library path in live search to reflect the new database structure. * refactor(analysis): update server ID handling and drop legacy keys Refactor server ID handling across analysis components to utilize scheme-less keys (host + optional path) instead of legacy scheme-based keys. Introduce SQL migrations to drop legacy analysis rows and library entries keyed by scheme URLs. Update relevant functions and tests to ensure consistent server ID resolution and remove references to the legacy '' scope, enhancing clarity and maintainability. * refactor(migration): switch to strategy C dual-db flow Replace destructive server-key migration paths with a blocking inspect/run pipeline that imports into v2 sqlite files, verifies data, then switches active databases with backup safety. Add frontend migration orchestration and post-switch key rewrites while preserving existing user settings behavior. * fix(migration): harden runtime db switch and startup gate Switch database promotion through live runtime store/cache connection swaps so migration cannot leave writers on old sqlite inodes, and tighten startup gating to block initialization until migration completes. Also fix empty-bucket warning detection and set the done flag only after a post-run inspect confirms no pending legacy rows. * feat(migration): enhance migration reporting with skipped server rows tracking Add new fields to migration interfaces and reports to track skipped rows for removed servers. Update relevant components to display warnings and log messages when such rows are encountered during migration processes, improving visibility and user awareness of migration status. * fix(migration): avoid startup blocking modal on no-op runs Keep migration gate completed by default after successful runs and perform done-flag inspections without forcing a blocking phase, so normal app startup no longer flashes migration preparation when no migration is needed. * fix(migration): enforce startup precheck and purge unknown rows Prevent stale done-flag bypass by starting migration state in idle and gating completion on orchestrator precheck, and delete unknown removed-server rows from v2 databases before switch so skipped rows are not carried into the new active DB. * fix(migration): block UI during done-flag precheck Set inspecting phase before the first migration inspection and treat idle as blocking in the migration gate, so startup precheck cannot render the app before migration status is confirmed. * fix(migration): hide precheck modal when no migration is needed Keep startup precheck in a non-blocking idle phase and show the migration modal only after inspect confirms real migration work, removing the recurring half-second migration flash for already-migrated users. * fix(migration): cleanup legacy db files after path migration Always remove legacy analysis and library sqlite files (including wal/shm sidecars) when the new database paths are active, so old-path artifacts from previous builds do not linger after migration. * docs(changelog): add PR #864 release notes and contributor credit Document the full index-key rebuild scope for 1.47.0 and add the corresponding settings credit entry for PR #864. * test(analysis): raise hot-path coverage for analysis cache Add focused unit tests for analysis cache compute/store hot paths and edge branches so coverage regressions are caught before CI. Make AppHandle entrypoints runtime-generic and enable tauri test utilities in dev dependencies to cover no-cache and registered-cache execute paths. * fix(migration): make rebind pass resilient to foreign key ordering Run library and analysis server_id rebind operations inside a foreign-key-disabled transaction and validate with PRAGMA foreign_key_check after commit, so migrations from older databases do not fail on transient FK ordering during bulk updates. * feat(backup): add dual-database backup flow and blocking UX Extend backup/export and restore flows to handle library databases with unified archive detection and asynchronous backend execution. Improve backup UI with a global blocking modal and clearer localized copy so long operations do not look like app hangs. * docs(changelog): add PR #864 backup notes and contributor credit Update 1.47.0 release notes with backup/restore UX and archive-flow entries for PR #864, and add the matching settings credits contribution line for cucadmuh. * docs(changelog): sort 1.47.0 entries from old to new Reorder Added, Changed, and Fixed subsections in the 1.47.0 changelog so entries follow chronological PR order inside each block. * fix(playback): align offline/hot cache lookup with indexKey scope Use a canonical playback cache key based on indexKey with legacy UUID fallback so migrated offline and hot-cache entries are still resolved on normal play, resume, queue-undo, and prefetch paths. Refresh PR #864 changelog/credits text to reflect the full migration and backup scope.
This commit is contained in:
cucadmuh
GitHub
@@ -218,6 +218,8 @@ pub fn run() {
|
||||
app.manage(handle);
|
||||
}
|
||||
|
||||
app.manage(psysonic_analysis::analysis_runtime::PlaybackPriorityHints::default());
|
||||
|
||||
// ── Content-hash sink (analysis → library E2 back-edge) ───────
|
||||
// After a seed the analysis pipeline records the playback-derived
|
||||
// md5_16kb as `track.content_hash` so id-remap can rebind a track
|
||||
@@ -274,6 +276,22 @@ pub fn run() {
|
||||
app.manage(query);
|
||||
}
|
||||
|
||||
// ── Analysis needs-work probe (library → analysis batch scan) ──
|
||||
{
|
||||
use psysonic_core::ports::TrackAnalysisNeedsWorkQuery;
|
||||
let app_for_needs_work = app.handle().clone();
|
||||
let needs_work = TrackAnalysisNeedsWorkQuery::new(
|
||||
move |server_id: &str, track_id: &str| {
|
||||
psysonic_analysis::analysis_runtime::track_analysis_needs_work(
|
||||
&app_for_needs_work,
|
||||
server_id,
|
||||
track_id,
|
||||
)
|
||||
},
|
||||
);
|
||||
app.manage(needs_work);
|
||||
}
|
||||
|
||||
// ── Track enrichment port (analysis → library facts) ───────────
|
||||
{
|
||||
use psysonic_core::track_enrichment::{TrackEnrichmentPlan, TrackEnrichmentPort};
|
||||
@@ -545,6 +563,12 @@ pub fn run() {
|
||||
})
|
||||
.invoke_handler(tauri::generate_handler![
|
||||
greet,
|
||||
backup_export_library_db,
|
||||
backup_import_library_db,
|
||||
backup_export_full,
|
||||
backup_import_full,
|
||||
migration_inspect,
|
||||
migration_run,
|
||||
psysonic_syncfs::sync::batch::calculate_sync_payload,
|
||||
exit_app,
|
||||
cli_publish_player_snapshot,
|
||||
@@ -636,7 +660,13 @@ pub fn run() {
|
||||
psysonic_analysis::commands::analysis_delete_loudness_for_track,
|
||||
psysonic_analysis::commands::analysis_delete_waveform_for_track,
|
||||
psysonic_analysis::commands::analysis_delete_all_waveforms,
|
||||
psysonic_analysis::commands::analysis_delete_all_for_server,
|
||||
psysonic_analysis::commands::analysis_migrate_server_index_keys,
|
||||
psysonic_analysis::commands::analysis_enqueue_seed_from_url,
|
||||
psysonic_analysis::commands::analysis_set_playback_priority_hints,
|
||||
psysonic_analysis::commands::analysis_set_pipeline_parallelism,
|
||||
psysonic_analysis::commands::analysis_get_pipeline_queue_stats,
|
||||
psysonic_analysis::commands::analysis_get_backfill_queue_stats,
|
||||
psysonic_analysis::commands::analysis_prune_pending_to_track_ids,
|
||||
psysonic_library::commands::library_get_status,
|
||||
psysonic_library::commands::library_search,
|
||||
@@ -649,6 +679,7 @@ pub fn run() {
|
||||
psysonic_library::commands::library_get_artifact,
|
||||
psysonic_library::commands::library_get_facts,
|
||||
psysonic_library::commands::library_get_offline_path,
|
||||
psysonic_library::commands::library_analysis_progress,
|
||||
psysonic_library::commands::library_sync_bind_session,
|
||||
psysonic_library::commands::library_sync_clear_session,
|
||||
psysonic_library::commands::library_set_playback_hint,
|
||||
@@ -666,7 +697,9 @@ pub fn run() {
|
||||
psysonic_library::commands::library_get_player_stats_year_bounds,
|
||||
psysonic_library::commands::library_get_player_stats_recent_days,
|
||||
psysonic_library::commands::library_purge_server,
|
||||
psysonic_library::commands::library_migrate_server_index_keys,
|
||||
psysonic_library::commands::library_delete_server_data,
|
||||
psysonic_library::commands::library_analysis_backfill_batch,
|
||||
psysonic_syncfs::cache::offline::download_track_offline,
|
||||
psysonic_syncfs::cache::offline::cancel_offline_downloads,
|
||||
psysonic_syncfs::cache::offline::clear_offline_cancel,
|
||||
|
||||
@@ -0,0 +1,447 @@
|
||||
use std::fs;
|
||||
use std::io::{self, Write};
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use rusqlite::{Connection, OpenFlags};
|
||||
use serde_json::Value;
|
||||
use tauri::{AppHandle, Manager};
|
||||
use zip::write::FileOptions;
|
||||
use zip::{CompressionMethod, ZipArchive, ZipWriter};
|
||||
|
||||
const LIBRARY_ARCHIVE_ENTRY: &str = "library.sqlite";
|
||||
const ANALYSIS_ARCHIVE_ENTRY: &str = "audio-analysis.sqlite";
|
||||
const FULL_ARCHIVE_SETTINGS_ENTRY: &str = "settings.json";
|
||||
const FULL_ARCHIVE_VERSION: u64 = 1;
|
||||
|
||||
#[derive(Debug, serde::Serialize, serde::Deserialize)]
|
||||
struct FullBackupPayload {
|
||||
version: u64,
|
||||
app_version: String,
|
||||
stores: Value,
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub(crate) async fn backup_export_library_db(
|
||||
app: AppHandle,
|
||||
destination_path: String,
|
||||
) -> Result<(), String> {
|
||||
tauri::async_runtime::spawn_blocking(move || {
|
||||
backup_export_library_db_blocking(&app, destination_path)
|
||||
})
|
||||
.await
|
||||
.map_err(|e| e.to_string())?
|
||||
}
|
||||
|
||||
fn backup_export_library_db_blocking(app: &AppHandle, destination_path: String) -> Result<(), String> {
|
||||
let destination = PathBuf::from(destination_path);
|
||||
if let Some(parent) = destination.parent() {
|
||||
fs::create_dir_all(parent).map_err(|e| e.to_string())?;
|
||||
}
|
||||
let source_library = library_db_path(app)?;
|
||||
let source_analysis = analysis_db_path(app)?;
|
||||
if !source_library.exists() {
|
||||
return Err("library database does not exist".to_string());
|
||||
}
|
||||
if !source_analysis.exists() {
|
||||
return Err("analysis database does not exist".to_string());
|
||||
}
|
||||
remove_if_exists(&destination)?;
|
||||
|
||||
let snapshot_library_tmp = source_library.with_file_name("library-export.sqlite");
|
||||
let snapshot_analysis_tmp = source_analysis.with_file_name("audio-analysis-export.sqlite");
|
||||
remove_db_with_sidecars(&snapshot_library_tmp)?;
|
||||
remove_db_with_sidecars(&snapshot_analysis_tmp)?;
|
||||
vacuum_copy(&source_library, &snapshot_library_tmp)?;
|
||||
vacuum_copy(&source_analysis, &snapshot_analysis_tmp)?;
|
||||
let result = write_databases_archive(
|
||||
&snapshot_library_tmp,
|
||||
&snapshot_analysis_tmp,
|
||||
&destination,
|
||||
);
|
||||
remove_db_with_sidecars(&snapshot_library_tmp).ok();
|
||||
remove_db_with_sidecars(&snapshot_analysis_tmp).ok();
|
||||
result
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub(crate) async fn backup_import_library_db(app: AppHandle, source_path: String) -> Result<(), String> {
|
||||
tauri::async_runtime::spawn_blocking(move || {
|
||||
backup_import_library_db_blocking(&app, source_path)
|
||||
})
|
||||
.await
|
||||
.map_err(|e| e.to_string())?
|
||||
}
|
||||
|
||||
fn backup_import_library_db_blocking(app: &AppHandle, source_path: String) -> Result<(), String> {
|
||||
let source = PathBuf::from(source_path);
|
||||
if !source.exists() {
|
||||
return Err("backup file not found".to_string());
|
||||
}
|
||||
|
||||
let active_library = library_db_path(app)?;
|
||||
let active_analysis = analysis_db_path(app)?;
|
||||
let import_library_tmp = active_library.with_file_name("library-import.sqlite");
|
||||
let import_analysis_tmp = active_analysis.with_file_name("audio-analysis-import.sqlite");
|
||||
remove_db_with_sidecars(&import_library_tmp)?;
|
||||
remove_db_with_sidecars(&import_analysis_tmp)?;
|
||||
extract_databases_archive(&source, &import_library_tmp, &import_analysis_tmp)?;
|
||||
validate_sqlite_file(&import_library_tmp)?;
|
||||
validate_sqlite_file(&import_analysis_tmp)?;
|
||||
import_databases_from_sqlite(app, &import_library_tmp, &import_analysis_tmp)
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub(crate) async fn backup_export_full(
|
||||
app: AppHandle,
|
||||
destination_path: String,
|
||||
stores: Value,
|
||||
app_version: String,
|
||||
) -> Result<(), String> {
|
||||
tauri::async_runtime::spawn_blocking(move || {
|
||||
backup_export_full_blocking(&app, destination_path, stores, app_version)
|
||||
})
|
||||
.await
|
||||
.map_err(|e| e.to_string())?
|
||||
}
|
||||
|
||||
fn backup_export_full_blocking(
|
||||
app: &AppHandle,
|
||||
destination_path: String,
|
||||
stores: Value,
|
||||
app_version: String,
|
||||
) -> Result<(), String> {
|
||||
if !stores.is_object() {
|
||||
return Err("stores payload must be an object".to_string());
|
||||
}
|
||||
let destination = PathBuf::from(destination_path);
|
||||
if let Some(parent) = destination.parent() {
|
||||
fs::create_dir_all(parent).map_err(|e| e.to_string())?;
|
||||
}
|
||||
remove_if_exists(&destination)?;
|
||||
|
||||
let source_library = library_db_path(app)?;
|
||||
let source_analysis = analysis_db_path(app)?;
|
||||
if !source_library.exists() {
|
||||
return Err("library database does not exist".to_string());
|
||||
}
|
||||
if !source_analysis.exists() {
|
||||
return Err("analysis database does not exist".to_string());
|
||||
}
|
||||
let snapshot_library_tmp = source_library.with_file_name("library-export.sqlite");
|
||||
let snapshot_analysis_tmp = source_analysis.with_file_name("audio-analysis-export.sqlite");
|
||||
remove_db_with_sidecars(&snapshot_library_tmp)?;
|
||||
remove_db_with_sidecars(&snapshot_analysis_tmp)?;
|
||||
vacuum_copy(&source_library, &snapshot_library_tmp)?;
|
||||
vacuum_copy(&source_analysis, &snapshot_analysis_tmp)?;
|
||||
|
||||
let payload = FullBackupPayload {
|
||||
version: FULL_ARCHIVE_VERSION,
|
||||
app_version,
|
||||
stores,
|
||||
};
|
||||
let result = write_full_archive(
|
||||
&snapshot_library_tmp,
|
||||
&snapshot_analysis_tmp,
|
||||
&destination,
|
||||
&payload,
|
||||
);
|
||||
remove_db_with_sidecars(&snapshot_library_tmp).ok();
|
||||
remove_db_with_sidecars(&snapshot_analysis_tmp).ok();
|
||||
result
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub(crate) async fn backup_import_full(app: AppHandle, source_path: String) -> Result<Value, String> {
|
||||
tauri::async_runtime::spawn_blocking(move || backup_import_full_blocking(&app, source_path))
|
||||
.await
|
||||
.map_err(|e| e.to_string())?
|
||||
}
|
||||
|
||||
fn backup_import_full_blocking(app: &AppHandle, source_path: String) -> Result<Value, String> {
|
||||
let source = PathBuf::from(source_path);
|
||||
if !source.exists() {
|
||||
return Err("backup file not found".to_string());
|
||||
}
|
||||
|
||||
let active_library = library_db_path(app)?;
|
||||
let active_analysis = analysis_db_path(app)?;
|
||||
let import_library_tmp = active_library.with_file_name("library-import.sqlite");
|
||||
let import_analysis_tmp = active_analysis.with_file_name("audio-analysis-import.sqlite");
|
||||
remove_db_with_sidecars(&import_library_tmp)?;
|
||||
remove_db_with_sidecars(&import_analysis_tmp)?;
|
||||
let payload = extract_full_archive(&source, &import_library_tmp, &import_analysis_tmp)?;
|
||||
validate_sqlite_file(&import_library_tmp)?;
|
||||
validate_sqlite_file(&import_analysis_tmp)?;
|
||||
let stores = payload.stores;
|
||||
if !stores.is_object() {
|
||||
remove_db_with_sidecars(&import_library_tmp).ok();
|
||||
remove_db_with_sidecars(&import_analysis_tmp).ok();
|
||||
return Err("backup payload stores must be an object".to_string());
|
||||
}
|
||||
|
||||
import_databases_from_sqlite(app, &import_library_tmp, &import_analysis_tmp)?;
|
||||
Ok(stores)
|
||||
}
|
||||
|
||||
fn library_db_path(app: &AppHandle) -> Result<PathBuf, String> {
|
||||
let base = app.path().app_data_dir().map_err(|e| e.to_string())?;
|
||||
let dir = base.join("databases").join("library");
|
||||
fs::create_dir_all(&dir).map_err(|e| e.to_string())?;
|
||||
Ok(dir.join("library.sqlite"))
|
||||
}
|
||||
|
||||
fn analysis_db_path(app: &AppHandle) -> Result<PathBuf, String> {
|
||||
let base = app.path().app_data_dir().map_err(|e| e.to_string())?;
|
||||
let dir = base.join("databases").join("analysis");
|
||||
fs::create_dir_all(&dir).map_err(|e| e.to_string())?;
|
||||
Ok(dir.join("audio-analysis.sqlite"))
|
||||
}
|
||||
|
||||
fn vacuum_copy(source: &Path, destination: &Path) -> Result<(), String> {
|
||||
let conn = Connection::open_with_flags(source, OpenFlags::SQLITE_OPEN_READ_WRITE)
|
||||
.map_err(|e| e.to_string())?;
|
||||
let escaped = destination.to_string_lossy().replace('\'', "''");
|
||||
let sql = format!("VACUUM INTO '{escaped}';");
|
||||
conn.execute_batch(&sql).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn write_databases_archive(
|
||||
library_snapshot: &Path,
|
||||
analysis_snapshot: &Path,
|
||||
destination_archive: &Path,
|
||||
) -> Result<(), String> {
|
||||
let file = fs::File::create(destination_archive).map_err(|e| e.to_string())?;
|
||||
let mut zip = ZipWriter::new(file);
|
||||
let options = FileOptions::default().compression_method(CompressionMethod::Deflated);
|
||||
zip.start_file(LIBRARY_ARCHIVE_ENTRY, options)
|
||||
.map_err(|e| e.to_string())?;
|
||||
let mut src = fs::File::open(library_snapshot).map_err(|e| e.to_string())?;
|
||||
io::copy(&mut src, &mut zip).map_err(|e| e.to_string())?;
|
||||
zip.start_file(ANALYSIS_ARCHIVE_ENTRY, options)
|
||||
.map_err(|e| e.to_string())?;
|
||||
let mut analysis_src = fs::File::open(analysis_snapshot).map_err(|e| e.to_string())?;
|
||||
io::copy(&mut analysis_src, &mut zip).map_err(|e| e.to_string())?;
|
||||
zip.finish().map_err(|e| e.to_string())?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn write_full_archive(
|
||||
library_snapshot: &Path,
|
||||
analysis_snapshot: &Path,
|
||||
destination_archive: &Path,
|
||||
payload: &FullBackupPayload,
|
||||
) -> Result<(), String> {
|
||||
let file = fs::File::create(destination_archive).map_err(|e| e.to_string())?;
|
||||
let mut zip = ZipWriter::new(file);
|
||||
let options = FileOptions::default().compression_method(CompressionMethod::Deflated);
|
||||
|
||||
zip.start_file(FULL_ARCHIVE_SETTINGS_ENTRY, options)
|
||||
.map_err(|e| e.to_string())?;
|
||||
let settings = serde_json::to_vec_pretty(payload).map_err(|e| e.to_string())?;
|
||||
zip.write_all(&settings).map_err(|e| e.to_string())?;
|
||||
|
||||
zip.start_file(LIBRARY_ARCHIVE_ENTRY, options)
|
||||
.map_err(|e| e.to_string())?;
|
||||
let mut src = fs::File::open(library_snapshot).map_err(|e| e.to_string())?;
|
||||
io::copy(&mut src, &mut zip).map_err(|e| e.to_string())?;
|
||||
|
||||
zip.start_file(ANALYSIS_ARCHIVE_ENTRY, options)
|
||||
.map_err(|e| e.to_string())?;
|
||||
let mut analysis_src = fs::File::open(analysis_snapshot).map_err(|e| e.to_string())?;
|
||||
io::copy(&mut analysis_src, &mut zip).map_err(|e| e.to_string())?;
|
||||
zip.finish().map_err(|e| e.to_string())?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn extract_databases_archive(
|
||||
source_archive: &Path,
|
||||
library_destination_sqlite: &Path,
|
||||
analysis_destination_sqlite: &Path,
|
||||
) -> Result<(), String> {
|
||||
let file = fs::File::open(source_archive).map_err(|e| e.to_string())?;
|
||||
let mut archive = ZipArchive::new(file).map_err(|e| e.to_string())?;
|
||||
let library_target_index = archive
|
||||
.file_names()
|
||||
.enumerate()
|
||||
.find_map(|(i, name)| (name == LIBRARY_ARCHIVE_ENTRY).then_some(i))
|
||||
.ok_or_else(|| "archive does not contain library.sqlite".to_string())?;
|
||||
let analysis_target_index = archive
|
||||
.file_names()
|
||||
.enumerate()
|
||||
.find_map(|(i, name)| (name == ANALYSIS_ARCHIVE_ENTRY).then_some(i))
|
||||
.ok_or_else(|| "archive does not contain audio-analysis.sqlite".to_string())?;
|
||||
|
||||
if let Some(parent) = library_destination_sqlite.parent() {
|
||||
fs::create_dir_all(parent).map_err(|e| e.to_string())?;
|
||||
}
|
||||
{
|
||||
let mut library_entry = archive.by_index(library_target_index).map_err(|e| e.to_string())?;
|
||||
let mut out = fs::File::create(library_destination_sqlite).map_err(|e| e.to_string())?;
|
||||
io::copy(&mut library_entry, &mut out).map_err(|e| e.to_string())?;
|
||||
}
|
||||
|
||||
if let Some(parent) = analysis_destination_sqlite.parent() {
|
||||
fs::create_dir_all(parent).map_err(|e| e.to_string())?;
|
||||
}
|
||||
{
|
||||
let mut analysis_entry = archive.by_index(analysis_target_index).map_err(|e| e.to_string())?;
|
||||
let mut analysis_out = fs::File::create(analysis_destination_sqlite).map_err(|e| e.to_string())?;
|
||||
io::copy(&mut analysis_entry, &mut analysis_out).map_err(|e| e.to_string())?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn extract_full_archive(
|
||||
source_archive: &Path,
|
||||
library_destination_sqlite: &Path,
|
||||
analysis_destination_sqlite: &Path,
|
||||
) -> Result<FullBackupPayload, String> {
|
||||
let file = fs::File::open(source_archive).map_err(|e| e.to_string())?;
|
||||
let mut archive = ZipArchive::new(file).map_err(|e| e.to_string())?;
|
||||
|
||||
let payload = {
|
||||
let mut entry = archive
|
||||
.by_name(FULL_ARCHIVE_SETTINGS_ENTRY)
|
||||
.map_err(|_| "archive does not contain settings.json".to_string())?;
|
||||
let mut buf = Vec::new();
|
||||
io::copy(&mut entry, &mut buf).map_err(|e| e.to_string())?;
|
||||
serde_json::from_slice::<FullBackupPayload>(&buf).map_err(|e| e.to_string())?
|
||||
};
|
||||
if payload.version != FULL_ARCHIVE_VERSION {
|
||||
return Err("unsupported full backup version".to_string());
|
||||
}
|
||||
|
||||
extract_databases_archive(
|
||||
source_archive,
|
||||
library_destination_sqlite,
|
||||
analysis_destination_sqlite,
|
||||
)?;
|
||||
Ok(payload)
|
||||
}
|
||||
|
||||
fn validate_sqlite_file(path: &Path) -> Result<(), String> {
|
||||
let conn = Connection::open_with_flags(path, OpenFlags::SQLITE_OPEN_READ_ONLY)
|
||||
.map_err(|e| e.to_string())?;
|
||||
let integrity: String = conn
|
||||
.query_row("PRAGMA integrity_check;", [], |row| row.get(0))
|
||||
.map_err(|e| e.to_string())?;
|
||||
if integrity != "ok" {
|
||||
return Err("backup file integrity check failed".to_string());
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn library_health_check(active_path: &Path) -> Result<(), String> {
|
||||
let conn = Connection::open_with_flags(active_path, OpenFlags::SQLITE_OPEN_READ_ONLY)
|
||||
.map_err(|e| e.to_string())?;
|
||||
conn.query_row("SELECT COUNT(*) FROM track", [], |_row| Ok(()))
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn analysis_health_check(active_path: &Path) -> Result<(), String> {
|
||||
let conn = Connection::open_with_flags(active_path, OpenFlags::SQLITE_OPEN_READ_ONLY)
|
||||
.map_err(|e| e.to_string())?;
|
||||
conn.query_row("SELECT COUNT(*) FROM analysis_track", [], |_row| Ok(()))
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn import_databases_from_sqlite(
|
||||
app: &AppHandle,
|
||||
import_library_tmp: &Path,
|
||||
import_analysis_tmp: &Path,
|
||||
) -> Result<(), String> {
|
||||
let active_path = library_db_path(app)?;
|
||||
let analysis_active_path = analysis_db_path(app)?;
|
||||
let Some(runtime) = app.try_state::<psysonic_library::LibraryRuntime>() else {
|
||||
remove_db_with_sidecars(import_library_tmp).ok();
|
||||
remove_db_with_sidecars(import_analysis_tmp).ok();
|
||||
return Err("library runtime unavailable".to_string());
|
||||
};
|
||||
let Some(cache) = app.try_state::<psysonic_analysis::analysis_cache::AnalysisCache>() else {
|
||||
remove_db_with_sidecars(import_library_tmp).ok();
|
||||
remove_db_with_sidecars(import_analysis_tmp).ok();
|
||||
return Err("analysis runtime unavailable".to_string());
|
||||
};
|
||||
|
||||
let library_backup = runtime
|
||||
.store
|
||||
.swap_database_file(&active_path, import_library_tmp)?
|
||||
.ok_or_else(|| "import switch failed".to_string())?;
|
||||
let analysis_backup = match cache.swap_database_file(&analysis_active_path, import_analysis_tmp) {
|
||||
Ok(Some(backup)) => backup,
|
||||
Ok(None) => {
|
||||
let _ = runtime.store.restore_database_backup(&library_backup, &active_path);
|
||||
let _ = remove_db_with_sidecars(&library_backup);
|
||||
let _ = remove_db_with_sidecars(import_library_tmp);
|
||||
let _ = remove_db_with_sidecars(import_analysis_tmp);
|
||||
return Err("analysis import switch failed".to_string());
|
||||
}
|
||||
Err(err) => {
|
||||
let _ = runtime.store.restore_database_backup(&library_backup, &active_path);
|
||||
let _ = remove_db_with_sidecars(&library_backup);
|
||||
let _ = remove_db_with_sidecars(import_library_tmp);
|
||||
let _ = remove_db_with_sidecars(import_analysis_tmp);
|
||||
return Err(err);
|
||||
}
|
||||
};
|
||||
|
||||
if let Err(err) = library_health_check(&active_path)
|
||||
.and_then(|_| analysis_health_check(&analysis_active_path))
|
||||
{
|
||||
let _ = runtime.store.restore_database_backup(&library_backup, &active_path);
|
||||
let _ = cache.restore_database_backup(&analysis_backup, &analysis_active_path);
|
||||
let _ = remove_db_with_sidecars(&library_backup);
|
||||
let _ = remove_db_with_sidecars(&analysis_backup);
|
||||
let _ = remove_db_with_sidecars(import_library_tmp);
|
||||
let _ = remove_db_with_sidecars(import_analysis_tmp);
|
||||
return Err(err);
|
||||
}
|
||||
|
||||
let library_bak_path = active_path.with_file_name("library.sqlite.import.bak");
|
||||
remove_db_with_sidecars(&library_bak_path).ok();
|
||||
if library_backup.exists() {
|
||||
fs::rename(&library_backup, &library_bak_path).map_err(|e| e.to_string())?;
|
||||
move_sidecar(&library_backup, &library_bak_path, "-wal")?;
|
||||
move_sidecar(&library_backup, &library_bak_path, "-shm")?;
|
||||
}
|
||||
|
||||
let analysis_bak_path = analysis_active_path.with_file_name("audio-analysis.sqlite.import.bak");
|
||||
remove_db_with_sidecars(&analysis_bak_path).ok();
|
||||
if analysis_backup.exists() {
|
||||
fs::rename(&analysis_backup, &analysis_bak_path).map_err(|e| e.to_string())?;
|
||||
move_sidecar(&analysis_backup, &analysis_bak_path, "-wal")?;
|
||||
move_sidecar(&analysis_backup, &analysis_bak_path, "-shm")?;
|
||||
}
|
||||
|
||||
remove_db_with_sidecars(import_library_tmp).ok();
|
||||
remove_db_with_sidecars(import_analysis_tmp).ok();
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn remove_db_with_sidecars(path: &Path) -> Result<(), String> {
|
||||
remove_if_exists(path)?;
|
||||
for suffix in ["-wal", "-shm"] {
|
||||
let sidecar = PathBuf::from(format!("{}{}", path.to_string_lossy(), suffix));
|
||||
remove_if_exists(&sidecar)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn move_sidecar(from_base: &Path, to_base: &Path, suffix: &str) -> Result<(), String> {
|
||||
let from = PathBuf::from(format!("{}{}", from_base.display(), suffix));
|
||||
if !from.exists() {
|
||||
return Ok(());
|
||||
}
|
||||
let to = PathBuf::from(format!("{}{}", to_base.display(), suffix));
|
||||
if let Some(parent) = to.parent() {
|
||||
fs::create_dir_all(parent).map_err(|e| e.to_string())?;
|
||||
}
|
||||
fs::rename(from, to).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn remove_if_exists(path: &Path) -> Result<(), String> {
|
||||
if path.exists() {
|
||||
fs::remove_file(path).map_err(|e| e.to_string())?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
@@ -2,6 +2,7 @@ use tauri::Manager;
|
||||
|
||||
use crate::lib_commands::sync::stop_audio_engine;
|
||||
use crate::runtime_subsonic_wire_user_agent;
|
||||
use crate::analysis_cache;
|
||||
|
||||
#[tauri::command]
|
||||
pub(crate) fn greet(name: &str) -> String {
|
||||
@@ -10,6 +11,9 @@ pub(crate) fn greet(name: &str) -> String {
|
||||
|
||||
#[tauri::command]
|
||||
pub(crate) fn exit_app(app_handle: tauri::AppHandle) {
|
||||
if let Some(cache) = app_handle.try_state::<analysis_cache::AnalysisCache>() {
|
||||
let _ = cache.checkpoint_wal("exit");
|
||||
}
|
||||
stop_audio_engine(&app_handle);
|
||||
app_handle.exit(0);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,756 @@
|
||||
use std::collections::HashMap;
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::{Mutex, OnceLock};
|
||||
|
||||
use rusqlite::{params_from_iter, Connection, OpenFlags};
|
||||
use tauri::{AppHandle, Emitter, Manager};
|
||||
|
||||
const LIBRARY_TABLES: &[&str] = &[
|
||||
"track_extension",
|
||||
"track_fact",
|
||||
"track_artifact",
|
||||
"track_canonical_link",
|
||||
"track_id_history",
|
||||
"play_session",
|
||||
"track_offline",
|
||||
"track",
|
||||
"album",
|
||||
"artist",
|
||||
"sync_state",
|
||||
];
|
||||
|
||||
const ANALYSIS_TABLES: &[&str] = &["analysis_track", "waveform_cache", "loudness_cache"];
|
||||
|
||||
fn migration_lock() -> &'static Mutex<()> {
|
||||
static LOCK: OnceLock<Mutex<()>> = OnceLock::new();
|
||||
LOCK.get_or_init(|| Mutex::new(()))
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct ServerIndexMapping {
|
||||
pub legacy_id: String,
|
||||
pub index_key: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, serde::Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct MigrationScopeInspect {
|
||||
pub total_legacy_rows: u64,
|
||||
pub skipped_unknown_server_rows: u64,
|
||||
pub tables: HashMap<String, u64>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, serde::Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct MigrationInspectReport {
|
||||
pub needs_migration: bool,
|
||||
pub has_skipped_unknown_server_rows: bool,
|
||||
pub can_run: bool,
|
||||
pub warnings: Vec<String>,
|
||||
pub unmapped_empty_bucket: bool,
|
||||
pub library: MigrationScopeInspect,
|
||||
pub analysis: MigrationScopeInspect,
|
||||
pub mappings: Vec<ServerIndexMapping>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, serde::Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct MigrationProgressEvent {
|
||||
pub stage: String,
|
||||
pub table: String,
|
||||
pub done: u64,
|
||||
pub total: u64,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, serde::Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct MigrationRunScope {
|
||||
pub imported_rows: u64,
|
||||
pub source_rows: u64,
|
||||
pub skipped_unknown_server_rows: u64,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, serde::Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct MigrationRunResult {
|
||||
pub library: MigrationRunScope,
|
||||
pub analysis: MigrationRunScope,
|
||||
pub has_skipped_unknown_server_rows: bool,
|
||||
pub switched: bool,
|
||||
pub backup_removed: bool,
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub fn migration_inspect(
|
||||
app: AppHandle,
|
||||
mappings: Vec<ServerIndexMapping>,
|
||||
) -> Result<MigrationInspectReport, String> {
|
||||
inspect_internal(&app, mappings)
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub fn migration_run(
|
||||
app: AppHandle,
|
||||
mappings: Vec<ServerIndexMapping>,
|
||||
) -> Result<MigrationRunResult, String> {
|
||||
let _guard = migration_lock()
|
||||
.lock()
|
||||
.map_err(|_| "migration lock poisoned".to_string())?;
|
||||
run_internal(&app, mappings)
|
||||
}
|
||||
|
||||
fn inspect_internal(
|
||||
app: &AppHandle,
|
||||
mappings: Vec<ServerIndexMapping>,
|
||||
) -> Result<MigrationInspectReport, String> {
|
||||
let normalized = normalize_mappings(mappings);
|
||||
let legacy_ids: Vec<String> = normalized.iter().map(|m| m.legacy_id.clone()).collect();
|
||||
let index_keys: Vec<String> = normalized.iter().map(|m| m.index_key.clone()).collect();
|
||||
let paths = migration_paths(app)?;
|
||||
|
||||
let (library_tables, library_total, library_skipped_unknown_rows) = inspect_tables(
|
||||
&paths.library_active,
|
||||
LIBRARY_TABLES,
|
||||
&legacy_ids,
|
||||
&index_keys,
|
||||
)?;
|
||||
let (analysis_tables, mut analysis_total, analysis_skipped_unknown_rows) = inspect_tables(
|
||||
&paths.analysis_active,
|
||||
ANALYSIS_TABLES,
|
||||
&legacy_ids,
|
||||
&index_keys,
|
||||
)?;
|
||||
let mut analysis_tables = analysis_tables;
|
||||
let mut warnings = Vec::new();
|
||||
let mut unmapped_empty_bucket = false;
|
||||
let mut has_empty_bucket_rows = false;
|
||||
if paths.analysis_active.exists() {
|
||||
let conn = open_readonly(&paths.analysis_active)?;
|
||||
for table in ANALYSIS_TABLES {
|
||||
let empty_count = count_rows_eq(&conn, table, "")?;
|
||||
if empty_count > 0 {
|
||||
has_empty_bucket_rows = true;
|
||||
if normalized.len() == 1 {
|
||||
let entry = analysis_tables.entry((*table).to_string()).or_insert(0);
|
||||
*entry = entry.saturating_add(empty_count as u64);
|
||||
analysis_total = analysis_total.saturating_add(empty_count as u64);
|
||||
}
|
||||
}
|
||||
}
|
||||
if normalized.len() > 1 && has_empty_bucket_rows {
|
||||
unmapped_empty_bucket = true;
|
||||
warnings.push("analysis empty server bucket kept for multi-server install".to_string());
|
||||
}
|
||||
}
|
||||
|
||||
let needs_migration = library_total > 0 || analysis_total > 0;
|
||||
let can_run = !normalized.is_empty();
|
||||
if needs_migration && !can_run {
|
||||
warnings.push("no server mappings available".to_string());
|
||||
}
|
||||
let has_skipped_unknown_server_rows =
|
||||
library_skipped_unknown_rows > 0 || analysis_skipped_unknown_rows > 0;
|
||||
if has_skipped_unknown_server_rows {
|
||||
warnings.push("rows for removed servers were skipped".to_string());
|
||||
}
|
||||
|
||||
Ok(MigrationInspectReport {
|
||||
needs_migration,
|
||||
has_skipped_unknown_server_rows,
|
||||
can_run,
|
||||
warnings,
|
||||
unmapped_empty_bucket,
|
||||
library: MigrationScopeInspect {
|
||||
total_legacy_rows: library_total,
|
||||
skipped_unknown_server_rows: library_skipped_unknown_rows,
|
||||
tables: library_tables,
|
||||
},
|
||||
analysis: MigrationScopeInspect {
|
||||
total_legacy_rows: analysis_total,
|
||||
skipped_unknown_server_rows: analysis_skipped_unknown_rows,
|
||||
tables: analysis_tables,
|
||||
},
|
||||
mappings: normalized,
|
||||
})
|
||||
}
|
||||
|
||||
fn run_internal(app: &AppHandle, mappings: Vec<ServerIndexMapping>) -> Result<MigrationRunResult, String> {
|
||||
let inspect = inspect_internal(app, mappings)?;
|
||||
if !inspect.needs_migration {
|
||||
return Ok(MigrationRunResult {
|
||||
library: MigrationRunScope {
|
||||
imported_rows: 0,
|
||||
source_rows: 0,
|
||||
skipped_unknown_server_rows: inspect.library.skipped_unknown_server_rows,
|
||||
},
|
||||
analysis: MigrationRunScope {
|
||||
imported_rows: 0,
|
||||
source_rows: 0,
|
||||
skipped_unknown_server_rows: inspect.analysis.skipped_unknown_server_rows,
|
||||
},
|
||||
has_skipped_unknown_server_rows: inspect.has_skipped_unknown_server_rows,
|
||||
switched: false,
|
||||
backup_removed: false,
|
||||
});
|
||||
}
|
||||
if !inspect.can_run {
|
||||
return Err("migration requires at least one server mapping".to_string());
|
||||
}
|
||||
|
||||
let paths = migration_paths(app)?;
|
||||
let mappings = inspect.mappings;
|
||||
let single_mapping = if mappings.len() == 1 {
|
||||
Some(mappings[0].index_key.clone())
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
emit_progress(
|
||||
app,
|
||||
"library",
|
||||
"prepare",
|
||||
0,
|
||||
LIBRARY_TABLES.len() as u64,
|
||||
)?;
|
||||
let (library_source_rows, library_imported_rows, library_skipped_unknown_rows) =
|
||||
run_library_import(app, &paths, &mappings)?;
|
||||
let (analysis_source_rows, analysis_imported_rows, analysis_skipped_unknown_rows) =
|
||||
run_analysis_import(app, &paths, &mappings, single_mapping.as_deref())?;
|
||||
|
||||
let mut backup_removed = false;
|
||||
let mut library_backup: Option<PathBuf> = None;
|
||||
let mut analysis_backup: Option<PathBuf> = None;
|
||||
|
||||
if paths.library_v2.exists() {
|
||||
if let Some(runtime) = app.try_state::<psysonic_library::LibraryRuntime>() {
|
||||
library_backup = runtime
|
||||
.store
|
||||
.swap_database_file(&paths.library_active, &paths.library_v2)?;
|
||||
} else {
|
||||
library_backup = Some(switch_file(&paths.library_active, &paths.library_v2)?);
|
||||
}
|
||||
}
|
||||
if paths.analysis_v2.exists() {
|
||||
if let Some(cache) = app.try_state::<psysonic_analysis::analysis_cache::AnalysisCache>() {
|
||||
analysis_backup = cache.swap_database_file(&paths.analysis_active, &paths.analysis_v2)?;
|
||||
} else {
|
||||
analysis_backup = Some(switch_file(&paths.analysis_active, &paths.analysis_v2)?);
|
||||
}
|
||||
}
|
||||
let switched = library_backup.is_some() || analysis_backup.is_some();
|
||||
|
||||
if let Err(err) = health_check(&paths.library_active, &paths.analysis_active) {
|
||||
if let Some(ref backup) = library_backup {
|
||||
if let Some(runtime) = app.try_state::<psysonic_library::LibraryRuntime>() {
|
||||
let _ = runtime
|
||||
.store
|
||||
.restore_database_backup(backup, &paths.library_active);
|
||||
} else {
|
||||
let _ = restore_backup(backup, &paths.library_active);
|
||||
}
|
||||
}
|
||||
if let Some(ref backup) = analysis_backup {
|
||||
if let Some(cache) = app.try_state::<psysonic_analysis::analysis_cache::AnalysisCache>() {
|
||||
let _ = cache.restore_database_backup(backup, &paths.analysis_active);
|
||||
} else {
|
||||
let _ = restore_backup(backup, &paths.analysis_active);
|
||||
}
|
||||
}
|
||||
return Err(err);
|
||||
}
|
||||
|
||||
if let Some(backup) = library_backup {
|
||||
remove_db_with_sidecars(&backup)?;
|
||||
backup_removed = true;
|
||||
}
|
||||
if let Some(backup) = analysis_backup {
|
||||
remove_db_with_sidecars(&backup)?;
|
||||
backup_removed = true;
|
||||
}
|
||||
|
||||
Ok(MigrationRunResult {
|
||||
library: MigrationRunScope {
|
||||
imported_rows: library_imported_rows,
|
||||
source_rows: library_source_rows,
|
||||
skipped_unknown_server_rows: library_skipped_unknown_rows,
|
||||
},
|
||||
analysis: MigrationRunScope {
|
||||
imported_rows: analysis_imported_rows,
|
||||
source_rows: analysis_source_rows,
|
||||
skipped_unknown_server_rows: analysis_skipped_unknown_rows,
|
||||
},
|
||||
has_skipped_unknown_server_rows: library_skipped_unknown_rows > 0 || analysis_skipped_unknown_rows > 0,
|
||||
switched,
|
||||
backup_removed,
|
||||
})
|
||||
}
|
||||
|
||||
fn run_library_import(
|
||||
app: &AppHandle,
|
||||
paths: &MigrationPaths,
|
||||
mappings: &[ServerIndexMapping],
|
||||
) -> Result<(u64, u64, u64), String> {
|
||||
if !paths.library_active.exists() {
|
||||
return Ok((0, 0, 0));
|
||||
}
|
||||
remove_db_with_sidecars(&paths.library_v2).ok();
|
||||
vacuum_copy(&paths.library_active, &paths.library_v2)?;
|
||||
|
||||
let source = open_readonly(&paths.library_active)?;
|
||||
let dest = Connection::open(&paths.library_v2).map_err(|e| e.to_string())?;
|
||||
let legacy_ids: Vec<String> = mappings.iter().map(|m| m.legacy_id.clone()).collect();
|
||||
let index_keys: Vec<String> = mappings.iter().map(|m| m.index_key.clone()).collect();
|
||||
let total = LIBRARY_TABLES.len() as u64;
|
||||
let mut done = 0_u64;
|
||||
with_foreign_keys_disabled(&dest, || {
|
||||
for table in LIBRARY_TABLES {
|
||||
purge_unknown_rows(&dest, table, &legacy_ids, &index_keys)?;
|
||||
for mapping in mappings {
|
||||
dest.execute(
|
||||
&format!("UPDATE OR REPLACE {table} SET server_id = ?2 WHERE server_id = ?1"),
|
||||
[&mapping.legacy_id, &mapping.index_key],
|
||||
)
|
||||
.map_err(|e| e.to_string())?;
|
||||
}
|
||||
done = done.saturating_add(1);
|
||||
emit_progress(app, "library", table, done, total)?;
|
||||
}
|
||||
Ok(())
|
||||
})?;
|
||||
let source_rows = sum_table_rows(&source, LIBRARY_TABLES)?;
|
||||
let imported_rows = sum_table_rows(&dest, LIBRARY_TABLES)?;
|
||||
let skipped_unknown_server_rows = sum_unknown_rows(&source, LIBRARY_TABLES, &legacy_ids, &index_keys)?;
|
||||
Ok((source_rows, imported_rows, skipped_unknown_server_rows))
|
||||
}
|
||||
|
||||
fn run_analysis_import(
|
||||
app: &AppHandle,
|
||||
paths: &MigrationPaths,
|
||||
mappings: &[ServerIndexMapping],
|
||||
single_mapping: Option<&str>,
|
||||
) -> Result<(u64, u64, u64), String> {
|
||||
if !paths.analysis_active.exists() {
|
||||
return Ok((0, 0, 0));
|
||||
}
|
||||
remove_db_with_sidecars(&paths.analysis_v2).ok();
|
||||
vacuum_copy(&paths.analysis_active, &paths.analysis_v2)?;
|
||||
|
||||
let source = open_readonly(&paths.analysis_active)?;
|
||||
let dest = Connection::open(&paths.analysis_v2).map_err(|e| e.to_string())?;
|
||||
let legacy_ids: Vec<String> = mappings.iter().map(|m| m.legacy_id.clone()).collect();
|
||||
let index_keys: Vec<String> = mappings.iter().map(|m| m.index_key.clone()).collect();
|
||||
let total = ANALYSIS_TABLES.len() as u64;
|
||||
let mut done = 0_u64;
|
||||
with_foreign_keys_disabled(&dest, || {
|
||||
for table in ANALYSIS_TABLES {
|
||||
purge_unknown_rows(&dest, table, &legacy_ids, &index_keys)?;
|
||||
for mapping in mappings {
|
||||
dest.execute(
|
||||
&format!("UPDATE OR REPLACE {table} SET server_id = ?2 WHERE server_id = ?1"),
|
||||
[&mapping.legacy_id, &mapping.index_key],
|
||||
)
|
||||
.map_err(|e| e.to_string())?;
|
||||
}
|
||||
if let Some(index_key) = single_mapping {
|
||||
dest.execute(
|
||||
&format!("UPDATE OR REPLACE {table} SET server_id = ?2 WHERE server_id = ?1"),
|
||||
["", index_key],
|
||||
)
|
||||
.map_err(|e| e.to_string())?;
|
||||
}
|
||||
done = done.saturating_add(1);
|
||||
emit_progress(app, "analysis", table, done, total)?;
|
||||
}
|
||||
Ok(())
|
||||
})?;
|
||||
let source_rows = sum_table_rows(&source, ANALYSIS_TABLES)?;
|
||||
let imported_rows = sum_table_rows(&dest, ANALYSIS_TABLES)?;
|
||||
let skipped_unknown_server_rows = sum_unknown_rows(&source, ANALYSIS_TABLES, &legacy_ids, &index_keys)?;
|
||||
Ok((source_rows, imported_rows, skipped_unknown_server_rows))
|
||||
}
|
||||
|
||||
fn normalize_mappings(mappings: Vec<ServerIndexMapping>) -> Vec<ServerIndexMapping> {
|
||||
let mut out: Vec<ServerIndexMapping> = Vec::new();
|
||||
for mapping in mappings {
|
||||
let legacy_id = mapping.legacy_id.trim().to_string();
|
||||
let index_key = mapping.index_key.trim().to_string();
|
||||
if legacy_id.is_empty() || index_key.is_empty() {
|
||||
continue;
|
||||
}
|
||||
if let Some(existing) = out.iter_mut().find(|v| v.legacy_id == legacy_id) {
|
||||
existing.index_key = index_key;
|
||||
} else {
|
||||
out.push(ServerIndexMapping {
|
||||
legacy_id,
|
||||
index_key,
|
||||
});
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn inspect_tables(
|
||||
db_path: &Path,
|
||||
tables: &[&str],
|
||||
legacy_ids: &[String],
|
||||
known_index_keys: &[String],
|
||||
) -> Result<(HashMap<String, u64>, u64, u64), String> {
|
||||
let mut counts = HashMap::new();
|
||||
if !db_path.exists() {
|
||||
return Ok((counts, 0, 0));
|
||||
}
|
||||
let conn = open_readonly(db_path)?;
|
||||
let mut total = 0_u64;
|
||||
let mut skipped_unknown_server_rows = 0_u64;
|
||||
for table in tables {
|
||||
let count = count_rows_in(&conn, table, legacy_ids)? as u64;
|
||||
if count > 0 {
|
||||
counts.insert((*table).to_string(), count);
|
||||
total = total.saturating_add(count);
|
||||
}
|
||||
let unknown =
|
||||
count_unknown_rows(&conn, table, legacy_ids, known_index_keys)? as u64;
|
||||
skipped_unknown_server_rows = skipped_unknown_server_rows.saturating_add(unknown);
|
||||
}
|
||||
Ok((counts, total, skipped_unknown_server_rows))
|
||||
}
|
||||
|
||||
fn count_rows_in(conn: &Connection, table: &str, values: &[String]) -> Result<i64, String> {
|
||||
if values.is_empty() {
|
||||
return Ok(0);
|
||||
}
|
||||
let placeholders = std::iter::repeat_n("?", values.len()).collect::<Vec<_>>().join(",");
|
||||
let sql = format!("SELECT COUNT(*) FROM {table} WHERE server_id IN ({placeholders})");
|
||||
conn.query_row(&sql, params_from_iter(values.iter()), |row| row.get(0))
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn count_rows_eq(conn: &Connection, table: &str, value: &str) -> Result<i64, String> {
|
||||
conn.query_row(
|
||||
&format!("SELECT COUNT(*) FROM {table} WHERE server_id = ?1"),
|
||||
[&value],
|
||||
|row| row.get(0),
|
||||
)
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn count_unknown_rows(
|
||||
conn: &Connection,
|
||||
table: &str,
|
||||
known_legacy_ids: &[String],
|
||||
known_index_keys: &[String],
|
||||
) -> Result<i64, String> {
|
||||
let known = known_server_ids(known_legacy_ids, known_index_keys);
|
||||
if known.is_empty() {
|
||||
return conn
|
||||
.query_row(
|
||||
&format!("SELECT COUNT(*) FROM {table} WHERE server_id <> ''"),
|
||||
[],
|
||||
|row| row.get(0),
|
||||
)
|
||||
.map_err(|e| e.to_string());
|
||||
}
|
||||
let placeholders = std::iter::repeat_n("?", known.len())
|
||||
.collect::<Vec<_>>()
|
||||
.join(",");
|
||||
let sql =
|
||||
format!("SELECT COUNT(*) FROM {table} WHERE server_id <> '' AND server_id NOT IN ({placeholders})");
|
||||
conn.query_row(&sql, params_from_iter(known.iter()), |row| row.get(0))
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn purge_unknown_rows(
|
||||
conn: &Connection,
|
||||
table: &str,
|
||||
known_legacy_ids: &[String],
|
||||
known_index_keys: &[String],
|
||||
) -> Result<(), String> {
|
||||
let known = known_server_ids(known_legacy_ids, known_index_keys);
|
||||
if known.is_empty() {
|
||||
conn.execute(&format!("DELETE FROM {table} WHERE server_id <> ''"), [])
|
||||
.map_err(|e| e.to_string())?;
|
||||
return Ok(());
|
||||
}
|
||||
let placeholders = std::iter::repeat_n("?", known.len())
|
||||
.collect::<Vec<_>>()
|
||||
.join(",");
|
||||
let sql =
|
||||
format!("DELETE FROM {table} WHERE server_id <> '' AND server_id NOT IN ({placeholders})");
|
||||
conn.execute(&sql, params_from_iter(known.iter()))
|
||||
.map_err(|e| e.to_string())?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn known_server_ids(known_legacy_ids: &[String], known_index_keys: &[String]) -> Vec<String> {
|
||||
let mut known: Vec<String> = Vec::new();
|
||||
known.extend(known_legacy_ids.iter().cloned());
|
||||
known.extend(known_index_keys.iter().cloned());
|
||||
known.sort();
|
||||
known.dedup();
|
||||
known
|
||||
}
|
||||
|
||||
fn sum_table_rows(conn: &Connection, tables: &[&str]) -> Result<u64, String> {
|
||||
let mut total = 0_u64;
|
||||
for table in tables {
|
||||
let rows: i64 = conn
|
||||
.query_row(&format!("SELECT COUNT(*) FROM {table}"), [], |row| row.get(0))
|
||||
.map_err(|e| e.to_string())?;
|
||||
total = total.saturating_add(rows.max(0) as u64);
|
||||
}
|
||||
Ok(total)
|
||||
}
|
||||
|
||||
fn sum_unknown_rows(
|
||||
conn: &Connection,
|
||||
tables: &[&str],
|
||||
known_legacy_ids: &[String],
|
||||
known_index_keys: &[String],
|
||||
) -> Result<u64, String> {
|
||||
let mut total = 0_u64;
|
||||
for table in tables {
|
||||
let rows = count_unknown_rows(conn, table, known_legacy_ids, known_index_keys)?;
|
||||
total = total.saturating_add(rows.max(0) as u64);
|
||||
}
|
||||
Ok(total)
|
||||
}
|
||||
|
||||
fn with_foreign_keys_disabled<T>(
|
||||
conn: &Connection,
|
||||
operation: impl FnOnce() -> Result<T, String>,
|
||||
) -> Result<T, String> {
|
||||
conn.execute_batch("PRAGMA foreign_keys = OFF; BEGIN IMMEDIATE;")
|
||||
.map_err(|e| e.to_string())?;
|
||||
let result = operation();
|
||||
match result {
|
||||
Ok(value) => {
|
||||
if let Err(err) = conn
|
||||
.execute_batch("COMMIT; PRAGMA foreign_keys = ON;")
|
||||
.map_err(|e| e.to_string())
|
||||
{
|
||||
let _ = conn.execute_batch("ROLLBACK; PRAGMA foreign_keys = ON;");
|
||||
return Err(err);
|
||||
}
|
||||
ensure_foreign_keys_clean(conn)?;
|
||||
Ok(value)
|
||||
}
|
||||
Err(err) => {
|
||||
let _ = conn.execute_batch("ROLLBACK; PRAGMA foreign_keys = ON;");
|
||||
Err(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn ensure_foreign_keys_clean(conn: &Connection) -> Result<(), String> {
|
||||
let mut stmt = conn
|
||||
.prepare("PRAGMA foreign_key_check")
|
||||
.map_err(|e| e.to_string())?;
|
||||
let mut rows = stmt.query([]).map_err(|e| e.to_string())?;
|
||||
if let Some(row) = rows.next().map_err(|e| e.to_string())? {
|
||||
let table: String = row.get(0).map_err(|e| e.to_string())?;
|
||||
let rowid: i64 = row.get(1).map_err(|e| e.to_string())?;
|
||||
let parent: String = row.get(2).map_err(|e| e.to_string())?;
|
||||
let fkid: i64 = row.get(3).map_err(|e| e.to_string())?;
|
||||
return Err(format!(
|
||||
"foreign key check failed table={table} rowid={rowid} parent={parent} fkid={fkid}"
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn emit_progress(app: &AppHandle, stage: &str, table: &str, done: u64, total: u64) -> Result<(), String> {
|
||||
app.emit(
|
||||
"migration:progress",
|
||||
MigrationProgressEvent {
|
||||
stage: stage.to_string(),
|
||||
table: table.to_string(),
|
||||
done,
|
||||
total,
|
||||
},
|
||||
)
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn open_readonly(path: &Path) -> Result<Connection, String> {
|
||||
Connection::open_with_flags(path, OpenFlags::SQLITE_OPEN_READ_ONLY).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn vacuum_copy(source: &Path, destination: &Path) -> Result<(), String> {
|
||||
if let Some(parent) = destination.parent() {
|
||||
fs::create_dir_all(parent).map_err(|e| e.to_string())?;
|
||||
}
|
||||
let conn = Connection::open(source).map_err(|e| e.to_string())?;
|
||||
let sql = format!(
|
||||
"VACUUM INTO '{}';",
|
||||
destination.to_string_lossy().replace('\'', "''")
|
||||
);
|
||||
conn.execute_batch(&sql).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn switch_file(active: &Path, destination: &Path) -> Result<PathBuf, String> {
|
||||
let backup = active.with_file_name(format!(
|
||||
"{}.backup-pre-indexkey",
|
||||
active
|
||||
.file_name()
|
||||
.and_then(|n| n.to_str())
|
||||
.unwrap_or("db.sqlite")
|
||||
));
|
||||
remove_db_with_sidecars(&backup).ok();
|
||||
if active.exists() {
|
||||
fs::rename(active, &backup).map_err(|e| e.to_string())?;
|
||||
}
|
||||
fs::rename(destination, active).map_err(|e| e.to_string())?;
|
||||
Ok(backup)
|
||||
}
|
||||
|
||||
fn restore_backup(backup: &Path, active: &Path) -> Result<(), String> {
|
||||
if active.exists() {
|
||||
fs::remove_file(active).map_err(|e| e.to_string())?;
|
||||
}
|
||||
if backup.exists() {
|
||||
fs::rename(backup, active).map_err(|e| e.to_string())?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn health_check(library_active: &Path, analysis_active: &Path) -> Result<(), String> {
|
||||
if library_active.exists() {
|
||||
let conn = open_readonly(library_active)?;
|
||||
conn.query_row("SELECT COUNT(*) FROM track", [], |_row| Ok(()))
|
||||
.map_err(|e| e.to_string())?;
|
||||
}
|
||||
if analysis_active.exists() {
|
||||
let conn = open_readonly(analysis_active)?;
|
||||
conn.query_row("SELECT COUNT(*) FROM analysis_track", [], |_row| Ok(()))
|
||||
.map_err(|e| e.to_string())?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn remove_db_with_sidecars(path: &Path) -> Result<(), String> {
|
||||
remove_if_exists(path)?;
|
||||
let wal = PathBuf::from(format!("{}-wal", path.to_string_lossy()));
|
||||
let shm = PathBuf::from(format!("{}-shm", path.to_string_lossy()));
|
||||
remove_if_exists(&wal)?;
|
||||
remove_if_exists(&shm)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn remove_if_exists(path: &Path) -> Result<(), String> {
|
||||
if path.exists() {
|
||||
fs::remove_file(path).map_err(|e| e.to_string())?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
struct MigrationPaths {
|
||||
library_active: PathBuf,
|
||||
library_v2: PathBuf,
|
||||
analysis_active: PathBuf,
|
||||
analysis_v2: PathBuf,
|
||||
}
|
||||
|
||||
fn migration_paths(app: &AppHandle) -> Result<MigrationPaths, String> {
|
||||
let base = app.path().app_data_dir().map_err(|e| e.to_string())?;
|
||||
let library_dir = base.join("databases").join("library");
|
||||
let analysis_dir = base.join("databases").join("analysis");
|
||||
fs::create_dir_all(&library_dir).map_err(|e| e.to_string())?;
|
||||
fs::create_dir_all(&analysis_dir).map_err(|e| e.to_string())?;
|
||||
Ok(MigrationPaths {
|
||||
library_active: library_dir.join("library.sqlite"),
|
||||
library_v2: library_dir.join("library-v2.sqlite"),
|
||||
analysis_active: analysis_dir.join("audio-analysis.sqlite"),
|
||||
analysis_v2: analysis_dir.join("analysis-v2.sqlite"),
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn inspect_reports_skipped_unknown_rows() {
|
||||
let conn = Connection::open_in_memory().expect("in-memory sqlite");
|
||||
conn.execute_batch("CREATE TABLE track (server_id TEXT NOT NULL);")
|
||||
.expect("create table");
|
||||
conn.execute("INSERT INTO track(server_id) VALUES (?1)", ["legacy-a"])
|
||||
.expect("insert known legacy");
|
||||
conn.execute("INSERT INTO track(server_id) VALUES (?1)", ["removed-x"])
|
||||
.expect("insert unknown");
|
||||
|
||||
let known_legacy_ids = vec!["legacy-a".to_string()];
|
||||
let known_index_keys = vec!["idx-a".to_string()];
|
||||
let unknown = count_unknown_rows(&conn, "track", &known_legacy_ids, &known_index_keys)
|
||||
.expect("unknown count");
|
||||
assert_eq!(unknown, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn run_reports_skipped_unknown_rows_without_failure() {
|
||||
let conn = Connection::open_in_memory().expect("in-memory sqlite");
|
||||
conn.execute_batch("CREATE TABLE analysis_track (server_id TEXT NOT NULL);")
|
||||
.expect("create table");
|
||||
conn.execute("INSERT INTO analysis_track(server_id) VALUES (?1)", ["legacy-a"])
|
||||
.expect("insert known legacy");
|
||||
conn.execute("INSERT INTO analysis_track(server_id) VALUES (?1)", ["removed-x"])
|
||||
.expect("insert unknown");
|
||||
|
||||
let known_legacy_ids = vec!["legacy-a".to_string()];
|
||||
let known_index_keys = vec!["idx-a".to_string()];
|
||||
let skipped = sum_unknown_rows(&conn, &["analysis_track"], &known_legacy_ids, &known_index_keys)
|
||||
.expect("sum unknown rows");
|
||||
assert_eq!(skipped, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn needs_migration_false_when_only_unknown_rows_present() {
|
||||
let conn = Connection::open_in_memory().expect("in-memory sqlite");
|
||||
conn.execute_batch("CREATE TABLE track (server_id TEXT NOT NULL);")
|
||||
.expect("create table");
|
||||
conn.execute("INSERT INTO track(server_id) VALUES (?1)", ["removed-x"])
|
||||
.expect("insert unknown");
|
||||
|
||||
let known_legacy_ids = vec!["legacy-a".to_string()];
|
||||
let known_index_keys = vec!["idx-a".to_string()];
|
||||
let legacy = count_rows_in(&conn, "track", &known_legacy_ids).expect("legacy count");
|
||||
let unknown = count_unknown_rows(&conn, "track", &known_legacy_ids, &known_index_keys)
|
||||
.expect("unknown count");
|
||||
assert_eq!(legacy, 0);
|
||||
assert_eq!(unknown, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn purge_unknown_rows_removes_only_removed_servers() {
|
||||
let conn = Connection::open_in_memory().expect("in-memory sqlite");
|
||||
conn.execute_batch("CREATE TABLE track (server_id TEXT NOT NULL);")
|
||||
.expect("create table");
|
||||
conn.execute("INSERT INTO track(server_id) VALUES (?1)", ["legacy-a"])
|
||||
.expect("insert legacy");
|
||||
conn.execute("INSERT INTO track(server_id) VALUES (?1)", ["idx-a"])
|
||||
.expect("insert index key");
|
||||
conn.execute("INSERT INTO track(server_id) VALUES (?1)", [""])
|
||||
.expect("insert empty bucket");
|
||||
conn.execute("INSERT INTO track(server_id) VALUES (?1)", ["removed-x"])
|
||||
.expect("insert removed server");
|
||||
|
||||
let known_legacy_ids = vec!["legacy-a".to_string()];
|
||||
let known_index_keys = vec!["idx-a".to_string()];
|
||||
purge_unknown_rows(&conn, "track", &known_legacy_ids, &known_index_keys)
|
||||
.expect("purge unknown rows");
|
||||
|
||||
let remaining: i64 = conn
|
||||
.query_row("SELECT COUNT(*) FROM track", [], |row| row.get(0))
|
||||
.expect("count remaining");
|
||||
let removed_left: i64 = conn
|
||||
.query_row(
|
||||
"SELECT COUNT(*) FROM track WHERE server_id = 'removed-x'",
|
||||
[],
|
||||
|row| row.get(0),
|
||||
)
|
||||
.expect("count removed server rows");
|
||||
assert_eq!(remaining, 3);
|
||||
assert_eq!(removed_left, 0);
|
||||
}
|
||||
}
|
||||
@@ -1,10 +1,15 @@
|
||||
mod backup;
|
||||
mod cli_bridge;
|
||||
mod core;
|
||||
mod integration;
|
||||
mod migration;
|
||||
mod perf;
|
||||
pub(crate) mod platform;
|
||||
|
||||
// Tauri commands re-exported for the lib.rs invoke_handler.
|
||||
pub(crate) use backup::{
|
||||
backup_export_full, backup_export_library_db, backup_import_full, backup_import_library_db,
|
||||
};
|
||||
pub(crate) use cli_bridge::{
|
||||
cli_publish_library_list, cli_publish_player_snapshot, cli_publish_search_results,
|
||||
cli_publish_server_list,
|
||||
@@ -27,6 +32,7 @@ pub(crate) use integration::{
|
||||
check_dir_accessible, mpris_set_metadata, mpris_set_playback, register_global_shortcut,
|
||||
unregister_global_shortcut,
|
||||
};
|
||||
pub(crate) use migration::{migration_inspect, migration_run};
|
||||
|
||||
// Discord, Navidrome admin, last.fm + radio-browser + CORS proxy, bandsintown,
|
||||
// and analysis admin commands now live in their domain crates. invoke_handler!
|
||||
|
||||
Reference in New Issue
Block a user