feat(archive): photo delete, tombstones, trash kinds and emptying, library API

Owner review 2, R6: the web panel manages the library without editing it, so
the engine and the server must delete single photos safely, keep them out of
sync afterwards, and let an administrator restore or finally erase the trash.

- Migration 0008: frame_removals (tombstones by sequence and SHA-256, reason,
  replacement); the sidecar gains "removed" (omitted when empty) and frames a
  reserved "source"; the indexer writes the tombstones on every sync, so a
  rebuild keeps them, and drops a hero frame that is gone.
- FrameTrash: dry run (busy, frames changed, offline, would empty), delete under
  the sequence lease with a write-ahead journal that the next writer undoes
  after a crash, files to trash/photos/<seq>-<time>/originals with a record,
  sidecar without the frames plus tombstones, index sync clearing analyses,
  thumbnails and proxies, proxies renamed to follow their frames, follow-up
  jobs. Restore re-hashes every file, keeps mismatches in the trash, puts the
  same frame records back and drops their tombstones, or files the photos into
  a new sequence through the ingest when theirs is gone.
- Project positions on vanished photos are translated by photo and wait as
  pending frames, so they come back with a restore.
- Trash entries have a kind (sequence, photos, replaced); emptying is a dry run
  with a fingerprint and a new EmptyTrash job (18) that erases exactly the
  planned entries, refuses a stale plan and erases replaced originals only
  while their replacements still hash right.
- Sync honours tombstones: nothing one side deleted is offered back, local
  files are never deleted, replaced photos count as their replacement; the
  server answers an upload of a tombstoned SHA-256 as already there.
- Server: sequence thumbnail, photo delete, bulk delete/retag/jobs, trash
  list/restore/empty endpoints with their error messages; a verify of
  sequences now targets their archive (POST /jobs failed before).
- Desktop trash dialog lists photo entries and empties the trash after a
  typed confirmation through the same job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 35bb56690a1d69eae57153d036e1409201cd73f7)
This commit is contained in:
Alby96 committed 2026-10-27 10:49:34 +01:00
1 parent 4b9ba04fe4
commit 51782a3ada
63 files changed
+3303 -60

No files matched your search

+6 -5
View File
@@ -54,13 +54,13 @@ Dependencies point one way: `Core` ← `Core.Windows`, `Data`, `Protocol` ← `S
|---|---|
| `Sqlite`, `Postgres` | SQLite P/Invoke binding (winsqlite3 / libsqlite3), in-house PostgreSQL wire protocol v3 client with pool and LISTEN |
| `Migrations/{sqlite,postgres}/NNNN_name.sql` | Embedded, applied in one transaction at open (`MigrationRunner`) |
| `Repositories` | `ChronosDatabase` facade: archives, sequences, frames, tags, renders, jobs, cards, users, tokens, uploads, render nodes, settings, audit, schedules (`ScheduleRepository`: system rows seeded without overwriting edits and never deleted, the compare-and-set claim of a run); `DatabaseBackups` (`VACUUM INTO` copies of an SQLite index into `backups` next to it, every copy keeping as many as the backup schedules keep (`KeptAsync`), the newest copy that seeds the `index-backup` schedule (`Newest`, `SystemSchedule.LastDone`), restore staged for the next start and carried out before the index opens, so it also recovers one that no longer opens) |
| `Repositories` | `ChronosDatabase` facade: archives, sequences, frames, tags, renders, jobs, cards, users, tokens, uploads, render nodes, settings, audit, schedules (`ScheduleRepository`: system rows seeded without overwriting edits and never deleted, the compare-and-set claim of a run), tombstones of removed photos (`FrameRemovalRepository`, `frame_removals`, written from the sidecar's `removed` list); `DatabaseBackups` (`VACUUM INTO` copies of an SQLite index into `backups` next to it, every copy keeping as many as the backup schedules keep (`KeptAsync`), the newest copy that seeds the `index-backup` schedule (`Newest`, `SystemSchedule.LastDone`), restore staged for the next start and carried out before the index opens, so it also recovers one that no longer opens) |
| `Jobs` | `JobEngine` (one job at a time; reviews the priority queue and makes a lower running job yield, requeued with its checkpoint; no history purge, the housekeeping schedule's clean-up does it), `JobPriorityQueue` (a level per job kind, the claim order, settings round trip), `IJobHandler`, `IJobHandlerProvider`, `JobContext` (progress, checkpoints, follow-up jobs that never interrupt their parent), `JobThroughput`/`JobEta` (rate and bounded time left) |
| `Scheduling` | `Scheduler` (one per host: seeds the host's system schedules, claims due runs by compare-and-set, catch-up, clock changes, create/edit/reset/delete/run now as API v1 DTOs), `ScheduleRecurrence` (the in-house `every:`/`daily:`/`weekly:`/`monthly:` form, wall-clock times across DST), `ScheduleActions` (action names, parameters, availability, the archive jobs a run queues), `IScheduleHost` and `SystemSchedule` (what the server and the desktop supply), `ScheduleOutcome`, `ScheduleMapping` (model ↔ DTO) |
| `Housekeeping` | `BackupJobHandler`, `CleanupJobHandler`, `ICleanupStep` with the shared steps (`CleanupSteps`: job history, audit log, trash report that never deletes); hosts add their own steps through `ArchiveServiceOptions.CleanupSteps` |
| `Archiving` | `ArchiveService` (create/adopt/open, archive settings written to `archive.json` then the index), `ArchiveIndexer` (scan, rebuild), `ArchiveIngest` (uploads and imports), `ImportService`, `ArchiveVerifier`, `ArchiveImages` (thumbnail and proxy passes, several photos at once through `OrderedWork`, which hands results back in frame order), `FrameTiers` (decode tier, `PreviewOnly`/`Undecodable` flags and preview size of frames, written to sidecar and index by the light analysis, deep analysis, renders and image passes), `GprCompanions` (the JPEG of each GoPro GPR, found in the index when an import kept it in another sequence), `ArchiveMaintenance`, `MassRenamer`, `SequenceTrash` (delete into the archive's hidden trash with a dry run, restore; the folder moves back when the index step fails), job handlers |
| `Archiving` | `ArchiveService` (create/adopt/open, archive settings written to `archive.json` then the index), `ArchiveIndexer` (scan, rebuild), `ArchiveIngest` (uploads and imports), `ImportService`, `ArchiveVerifier`, `ArchiveImages` (thumbnail and proxy passes, several photos at once through `OrderedWork`, which hands results back in frame order), `FrameTiers` (decode tier, `PreviewOnly`/`Undecodable` flags and preview size of frames, written to sidecar and index by the light analysis, deep analysis, renders and image passes), `GprCompanions` (the JPEG of each GoPro GPR, found in the index when an import kept it in another sequence), `ArchiveMaintenance`, `MassRenamer`, `SequenceTrash` (delete into the archive's hidden trash with a dry run, restore; the folder moves back when the index step fails; lists sequence and photo entries; `SequenceTrash.Empty`: the emptying's dry run, fingerprint and erase), `FrameTrash` (photos of a sequence into `trash/photos` with a dry run, a journal that undoes an interrupted deletion, tombstones, positions parked and placed back, proxies renamed to follow their frames; restore re-hashes each file, into a new sequence through the ingest when theirs is gone), job handlers |
| `Processing` | `ProcessingService`, `SequenceAnalyser`, `DeepReports` (a saved deep report while it still describes the frames), `SequenceRender`, `ProjectStore`, `ProjectBundles` (a `.chronos` file for a sequence, and the sequence a file names), `RenderNodeClient`, job handlers |
| `Sync` | `SyncService` (sync table of one server: compare, link by content, settle conflicts, enqueue transfers), `SyncRules` (status, settings merge and differences, no I/O), `SyncPositions` (the settings' frame positions translated by photo to the frames of the copy they go to), `SyncLinkStore` (`sync_links`, `sync_state`), `SyncTransfers` and `SyncJobHandler` (resumable, hash-verified uploads and downloads as jobs; downloads commit through the archive ingest), `SyncAgent` (background comparison, server events, backoff, offline proxies), `ProxyCache` and `RemoteSequence` (server sequences opened with proxies), `ISyncRemote` (the server calls; `ChronosSyncRemote` over `ChronosClient`) |
| `Sync` | `SyncService` (sync table of one server: compare, link by content, settle conflicts, enqueue transfers), `SyncRules` (status, settings merge and differences, no I/O), `SyncTombstones` (photos either side removed are not offered back, a replaced photo counts as its replacement), `SyncPositions` (the settings' frame positions translated by photo to the frames of the copy they go to), `SyncLinkStore` (`sync_links`, `sync_state`), `SyncTransfers` and `SyncJobHandler` (resumable, hash-verified uploads and downloads as jobs; downloads commit through the archive ingest), `SyncAgent` (background comparison, server events, backoff, offline proxies), `ProxyCache` and `RemoteSequence` (server sequences opened with proxies), `ISyncRemote` (the server calls; `ChronosSyncRemote` over `ChronosClient`) |
### 2.3 Chronos.App areas
@@ -71,7 +71,7 @@ Dependencies point one way: `Core` ← `Core.Windows`, `Data`, `Protocol` ← `S
| `Library` | Filter, facets, pager, timeline, saved views, quick and mass actions, remote cards and thumbnails, thumbnail cache, list export |
| `Editor` | `EditorSession` (project, undo/redo, saving, merges), `ISequenceSource` (`LocalSequenceSource`, `RemoteSequenceSource`), the steps of the Edit path (`EditPath`: their order, each step's state, the step of each finding, the memory of the step shown; tabs Analysis, Frames, Exposure, Motion, Camera, Time, Stack, Export), `PreviewRenderer`, `ExportPlanner`, `ExportSync`, `SavedCopies`, `SequenceJobs`, `ProjectFiles` (save and open `.chronos` project files) |
| `Import` | `CardPool` (inserted cards, remembered cards, folders), camera profiles, plans and choices, destinations (archive, folder, server through `ServerImportSink`), card actions (verified delete, format, eject) |
| `Archive` | `CurrentArchive`, `ArchiveTree`, figures for the charts, maintenance reports, mass rename and move, delete and trash dialogs (`SequenceDeletion`, `DeletePlans`), `ArchiveFailures` |
| `Archive` | `CurrentArchive`, `ArchiveTree`, figures for the charts, maintenance reports, mass rename and move, delete and trash dialogs (`SequenceDeletion`, `DeletePlans`, `TrashEmptyPlans`: Empty… behind a dry run and a typed confirmation), `ArchiveFailures` |
| `Server` | `ServerSession`, `ConnectFlow` (trust on first use), `SyncHost` (sync engine, agent, metered hold, editor connections, Export Sync), sync table, `RemoteSequenceApi` |
| `Jobs`, `Settings`, `About` | Job list and buffering, `JobNotices` (which job endings raise a toast; `JobEndings` lets a batch speak once); the schedules of Jobs › Scheduled (`DesktopScheduleHost`: the desktop's `IScheduleHost` and its system schedules `archive-scan`, `index-backup`, `housekeeping`; `CacheTrimStep`, the host's clean-up step; `SchedulesViewModel`, `ScheduleRowViewModel`, `ScheduleEditorViewModel` and `ScheduleDialogs.xaml`, the in-window editor; `ScheduleText`); settings rows (`LogSettingsRows`: Advanced's log rows), encoder and format inventories, `ProcessingTuning` (Settings › Rendering and Analysis applied to each job: parallelism, memory budget, quality), `InterfacePriority`, language catalog, naming preview; `AboutViewModel` (owned by the shell, not a section: Settings › About, the update notice of Settings › Updates and the update check on start), update check and download |
| `Infrastructure` | `AppServices` (composition root), `AppSettings`, `AppLogging` (Settings › Advanced › Logs applied to the log files from their first line, a launch that only hands over included, and at every save), `AppEnvironment` (installed or portable), `AppClock` (today's date, frozen by `--capture`), `StartWithWindows` (this copy's sign-in entry), `UiLanguage`, `LiveRegion`, `RenderNodeHost`, `DiagnosticsBundle`, `ShellActions` (BRIEF §8.6), `SingleInstance` (one Chronos per data folder; a later launch hands its file over a named pipe, or waits for a closing one to exit and starts in its place), device monitor |
@@ -85,7 +85,7 @@ Dependencies point one way: `Core` ← `Core.Windows`, `Data`, `Protocol` ← `S
| `Ui/Controls` | `ChronosWindow` (chrome, language direction), `NavRail`, `JobBar`, `ToastHost`, `Field` (placeholder, icon, error), `Chip`, `AutoSlider`, `StatusGlyph`, `EmptyState`, `Icon`, `Bidi` (right-to-left rules), `Motion` (animations off for screenshots) |
| `Ui/Interop` | DWM calls of the window chrome; `TrayIcon` (`Shell_NotifyIconW`: the Uploader's tray, the client's Windows notifications) |
| `Ui/Visuals` | `DrawingElement` (hover readout, item keyboard and activation) and its charts (`BarChart`, `SeriesChart`, `StackedBar`, `HeatmapStrip`, `TimelineBands`), `Filmstrip`, `PreviewViewport`, `CurveEditor`, `SequenceCard`, `VirtualizingWrapPanel`, `SegmentedControl`, `SplitButton` |
| `Server/Api`, `Server/Events` | Endpoint groups (`XxxEndpoints.Map`; `ScheduleEndpoints`: `/api/v1/schedules`), API errors, the WebSocket `EventHub` |
| `Server/Api`, `Server/Events` | Endpoint groups (`XxxEndpoints.Map`; `ScheduleEndpoints`: `/api/v1/schedules`; `LibraryEndpoints`: sequence thumbnail, photo delete, bulk delete/retag/jobs; `TrashEndpoints`: list, restore, empty), API errors, the WebSocket `EventHub` |
| `Server/Security` | Accounts, password hashing, tokens, session cookies, login guard, access rules, audit log |
| `Server/Storage`, `Server/Jobs` | Archive and processing modules, uploads (`UploadMaintenance`: recovery at start, and the sweep that is the `uploads` clean-up step), render store; `JobEngineHost`, `SchedulerHost` (the `Scheduler` on the engine, started after it; archive.json read every 10 minutes whatever the scan schedule does; every schedule change and run published as a `scheduleChanged` event) and `ServerScheduleHost` (time zone, archive refreshed from archive.json, system schedules), render node service, monitor and availability |
| `Server/Hosting`, `Server/Admin/web` | `ServerHost`, `ServerLogging` (console and file levels the logging filters read at every line, the file sink's limits, the engine's level; applied again when Settings › Logs is saved), certificates, discovery beacon, embedded static files, Windows service (start and stop failures in the Application event log); the admin panel (ES modules, one per page; `schedules.js` is the Jobs page's Scheduled tab) |
@@ -114,6 +114,7 @@ Dependencies point one way: `Core` ← `Core.Windows`, `Data`, `Protocol` ← `S
| `Duplicates`, `Orphans` | `Archiving/Jobs/ReportJobHandlers` (read-only reports into the job log, queued by schedules); the Archive screen still queries `ArchiveService` directly | yes | yes |
| `Backup` | `Housekeeping/BackupJobHandler` (via `ArchiveService.CreateJobHandlers`; an SQLite index file only) | yes | SQLite only |
| `Cleanup` | `Housekeeping/CleanupJobHandler` (shared steps, plus the host's from `ArchiveServiceOptions.CleanupSteps`) | yes | yes |
| `EmptyTrash` | `Archiving/Jobs/EmptyTrashJobHandler` (erases exactly a confirmed plan's trash entries; refuses a stale fingerprint) | yes | yes |
| `Adopt`, `Move` | none: done synchronously (`ArchiveService` actions) | — | — |
A job engine claims only kinds it has handlers for. Light analysis is queued by the engine (`JobContext` follow-up)
+1
View File
@@ -125,6 +125,7 @@
<data name="Admin.Job.Kind.Cleanup" xml:space="preserve"><value>Clean-up</value></data>
<data name="Admin.Job.Kind.DeepAnalyse" xml:space="preserve"><value>Deep analysis</value></data>
<data name="Admin.Job.Kind.Duplicates" xml:space="preserve"><value>Duplicates</value></data>
<data name="Admin.Job.Kind.EmptyTrash" xml:space="preserve"><value>Empty trash</value></data>
<data name="Admin.Job.Kind.Import" xml:space="preserve"><value>Import</value></data>
<data name="Admin.Job.Kind.LightAnalyse" xml:space="preserve"><value>Light analysis</value></data>
<data name="Admin.Job.Kind.Maintenance" xml:space="preserve"><value>Maintenance</value></data>
+20 -1
View File
@@ -89,6 +89,10 @@
<data name="Archive.Error.SourceChanged" xml:space="preserve"><value>{0} changed after the plan was made.</value><comment>long: error sentence</comment></data>
<data name="Archive.Error.SourceGone" xml:space="preserve"><value>The card or folder {0} is no longer available.</value><comment>long: error sentence</comment></data>
<data name="Archive.Error.StalePlan" xml:space="preserve"><value>The archive changed after this plan was made.</value><comment>long: error sentence</comment></data>
<data name="Archive.Error.TrashChanged" xml:space="preserve"><value>The trash changed after it was checked; nothing was erased.</value><comment>long: error sentence</comment></data>
<data name="Archive.Error.TrashKept.one" xml:space="preserve"><value>{0:N0} trash entry was kept: it changed, or a replacement no longer matches.</value><comment>long: error sentence of an emptying job</comment></data>
<data name="Archive.Error.TrashKept.other" xml:space="preserve"><value>{0:N0} trash entries were kept: they changed, or a replacement no longer matches.</value><comment>long: error sentence of an emptying job</comment></data>
<data name="Archive.Error.TrashMismatch" xml:space="preserve"><value>No file of this trash entry matches its record any more; it stays in the trash.</value><comment>long: error sentence</comment></data>
<data name="Archive.Failed.Access" xml:space="preserve"><value>Chronos was denied access to the archive.</value><comment>long</comment></data>
<data name="Archive.Failed.Files" xml:space="preserve"><value>A file of the archive could not be read or written.</value><comment>long</comment></data>
<data name="Archive.Failed.Gone" xml:space="preserve"><value>The sequence is no longer in the index.</value><comment>long</comment></data>
@@ -152,7 +156,9 @@
<data name="Archive.Log.SequenceMissing" xml:space="preserve"><value>Sequence not found: {0}</value><comment>long: job log line</comment></data>
<data name="Archive.Log.StepFailed" xml:space="preserve"><value>Clean-up step {0} failed: {1}</value><comment>long: job log line; {0} step name (job-history, audit…); {1} error message</comment></data>
<data name="Archive.Log.StepUnknown" xml:space="preserve"><value>Clean-up step unknown here: {0}</value><comment>long: job log line; {0} step name</comment></data>
<data name="Archive.Log.TrashEmptied" xml:space="preserve"><value>Entries erased: {0}, size: {1}</value><comment>long: job log line; {0} count; {1} size</comment></data>
<data name="Archive.Log.TrashEntry" xml:space="preserve"><value>In the trash since {0}: {1} ({2})</value><comment>long: job log line; {0} date; {1} sequence title; {2} size</comment></data>
<data name="Archive.Log.TrashKept" xml:space="preserve"><value>Kept in the trash: {0}</value><comment>long: job log line; {0} sequence title</comment></data>
<data name="Archive.Log.TrashOffline" xml:space="preserve"><value>Trash not checked: the archive cannot be reached</value><comment>long: job log line</comment></data>
<data name="Archive.Log.TrashReport" xml:space="preserve"><value>Old trash entries: {0}, size: {1}; nothing was deleted</value><comment>long: job log line; entries older than the clean-up's age limit</comment></data>
<data name="Archive.Log.VerifyDone" xml:space="preserve"><value>Photos checked: {0}, intact: {1}, hashed: {2}, with problems: {3}</value><comment>long: job log line</comment></data>
@@ -248,11 +254,24 @@
<data name="Archive.Toolbar.Rebuild" xml:space="preserve"><value>Rebuild index…</value></data>
<data name="Archive.Toolbar.Scan" xml:space="preserve"><value>Scan</value></data>
<data name="Archive.Toolbar.Verify" xml:space="preserve"><value>Verify</value></data>
<data name="Archive.Trash.Checking" xml:space="preserve"><value>Checking the trash…</value></data>
<data name="Archive.Trash.Deleted" xml:space="preserve"><value>Deleted {0}</value><comment>long</comment></data>
<data name="Archive.Trash.Empty" xml:space="preserve"><value>The trash is empty</value><comment>long</comment></data>
<data name="Archive.Trash.EmptyAction" xml:space="preserve"><value>Empty…</value></data>
<data name="Archive.Trash.EmptyHeading.one" xml:space="preserve"><value>Erase {0:N0} trash entry for good?</value><comment>long</comment></data>
<data name="Archive.Trash.EmptyHeading.other" xml:space="preserve"><value>Erase {0:N0} trash entries for good?</value><comment>long</comment></data>
<data name="Archive.Trash.EmptyMessage" xml:space="preserve"><value>{0} and {1} ({2}) are erased from the archive; this cannot be undone.</value><comment>long</comment></data>
<data name="Archive.Trash.Entries.one" xml:space="preserve"><value>{0:N0} entry</value></data>
<data name="Archive.Trash.Entries.other" xml:space="preserve"><value>{0:N0} entries</value></data>
<data name="Archive.Trash.Erase" xml:space="preserve"><value>Erase for good</value></data>
<data name="Archive.Trash.Failed" xml:space="preserve"><value>The sequence could not come back from the trash</value><comment>long</comment></data>
<data name="Archive.Trash.Heading" xml:space="preserve"><value>Trash · {0}</value><comment>long</comment></data>
<data name="Archive.Trash.List" xml:space="preserve"><value>Deleted sequences</value></data>
<data name="Archive.Trash.KeptFiles.one" xml:space="preserve"><value>{0:N0} file stayed in the trash: it no longer matches its record.</value><comment>long</comment></data>
<data name="Archive.Trash.KeptFiles.other" xml:space="preserve"><value>{0:N0} files stayed in the trash: they no longer match their record.</value><comment>long</comment></data>
<data name="Archive.Trash.Kind.Photos" xml:space="preserve"><value>Photos</value></data>
<data name="Archive.Trash.Kind.Replaced" xml:space="preserve"><value>Replaced originals</value></data>
<data name="Archive.Trash.Kind.Sequence" xml:space="preserve"><value>Sequence</value></data>
<data name="Archive.Trash.List" xml:space="preserve"><value>Trash entries</value></data>
<data name="Archive.Trash.Reading" xml:space="preserve"><value>Reading the trash…</value></data>
<data name="Archive.Trash.Restore" xml:space="preserve"><value>Restore</value></data>
<data name="Archive.Trash.Restored.one" xml:space="preserve"><value>{0:N0} sequence restored</value><comment>long</comment></data>
+1
View File
@@ -72,6 +72,7 @@
<data name="Jobs.Kind.DeepAnalyse" xml:space="preserve"><value>Deep analysis</value></data>
<data name="Jobs.Kind.Download" xml:space="preserve"><value>Download</value></data>
<data name="Jobs.Kind.Duplicates" xml:space="preserve"><value>Duplicates</value></data>
<data name="Jobs.Kind.EmptyTrash" xml:space="preserve"><value>Empty trash</value></data>
<data name="Jobs.Kind.Import" xml:space="preserve"><value>Import</value></data>
<data name="Jobs.Kind.LightAnalyse" xml:space="preserve"><value>Light analysis</value></data>
<data name="Jobs.Kind.Maintenance" xml:space="preserve"><value>Maintenance</value></data>
+4
View File
@@ -87,12 +87,15 @@
<data name="Server.Error.ScheduleSystem" xml:space="preserve"><value>A system schedule cannot be deleted; turn it off instead.</value><comment>long</comment></data>
<data name="Server.Error.Self" xml:space="preserve"><value>You cannot do this to your own account.</value><comment>long</comment></data>
<data name="Server.Error.SequenceBusy" xml:space="preserve"><value>A job is working on this sequence or its archive; try again when it is free.</value><comment>long</comment></data>
<data name="Server.Error.SequenceInLibrary" xml:space="preserve"><value>That sequence is in the library already.</value><comment>long</comment></data>
<data name="Server.Error.SessionExpired" xml:space="preserve"><value>The session expired; sign in again.</value><comment>long</comment></data>
<data name="Server.Error.SettingLocked" xml:space="preserve"><value>An environment variable sets this; change it there.</value><comment>long</comment></data>
<data name="Server.Error.SettingsConflict" xml:space="preserve"><value>The sequence changed on the server; reload it and apply your edit again.</value><comment>long</comment></data>
<data name="Server.Error.SidecarMismatch" xml:space="preserve"><value>Its folder holds another sequence; scan the archive first.</value><comment>long</comment></data>
<data name="Server.Error.TooLarge" xml:space="preserve"><value>The request is larger than the server accepts.</value><comment>long</comment></data>
<data name="Server.Error.TooManyUploads" xml:space="preserve"><value>Too many uploads at once; retry shortly.</value><comment>long</comment></data>
<data name="Server.Error.TrashChanged" xml:space="preserve"><value>The trash changed after it was checked; check it again before emptying it.</value><comment>long</comment></data>
<data name="Server.Error.TrashMismatch" xml:space="preserve"><value>No file of this trash entry matches its record any more; it stays in the trash.</value><comment>long</comment></data>
<data name="Server.Error.Unauthorized" xml:space="preserve"><value>Sign in to continue.</value><comment>long</comment></data>
<data name="Server.Error.Unreachable" xml:space="preserve"><value>The server does not answer; check the address and the network.</value><comment>long</comment></data>
<data name="Server.Error.UnsupportedMediaType" xml:space="preserve"><value>Send the body as JSON.</value><comment>long</comment></data>
@@ -101,6 +104,7 @@
<data name="Server.Error.UsernameTaken" xml:space="preserve"><value>That user name is taken.</value><comment>long</comment></data>
<data name="Server.Error.WeakPassword" xml:space="preserve"><value>Use at least {0} characters.</value><comment>long</comment></data>
<data name="Server.Error.WebsocketRequired" xml:space="preserve"><value>Connect with a WebSocket.</value><comment>long</comment></data>
<data name="Server.Error.WouldEmpty" xml:space="preserve"><value>Every photo of the sequence would go; delete the sequence instead.</value><comment>long</comment></data>
<data name="Server.Error.WrongPassword" xml:space="preserve"><value>The current password is wrong.</value><comment>long</comment></data>
<data name="Server.Error.WrongUploadKind" xml:space="preserve"><value>This upload belongs to a render job.</value><comment>long</comment></data>
<data name="Server.Found" xml:space="preserve"><value>Servers found</value></data>
+6 -2
View File
@@ -91,8 +91,12 @@
<data name="Tooltips.Archive.TagsAdd" xml:space="preserve"><value>Tags to add, separated by commas</value></data>
<data name="Tooltips.Archive.TagsRemove" xml:space="preserve"><value>Tags to remove, separated by commas</value></data>
<data name="Tooltips.Archive.TitleTemplate" xml:space="preserve"><value>Title template; the title token is the current title · checked as you type</value></data>
<data name="Tooltips.Archive.Trash" xml:space="preserve"><value>The sequences deleted from this archive, each one restorable</value></data>
<data name="Tooltips.Archive.TrashRestore" xml:space="preserve"><value>Move this sequence back into its folder and the library, with its settings and renders</value></data>
<data name="Tooltips.Archive.Trash" xml:space="preserve"><value>The sequences and photos deleted from this archive, each one restorable</value></data>
<data name="Tooltips.Archive.TrashEmpty" xml:space="preserve"><value>Erase the entries of this trash for good, after a check of what they hold and a typed confirmation</value></data>
<data name="Tooltips.Archive.TrashEmptyPlan" xml:space="preserve"><value>What is erased from the archive for good</value></data>
<data name="Tooltips.Archive.TrashErase.one" xml:space="preserve"><value>Erase {0:N0} entry for good; type the number above first</value></data>
<data name="Tooltips.Archive.TrashErase.other" xml:space="preserve"><value>Erase {0:N0} entries for good; type the number above first</value></data>
<data name="Tooltips.Archive.TrashRestore" xml:space="preserve"><value>Move this entry back: a sequence into its folder and the library, photos into their sequence after a hash check</value></data>
<data name="Tooltips.Archive.Verify" xml:space="preserve"><value>Re-hash every original and report bit rot · reads the whole archive</value></data>
<data name="Tooltips.Archive.YearRow" xml:space="preserve"><value>{0} · Right arrow opens the year, Left arrow closes it</value></data>
<data name="Tooltips.Archive.YearRowRtl" xml:space="preserve"><value>{0} · Left arrow opens the year, Right arrow closes it</value></data>
+1
View File
@@ -125,6 +125,7 @@
<data name="Admin.Job.Kind.Cleanup" xml:space="preserve"><value>Pulizia</value></data>
<data name="Admin.Job.Kind.DeepAnalyse" xml:space="preserve"><value>Analisi completa</value></data>
<data name="Admin.Job.Kind.Duplicates" xml:space="preserve"><value>Duplicati</value></data>
<data name="Admin.Job.Kind.EmptyTrash" xml:space="preserve"><value>Svuota cestino</value></data>
<data name="Admin.Job.Kind.Import" xml:space="preserve"><value>Importazione</value></data>
<data name="Admin.Job.Kind.LightAnalyse" xml:space="preserve"><value>Analisi rapida</value></data>
<data name="Admin.Job.Kind.Maintenance" xml:space="preserve"><value>Manutenzione</value></data>
+20 -1
View File
@@ -84,6 +84,10 @@
<data name="Archive.Error.SourceChanged" xml:space="preserve"><value>{0} è cambiato dopo la creazione del piano.</value></data>
<data name="Archive.Error.SourceGone" xml:space="preserve"><value>La scheda o cartella {0} non è più disponibile.</value></data>
<data name="Archive.Error.StalePlan" xml:space="preserve"><value>L'archivio è cambiato dopo la creazione del piano.</value></data>
<data name="Archive.Error.TrashChanged" xml:space="preserve"><value>Il cestino è cambiato dopo il controllo; non è stato cancellato nulla.</value></data>
<data name="Archive.Error.TrashKept.one" xml:space="preserve"><value>{0:N0} elemento del cestino è stato mantenuto: è cambiato, o una sostituzione non corrisponde più.</value></data>
<data name="Archive.Error.TrashKept.other" xml:space="preserve"><value>{0:N0} elementi del cestino sono stati mantenuti: sono cambiati, o una sostituzione non corrisponde più.</value></data>
<data name="Archive.Error.TrashMismatch" xml:space="preserve"><value>Nessun file di questo elemento del cestino corrisponde più al suo registro; resta nel cestino.</value></data>
<data name="Archive.Failed.Access" xml:space="preserve"><value>Chronos non ha accesso all'archivio.</value></data>
<data name="Archive.Failed.Files" xml:space="preserve"><value>Non è stato possibile leggere o scrivere un file dell'archivio.</value></data>
<data name="Archive.Failed.Gone" xml:space="preserve"><value>La sequenza non è più nell'indice.</value></data>
@@ -147,7 +151,9 @@
<data name="Archive.Log.SequenceMissing" xml:space="preserve"><value>Sequenza non trovata: {0}</value></data>
<data name="Archive.Log.StepFailed" xml:space="preserve"><value>Passo di pulizia {0} non riuscito: {1}</value></data>
<data name="Archive.Log.StepUnknown" xml:space="preserve"><value>Passo di pulizia sconosciuto qui: {0}</value></data>
<data name="Archive.Log.TrashEmptied" xml:space="preserve"><value>Elementi cancellati: {0}, dimensione: {1}</value></data>
<data name="Archive.Log.TrashEntry" xml:space="preserve"><value>Nel cestino dal {0}: {1} ({2})</value></data>
<data name="Archive.Log.TrashKept" xml:space="preserve"><value>Mantenuto nel cestino: {0}</value></data>
<data name="Archive.Log.TrashOffline" xml:space="preserve"><value>Cestino non controllato: l'archivio non è raggiungibile</value></data>
<data name="Archive.Log.TrashReport" xml:space="preserve"><value>Voci vecchie nel cestino: {0}, dimensione: {1}; nulla è stato eliminato</value></data>
<data name="Archive.Log.VerifyDone" xml:space="preserve"><value>Foto controllate: {0}, integre: {1}, con impronta nuova: {2}, con problemi: {3}</value></data>
@@ -241,11 +247,24 @@
<data name="Archive.Toolbar.Rebuild" xml:space="preserve"><value>Ricostruisci indice…</value></data>
<data name="Archive.Toolbar.Scan" xml:space="preserve"><value>Scansiona</value></data>
<data name="Archive.Toolbar.Verify" xml:space="preserve"><value>Verifica</value></data>
<data name="Archive.Trash.Checking" xml:space="preserve"><value>Controllo del cestino…</value></data>
<data name="Archive.Trash.Deleted" xml:space="preserve"><value>Eliminata il {0}</value></data>
<data name="Archive.Trash.Empty" xml:space="preserve"><value>Il cestino è vuoto</value></data>
<data name="Archive.Trash.EmptyAction" xml:space="preserve"><value>Svuota…</value></data>
<data name="Archive.Trash.EmptyHeading.one" xml:space="preserve"><value>Cancellare per sempre {0:N0} elemento del cestino?</value></data>
<data name="Archive.Trash.EmptyHeading.other" xml:space="preserve"><value>Cancellare per sempre {0:N0} elementi del cestino?</value></data>
<data name="Archive.Trash.EmptyMessage" xml:space="preserve"><value>{0} e {1} ({2}) vengono cancellati dall'archivio; non si può annullare.</value></data>
<data name="Archive.Trash.Entries.one" xml:space="preserve"><value>{0:N0} elemento</value></data>
<data name="Archive.Trash.Entries.other" xml:space="preserve"><value>{0:N0} elementi</value></data>
<data name="Archive.Trash.Erase" xml:space="preserve"><value>Cancella per sempre</value></data>
<data name="Archive.Trash.Failed" xml:space="preserve"><value>Impossibile ripristinare la sequenza dal cestino</value></data>
<data name="Archive.Trash.Heading" xml:space="preserve"><value>Cestino · {0}</value></data>
<data name="Archive.Trash.List" xml:space="preserve"><value>Sequenze eliminate</value></data>
<data name="Archive.Trash.KeptFiles.one" xml:space="preserve"><value>{0:N0} file è rimasto nel cestino: non corrisponde più al suo registro.</value></data>
<data name="Archive.Trash.KeptFiles.other" xml:space="preserve"><value>{0:N0} file sono rimasti nel cestino: non corrispondono più al loro registro.</value></data>
<data name="Archive.Trash.Kind.Photos" xml:space="preserve"><value>Foto</value></data>
<data name="Archive.Trash.Kind.Replaced" xml:space="preserve"><value>Originali sostituiti</value></data>
<data name="Archive.Trash.Kind.Sequence" xml:space="preserve"><value>Sequenza</value></data>
<data name="Archive.Trash.List" xml:space="preserve"><value>Elementi del cestino</value></data>
<data name="Archive.Trash.Reading" xml:space="preserve"><value>Lettura del cestino…</value></data>
<data name="Archive.Trash.Restore" xml:space="preserve"><value>Ripristina</value></data>
<data name="Archive.Trash.Restored.one" xml:space="preserve"><value>{0:N0} sequenza ripristinata</value></data>
+1
View File
@@ -72,6 +72,7 @@
<data name="Jobs.Kind.DeepAnalyse" xml:space="preserve"><value>Analisi approfondita</value></data>
<data name="Jobs.Kind.Download" xml:space="preserve"><value>Scaricamento</value></data>
<data name="Jobs.Kind.Duplicates" xml:space="preserve"><value>Duplicati</value></data>
<data name="Jobs.Kind.EmptyTrash" xml:space="preserve"><value>Svuota cestino</value></data>
<data name="Jobs.Kind.Import" xml:space="preserve"><value>Importazione</value></data>
<data name="Jobs.Kind.LightAnalyse" xml:space="preserve"><value>Analisi rapida</value></data>
<data name="Jobs.Kind.Maintenance" xml:space="preserve"><value>Manutenzione</value></data>
+4
View File
@@ -85,12 +85,15 @@
<data name="Server.Error.ScheduleSystem" xml:space="preserve"><value>Una pianificazione di sistema non si può eliminare; disattivala.</value><comment>long</comment></data>
<data name="Server.Error.Self" xml:space="preserve"><value>Non puoi farlo sul tuo account.</value></data>
<data name="Server.Error.SequenceBusy" xml:space="preserve"><value>Un'attività sta lavorando su questa sequenza o sul suo archivio; riprova quando è libera.</value></data>
<data name="Server.Error.SequenceInLibrary" xml:space="preserve"><value>Quella sequenza è già nella libreria.</value></data>
<data name="Server.Error.SessionExpired" xml:space="preserve"><value>La sessione è scaduta; accedi di nuovo.</value></data>
<data name="Server.Error.SettingLocked" xml:space="preserve"><value>È impostato da una variabile d'ambiente; modificala lì.</value></data>
<data name="Server.Error.SettingsConflict" xml:space="preserve"><value>La sequenza è cambiata sul server; ricaricala e riapplica la modifica.</value></data>
<data name="Server.Error.SidecarMismatch" xml:space="preserve"><value>La sua cartella contiene un'altra sequenza; prima esegui una scansione dell'archivio.</value></data>
<data name="Server.Error.TooLarge" xml:space="preserve"><value>La richiesta supera la dimensione accettata dal server.</value></data>
<data name="Server.Error.TooManyUploads" xml:space="preserve"><value>Troppi caricamenti insieme; riprova tra poco.</value></data>
<data name="Server.Error.TrashChanged" xml:space="preserve"><value>Il cestino è cambiato dopo il controllo; ricontrollalo prima di svuotarlo.</value></data>
<data name="Server.Error.TrashMismatch" xml:space="preserve"><value>Nessun file di questo elemento del cestino corrisponde più al suo registro; resta nel cestino.</value></data>
<data name="Server.Error.Unauthorized" xml:space="preserve"><value>Accedi per continuare.</value></data>
<data name="Server.Error.Unreachable" xml:space="preserve"><value>Il server non risponde; controlla l'indirizzo e la rete.</value></data>
<data name="Server.Error.UnsupportedMediaType" xml:space="preserve"><value>Invia il corpo come JSON.</value></data>
@@ -99,6 +102,7 @@
<data name="Server.Error.UsernameTaken" xml:space="preserve"><value>Questo nome utente è già in uso.</value></data>
<data name="Server.Error.WeakPassword" xml:space="preserve"><value>Usa almeno {0} caratteri.</value></data>
<data name="Server.Error.WebsocketRequired" xml:space="preserve"><value>Connettiti con un WebSocket.</value></data>
<data name="Server.Error.WouldEmpty" xml:space="preserve"><value>Se ne andrebbero tutte le foto della sequenza; elimina invece la sequenza.</value></data>
<data name="Server.Error.WrongPassword" xml:space="preserve"><value>La password attuale è errata.</value></data>
<data name="Server.Error.WrongUploadKind" xml:space="preserve"><value>Questo caricamento appartiene a un job di rendering.</value></data>
<data name="Server.Found" xml:space="preserve"><value>Server trovati</value></data>
+6 -2
View File
@@ -91,8 +91,12 @@
<data name="Tooltips.Archive.TagsAdd" xml:space="preserve"><value>Tag da aggiungere, separati da virgole</value></data>
<data name="Tooltips.Archive.TagsRemove" xml:space="preserve"><value>Tag da rimuovere, separati da virgole</value></data>
<data name="Tooltips.Archive.TitleTemplate" xml:space="preserve"><value>Modello del titolo; il segnaposto title è il titolo attuale · controllato mentre scrivi</value></data>
<data name="Tooltips.Archive.Trash" xml:space="preserve"><value>Le sequenze eliminate da questo archivio, ognuna ripristinabile</value></data>
<data name="Tooltips.Archive.TrashRestore" xml:space="preserve"><value>Riporta questa sequenza nella sua cartella e nella libreria, con impostazioni e render</value></data>
<data name="Tooltips.Archive.Trash" xml:space="preserve"><value>Le sequenze e le foto eliminate da questo archivio, ognuna ripristinabile</value></data>
<data name="Tooltips.Archive.TrashEmpty" xml:space="preserve"><value>Cancella per sempre gli elementi di questo cestino, dopo un controllo di cosa contengono e una conferma digitata</value></data>
<data name="Tooltips.Archive.TrashEmptyPlan" xml:space="preserve"><value>Cosa viene cancellato per sempre dall'archivio</value></data>
<data name="Tooltips.Archive.TrashErase.one" xml:space="preserve"><value>Cancella per sempre {0:N0} elemento; prima digita il numero qui sopra</value></data>
<data name="Tooltips.Archive.TrashErase.other" xml:space="preserve"><value>Cancella per sempre {0:N0} elementi; prima digita il numero qui sopra</value></data>
<data name="Tooltips.Archive.TrashRestore" xml:space="preserve"><value>Riporta indietro questo elemento: una sequenza nella sua cartella e nella libreria, le foto nella loro sequenza dopo un controllo dell'hash</value></data>
<data name="Tooltips.Archive.Verify" xml:space="preserve"><value>Ricalcola l'hash di ogni originale e segnala i danni · legge tutto l'archivio</value></data>
<data name="Tooltips.Archive.YearRow" xml:space="preserve"><value>{0} · Freccia destra apre l'anno, freccia sinistra lo chiude</value></data>
<data name="Tooltips.Archive.YearRowRtl" xml:space="preserve"><value>{0} · Freccia sinistra apre l'anno, freccia destra lo chiude</value></data>
@@ -242,6 +242,9 @@
<DockPanel Grid.Row="3" Margin="0,16,0,0">
<Button DockPanel.Dock="Right" Content="{l:Loc Archive.Dialog.Close}" Command="{Binding CloseCommand}" ToolTip="{Binding CloseToolTip}" IsCancel="True"
IsDefault="True" MinWidth="96" />
<!-- Erasing for good: a dry run and a typed confirmation first, never the default button. -->
<Button DockPanel.Dock="Right" Content="{l:Loc Archive.Trash.EmptyAction}" Command="{Binding EmptyCommand}" ToolTip="{Binding EmptyToolTip}"
ToolTipService.ShowOnDisabled="True" MinWidth="96" Margin="0,0,8,0" />
<Grid Margin="0,0,12,0">
<TextBlock Style="{StaticResource Text.Live}" Text="{Binding BusyText}" infra:LiveRegion.Text="{Binding BusyText}" />
<ui:StatusGlyph Style="{StaticResource Status.DialogError}" />
+8 -3
View File
@@ -32,14 +32,19 @@ public static class SampleReports
? new OrphanReport([], [], [])
: new OrphanReport([new SequenceOrphans(sequences[2].Folder, ["IMG_0412.CR2", "IMG_0413.CR2"], [])], [sequences[3]], ["2024/2024-12-31 Unsorted"]);
/// <summary>The trash dialog's rows: the first two sample sequences, deleted an hour and a day ago (nothing moves on disk).</summary>
/// <summary>
/// The trash dialog's rows: the first sample sequence, deleted an hour ago, and 24 photos of the second, deleted a day ago
/// (nothing moves on disk).
/// </summary>
public static IReadOnlyList<TrashEntry> Trash(Guid archiveId, IReadOnlyList<SequenceLocation> sequences)
{
var now = SampleData.Now;
return
[
.. sequences.Take(2).Select((s, i) => new TrashEntry(archiveId, s.Id.ToString("N"), s.Id, s.Title ?? s.Folder, s.Folder, 1_200 * (i + 1), 1_206 * (i + 1),
31_457_280L * 1_200 * (i + 1), now.AddHours(-1 - (23 * i)), "alberto")),
.. sequences.Take(1).Select(s => new TrashEntry(archiveId, s.Id.ToString("N"), s.Id, s.Title ?? s.Folder, s.Folder, 1_200, 1_206,
31_457_280L * 1_200, now.AddHours(-1), "alberto")),
.. sequences.Skip(1).Take(1).Select(s => new TrashEntry(archiveId, FrameTrash.EntryPrefix + s.Id.ToString("N") + "-20260927180400", s.Id,
s.Title ?? s.Folder, s.Folder, 24, 24, 31_457_280L * 24, now.AddHours(-24), "alberto", TrashKind.Photos)),
];
}
+119 -12
View File
@@ -1,3 +1,4 @@
using System.Globalization;
using System.IO;
using System.Windows.Input;
using Chronos.App.Infrastructure;
@@ -6,18 +7,24 @@ using Chronos.Core.Diagnostics;
using Chronos.Core.Localization;
using Chronos.Data;
using Chronos.Data.Archiving;
using Chronos.Data.Archiving.Jobs;
using Chronos.Data.Jobs;
using Chronos.Ui.Mvvm;
namespace Chronos.App.Archiving;
/// <summary>One deleted sequence in the trash: what it was, when it went, and Restore.</summary>
/// <summary>One entry of the trash: a deleted sequence or photos deleted from one, when it went, and Restore.</summary>
public sealed class TrashRowViewModel(TrashEntry entry, ICommand restore)
{
public TrashEntry Entry { get; } = entry;
public string Title => Entry.Title;
/// <summary>"3,600 photos · 96.2 GB · 2026/2026-09-28 Canon EOS R5" (the folder left to right in every language).</summary>
public string Detail => string.Join(Loc.Separator, Loc.Plural("Shell.Status.Photos", Entry.Frames), Formatting.Bytes(Entry.Bytes), Formatting.LeftToRight(Entry.Folder));
/// <summary>"Photos", "Sequence": what the entry holds.</summary>
public string KindText => TrashEmptyPlans.KindText(Entry.Kind);
/// <summary>"Photos · 12 photos · 340 MB · 2026/2026-09-28 Canon EOS R5" (the folder left to right in every language).</summary>
public string Detail => string.Join(Loc.Separator, KindText, Loc.Plural("Shell.Status.Photos", Entry.Frames), Formatting.Bytes(Entry.Bytes),
Formatting.LeftToRight(Entry.Folder));
/// <summary>"Deleted 02/10/2026 18:04".</summary>
public string DeletedText => Loc.L("Archive.Trash.Deleted", Formatting.Date(Entry.DeletedAt) + " " + Formatting.TimeOfDay(Entry.DeletedAt));
@@ -25,23 +32,71 @@ public sealed class TrashRowViewModel(TrashEntry entry, ICommand restore)
public ICommand RestoreCommand { get; } = restore;
}
/// <summary>The words of an emptying of the trash (<see cref="TrashEmptyPlan"/>): the destructive, typed confirmation.</summary>
public static class TrashEmptyPlans
{
/// <summary>"Photos", "Sequence", "Replaced".</summary>
public static string KindText(TrashKind kind) => Loc.L("Archive.Trash.Kind." + kind);
/// <summary>The confirmation of erasing the planned entries for good; null when the plan holds none.</summary>
public static ConfirmRequest? Request(TrashEmptyPlan plan)
{
ArgumentNullException.ThrowIfNull(plan);
if (plan.Entries.Count == 0) return null;
var count = plan.Entries.Count.ToString(CultureInfo.InvariantCulture);
var lines = plan.Entries.Take(DeletePlans.ListedLines).Select(e => Loc.L("Archive.Delete.Line", e.Title,
KindText(e.Kind) + Loc.Separator + Loc.Plural("Shell.Status.Photos", e.Frames), Formatting.Bytes(e.Bytes))).ToList();
if (plan.Entries.Count > DeletePlans.ListedLines) lines.Add(Loc.Plural("Archive.Delete.More", plan.Entries.Count - DeletePlans.ListedLines));
return new ConfirmRequest
{
Heading = Loc.Plural("Archive.Trash.EmptyHeading", plan.Entries.Count),
Message = Loc.L("Archive.Trash.EmptyMessage", Loc.Plural("Shell.Status.Photos", plan.Photos), Loc.Plural("Archive.Delete.Files", plan.Files),
Formatting.Bytes(plan.Bytes)),
Detail = string.Join(Environment.NewLine, lines),
IsDetailMonospaced = false,
DetailToolTip = Loc.L("Tooltips.Archive.TrashEmptyPlan"),
SafeText = Loc.L("Archive.Delete.Keep"),
SafeToolTip = Loc.L("Tooltips.Archive.DeleteKeep"),
ActionText = Loc.L("Archive.Trash.Erase"),
ActionToolTip = Loc.Plural("Tooltips.Archive.TrashErase", plan.Entries.Count),
IsDestructive = true,
TypedConfirmation = count,
TypedPrompt = Loc.L("Archive.Delete.Type", count),
};
}
}
/// <summary>
/// The archive's trash (BRIEF §0.2 rule 8): every sequence deleted from the Library or the Archive manager, newest
/// first, each restorable into its folder (or a free one next to it) with its ids, settings and renders.
/// The archive's trash (BRIEF §0.2 rule 8, §12.5): every sequence deleted from the Library or the Archive manager and the
/// photos deleted from sequences (owner review 2, R6), newest first, each restorable — a sequence into its folder (or a
/// free one next to it) with its ids, settings and renders, photos into their sequence after their files are re-hashed.
/// Empty… erases the entries for good: a dry run, a typed confirmation, then the <c>EmptyTrash</c> job, which refuses when
/// the trash changed since.
/// </summary>
public sealed class TrashDialogViewModel : ArchiveDialogViewModel
{
private readonly ArchiveService _archives;
private readonly Core.Model.Archive _archive;
private readonly Func<Task> _restored;
private readonly Func<ConfirmRequest, bool>? _confirm;
private readonly Func<ArchiveJobRequest, Task>? _enqueue;
private readonly JobEngine? _followUps;
private IReadOnlyList<TrashRowViewModel> _rows = [];
public TrashDialogViewModel(ArchiveService archives, Core.Model.Archive archive, Func<Task> restored, Action<ArchiveDialogViewModel> close)
/// <param name="confirm">Asks the typed confirmation of an emptying (null: the trash cannot be emptied here).</param>
/// <param name="enqueue">Queues the emptying job and says so (null: the trash cannot be emptied here).</param>
/// <param name="followUps">The engine that takes the analyses and images restored photos need again.</param>
public TrashDialogViewModel(ArchiveService archives, Core.Model.Archive archive, Func<Task> restored, Action<ArchiveDialogViewModel> close,
Func<ConfirmRequest, bool>? confirm = null, Func<ArchiveJobRequest, Task>? enqueue = null, JobEngine? followUps = null)
: base(close)
{
_archives = archives;
_archive = archive;
_restored = restored;
_confirm = confirm;
_enqueue = enqueue;
_followUps = followUps;
EmptyCommand = new AsyncRelayCommand(EmptyAsync, () => CanEmpty);
}
public string Heading => Loc.L("Archive.Trash.Heading", _archive.Name);
@@ -55,6 +110,8 @@ public sealed class TrashDialogViewModel : ArchiveDialogViewModel
OnPropertyChanged(nameof(IsEmpty));
OnPropertyChanged(nameof(HasRows));
OnPropertyChanged(nameof(SummaryText));
OnPropertyChanged(nameof(EmptyToolTip));
CommandManager.InvalidateRequerySuggested();
}
}
@@ -63,9 +120,17 @@ public sealed class TrashDialogViewModel : ArchiveDialogViewModel
/// <summary>The list shows; an empty trash is its summary sentence alone.</summary>
public bool HasRows => _rows.Count > 0;
/// <summary>"3 sequences · 182 GB", or "The trash is empty".</summary>
/// <summary>"3 entries · 182 GB", or "The trash is empty".</summary>
public string SummaryText => _rows.Count == 0 ? Loc.L("Archive.Trash.Empty")
: Loc.Plural("Shell.Status.Sequences", _rows.Count) + Loc.Separator + Formatting.Bytes(_rows.Sum(r => r.Entry.Bytes));
: Loc.Plural("Archive.Trash.Entries", _rows.Count) + Loc.Separator + Formatting.Bytes(_rows.Sum(r => r.Entry.Bytes));
/// <summary>Empty…: the dry run, the typed confirmation, then the job.</summary>
public ICommand EmptyCommand { get; }
/// <summary>Emptying is offered here, and there is something to erase.</summary>
public bool CanEmpty => !IsBusy && HasRows && _confirm is not null && _enqueue is not null;
public string EmptyToolTip => HasRows ? Loc.L("Tooltips.Archive.TrashEmpty") : Loc.L("Archive.Trash.Empty");
/// <summary>Reads the trash of the archive (off the UI thread).</summary>
public async Task LoadAsync()
@@ -75,7 +140,7 @@ public sealed class TrashDialogViewModel : ArchiveDialogViewModel
try
{
var entries = await Task.Run(() => _archives.ListTrashAsync(_archive.Id));
Rows = [.. entries.Select(e => new TrashRowViewModel(e, new AsyncRelayCommand(() => RestoreAsync(e), () => !IsBusy)))];
Rows = [.. entries.Select(Row)];
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or DatabaseException or ArgumentException)
{
@@ -92,6 +157,8 @@ public sealed class TrashDialogViewModel : ArchiveDialogViewModel
public void Show(IReadOnlyList<TrashEntry> entries) =>
Rows = [.. entries.Select(e => new TrashRowViewModel(e, new AsyncRelayCommand(() => Task.CompletedTask)))];
private TrashRowViewModel Row(TrashEntry entry) => new(entry, new AsyncRelayCommand(() => RestoreAsync(entry), () => !IsBusy));
private async Task RestoreAsync(TrashEntry entry)
{
ClearError();
@@ -99,10 +166,19 @@ public sealed class TrashDialogViewModel : ArchiveDialogViewModel
BusyText = Loc.L("Archive.Trash.Restoring", entry.Title);
try
{
await Task.Run(() => _archives.RestoreAsync(entry.ArchiveId, entry.Name));
Log.Info("archive", $"{entry.Title} restored from the trash");
Rows = [.. _rows.Where(r => r.Entry != entry)];
var result = await Task.Run(() => _archives.RestoreEntryAsync(entry.ArchiveId, entry.Name, _followUps));
Log.Info("archive", $"{entry.Title} restored from the trash ({entry.Kind}, {result.Photos} photo(s), {result.Kept.Count} kept)");
BusyText = null;
if (result.Kept.Count > 0)
{
// Files that no longer match their record stay in the trash: the entry stays, with what is left.
await LoadAsync();
ShowError(new InvalidDataException(Loc.Plural("Archive.Trash.KeptFiles", result.Kept.Count)));
}
else
{
Rows = [.. _rows.Where(r => r.Entry != entry)];
}
await _restored();
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or InvalidDataException or SequenceInLibraryException or NewerFormatException
@@ -117,4 +193,35 @@ public sealed class TrashDialogViewModel : ArchiveDialogViewModel
BusyText = null;
}
}
/// <summary>
/// Empty…: measures every entry (the dry run), asks to type how many entries go, and queues the job that erases exactly
/// those; the dialog closes, and the job says how it ended.
/// </summary>
private async Task EmptyAsync()
{
if (_confirm is null || _enqueue is null) return;
ClearError();
IsBusy = true;
BusyText = Loc.L("Archive.Trash.Checking");
TrashEmptyPlan plan;
try
{
plan = await Task.Run(() => _archives.PlanEmptyTrashAsync(_archive.Id));
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or DatabaseException or ArgumentException)
{
Log.Warning("archive", "The dry run of emptying the trash failed", ex);
ShowError(ex);
return;
}
finally
{
IsBusy = false;
BusyText = null;
}
if (TrashEmptyPlans.Request(plan) is not { } request || !_confirm(request)) return;
await _enqueue(ArchiveJobs.EmptyTrash(_archive, plan));
Close();
}
}
+1 -1
View File
@@ -40,7 +40,7 @@ public static class JobNotices
private static bool Announces(JobKind kind) =>
kind is JobKind.Render or JobKind.DeepAnalyse or JobKind.Verify or JobKind.Rebuild or JobKind.Maintenance or JobKind.UploadCommit
or JobKind.Move or JobKind.Adopt;
or JobKind.Move or JobKind.Adopt or JobKind.EmptyTrash;
/// <summary>
/// Work that keeps the index and its previews current: the next run repeats it, the Archive screen shows what is missing.
+6 -1
View File
@@ -782,11 +782,16 @@ public sealed class ArchiveViewModel : SectionViewModel
public async Task OpenTrashAsync()
{
if (Services.Archives is not { } archives || _archive is not { } archive) return;
var engine = Services.Engine;
var dialog = new TrashDialogViewModel(archives, archive, async () =>
{
await LoadAsync();
await Shell.Library.RefreshAsync();
}, CloseDialog);
}, CloseDialog, Services.Dialogs.Confirm, engine is null ? null : async request =>
{
await engine.EnqueueAsync(request);
Queued(request.Kind, archive.Name);
}, engine);
OpenDialog(dialog);
await dialog.LoadAsync();
}
@@ -1130,7 +1130,7 @@ public sealed class SettingsViewModel : SectionViewModel
[
JobKind.Render, JobKind.Import, JobKind.Download, JobKind.UploadCommit, JobKind.Scan, JobKind.Rebuild, JobKind.Verify, JobKind.LightAnalyse,
JobKind.DeepAnalyse, JobKind.Thumbs, JobKind.Proxies, JobKind.Maintenance, JobKind.Duplicates, JobKind.Orphans, JobKind.Adopt, JobKind.Move,
JobKind.Backup, JobKind.Cleanup,
JobKind.Backup, JobKind.Cleanup, JobKind.EmptyTrash,
];
/// <summary>Stores <paramref name="level"/> for <paramref name="kind"/>, keeping only the levels that differ from the defaults.</summary>
@@ -14,6 +14,7 @@ namespace Chronos.Core.Archiving;
[JsonSerializable(typeof(SequenceManifest))]
[JsonSerializable(typeof(LeaseRecord))]
[JsonSerializable(typeof(IngestJournal))]
[JsonSerializable(typeof(FrameRemovalJournal))]
[JsonSerializable(typeof(RenameJournal))]
internal sealed partial class ArchiveJson : JsonSerializerContext;
@@ -30,6 +30,9 @@ public static class ArchiveLayout
/// <summary>Moves of an ingest batch that were started but not yet recorded in <see cref="SequenceManifestFile"/>.</summary>
public const string IngestJournalFile = "ingest.pending.json";
/// <summary>Moves of a photo deletion into the trash that were started but not yet recorded in <see cref="SequenceManifestFile"/>.</summary>
public const string RemovalJournalFile = "remove.pending.json";
/// <summary>Folder names inside a sequence folder that never hold a sequence of their own.</summary>
public static bool IsReservedFolder(string name) =>
name.Equals(MetaFolder, StringComparison.OrdinalIgnoreCase)
@@ -74,11 +77,15 @@ public sealed class SequencePaths
internal SequencePaths(ArchivePaths archive, string relativeFolder)
{
ArgumentNullException.ThrowIfNull(relativeFolder);
Archive = archive;
RelativeFolder = relativeFolder.Replace('\\', '/').Trim('/');
Folder = archive.FromRelative(RelativeFolder);
MetaDirectory = Path.Combine(Folder, ArchiveLayout.MetaFolder);
}
/// <summary>The archive the sequence belongs to.</summary>
public ArchivePaths Archive { get; }
/// <summary>Folder relative to the archive root, forward slashes; empty when the root itself holds the photos (adopted folder).</summary>
public string RelativeFolder { get; }
@@ -92,6 +99,7 @@ public sealed class SequencePaths
public string ProjectPath => Path.Combine(MetaDirectory, ArchiveLayout.ProjectFile);
public string ThumbsPackPath => Path.Combine(MetaDirectory, ArchiveLayout.ThumbsPackFile);
public string IngestJournalPath => Path.Combine(MetaDirectory, ArchiveLayout.IngestJournalFile);
public string RemovalJournalPath => Path.Combine(MetaDirectory, ArchiveLayout.RemovalJournalFile);
public string Proxy(int index) => Path.Combine(Proxies, ArchiveLayout.ProxyFileName(index));
@@ -0,0 +1,94 @@
using System.Text.Json;
using Chronos.Core.Diagnostics;
using Chronos.Core.IO;
namespace Chronos.Core.Archiving;
/// <summary>One file of a photo deletion: where it was in the sequence folder and where it went in the archive's trash.</summary>
/// <param name="From">Path relative to the sequence folder (forward slashes).</param>
/// <param name="To">Path relative to the archive root (forward slashes), inside the trash entry.</param>
public sealed record FrameRemovalMove(string From, string To);
/// <summary>
/// Write-ahead record of a photo deletion (<c>.chronos/remove.pending.json</c>, owner review 2, R6): the trash entry the
/// photos move into, the frames leaving the sidecar and every file move. Written before the first move and removed once
/// <c>sequence.json</c> no longer lists the frames, so a crash in between is undone by <see cref="Recover"/>: the files go
/// back to their places, the entry's record and folder go, and the sequence is as it was.
/// </summary>
/// <param name="Record">The entry's record, relative to the archive root.</param>
/// <param name="Folder">The entry's folder, relative to the archive root.</param>
public sealed record FrameRemovalJournal(string Record, string Folder, IReadOnlyList<Guid> Frames, IReadOnlyList<FrameRemovalMove> Moves)
{
public static void Write(SequencePaths sequence, FrameRemovalJournal journal)
{
ArgumentNullException.ThrowIfNull(sequence);
Directory.CreateDirectory(sequence.MetaDirectory);
AtomicFile.WriteAllBytes(sequence.RemovalJournalPath, JsonSerializer.SerializeToUtf8Bytes(journal, ArchiveJson.Default.FrameRemovalJournal));
}
public static void Delete(SequencePaths sequence)
{
ArgumentNullException.ThrowIfNull(sequence);
if (File.Exists(sequence.RemovalJournalPath)) File.Delete(sequence.RemovalJournalPath);
}
/// <summary>
/// Undoes a deletion a crash interrupted before its sidecar was written: while <paramref name="manifest"/> still lists a
/// frame of the journal, each file that reached the trash moves back (one whose place is taken again stays in the trash,
/// logged, and so does the entry then), and the entry's record and its emptied folder go. A deletion whose sidecar was
/// written is complete: only the journal goes. Returns whether a deletion was undone. The caller holds the lease.
/// </summary>
public static bool Recover(SequencePaths sequence, SequenceManifest manifest)
{
ArgumentNullException.ThrowIfNull(sequence);
ArgumentNullException.ThrowIfNull(manifest);
if (!File.Exists(sequence.RemovalJournalPath)) return false;
FrameRemovalJournal? journal;
try
{
journal = JsonSerializer.Deserialize(AtomicFile.ReadAllBytes(sequence.RemovalJournalPath), ArchiveJson.Default.FrameRemovalJournal);
}
catch (JsonException)
{
journal = null; // torn while being written: nothing was moved yet
}
var undone = false;
if (journal is not null && manifest.Frames.Any(f => journal.Frames.Contains(f.Id)))
{
var archive = sequence.Archive;
var kept = 0;
foreach (var move in journal.Moves)
{
var trashed = archive.FromRelative(move.To);
var place = sequence.File(move.From);
if (!File.Exists(trashed)) continue;
if (File.Exists(place))
{
kept++;
Log.Warning("archive", $"{sequence.ArchiveRelative(move.From)} is there again; its deleted copy stays in {move.To}");
continue;
}
Directory.CreateDirectory(Path.GetDirectoryName(place)!);
File.Move(trashed, place, overwrite: false);
}
if (kept == 0)
{
var record = archive.FromRelative(journal.Record);
if (File.Exists(record)) File.Delete(record);
RemoveEmpty(archive.FromRelative(journal.Folder));
}
Log.Warning("archive", $"An interrupted photo deletion in {sequence.RelativeFolder} was undone");
undone = true;
}
Delete(sequence);
return undone;
}
/// <summary>Removes <paramref name="folder"/> and its subfolders when they hold no file.</summary>
public static void RemoveEmpty(string folder)
{
if (!Directory.Exists(folder)) return;
foreach (var child in Directory.EnumerateDirectories(folder)) RemoveEmpty(child);
if (!Directory.EnumerateFileSystemEntries(folder).Any()) Directory.Delete(folder);
}
}
@@ -42,6 +42,13 @@ public sealed record SequenceManifest
/// <summary>Session keys of automatic grouping (uploads, card imports) that resolve to this sequence.</summary>
public IReadOnlyList<string> SessionKeys { get; init; } = [];
/// <summary>
/// Tombstones of the photos removed from the sequence (owner review 2, R6), by SHA-256; null when there are none, so a
/// sidecar without removals is written as before. A build that does not know the field keeps it in its
/// <see cref="Extra"/> and writes it back unchanged.
/// </summary>
public IReadOnlyList<RemovedPhoto>? Removed { get; init; }
[JsonExtensionData]
public Dictionary<string, JsonElement>? Extra { get; set; }
}
@@ -127,6 +134,53 @@ public sealed record SidecarFrame
/// <summary><see cref="FrameFlags"/> bits, stored as a number so bits of newer builds survive.</summary>
public FrameFlags Flags { get; init; }
/// <summary>
/// The file this frame was converted from (reserved for the conversion of owner review 2, R7); null for an original.
/// Read and written only: nothing acts on it yet.
/// </summary>
public FrameSource? Source { get; init; }
[JsonExtensionData]
public Dictionary<string, JsonElement>? Extra { get; set; }
}
/// <summary>
/// The tombstone of a photo removed from a sequence (owner review 2, R6): sync never offers that content back to the
/// sequence, and an upload of it is answered as already there. <see cref="Replacement"/> names the converted file that took
/// its place (<see cref="RemovalReason.Replaced"/>), which sync then counts as the same photo.
/// </summary>
public sealed record RemovedPhoto
{
/// <summary>Lower-case hex SHA-256 of the removed photo.</summary>
public required string Sha256 { get; init; }
/// <summary>Lower-case hex SHA-256 of the file that replaced it; null for a photo deleted outright.</summary>
public string? Replacement { get; init; }
[JsonConverter(typeof(TolerantEnumConverter<RemovalReason>))]
public RemovalReason Reason { get; init; }
/// <summary>When it was removed (UTC).</summary>
public DateTimeOffset At { get; init; }
[JsonExtensionData]
public Dictionary<string, JsonElement>? Extra { get; set; }
}
/// <summary>The original a converted frame was made from (reserved for owner review 2, R7): its content, format, file and size.</summary>
public sealed record FrameSource
{
/// <summary>Lower-case hex SHA-256 of the source file.</summary>
public string? Sha256 { get; init; }
[JsonConverter(typeof(TolerantEnumConverter<ImageFileFormat>))]
public ImageFileFormat Format { get; init; }
/// <summary>The source's path relative to the sequence folder (forward slashes) when it was converted.</summary>
public string? File { get; init; }
public long Size { get; init; }
[JsonExtensionData]
public Dictionary<string, JsonElement>? Extra { get; set; }
}
+7
View File
@@ -17,6 +17,13 @@ public sealed record Archive
public int ProxyLongEdge { get; init; } = 1920;
}
/// <summary>
/// The tombstone of a photo removed from a sequence (owner review 2, R6): its SHA-256, the replacement's when it was
/// replaced by a converted file, why and when. Kept in the sidecar (<c>removed</c>) and the index (<c>frame_removals</c>), so
/// sync never offers the photo back and an upload of that content into the sequence is answered as already there.
/// </summary>
public sealed record FrameRemoval(Guid SequenceId, string Sha256, string? ReplacementSha256, RemovalReason Reason, DateTimeOffset RemovedAt);
/// <summary>One shoot: a folder of frames that becomes one project (BRIEF §5).</summary>
public sealed record Sequence
{
+16
View File
@@ -87,6 +87,22 @@ public enum JobKind
/// <summary>Housekeeping steps: job history, audit log, trash report and the host's own (owner review 2, R10); stored as 17.</summary>
Cleanup = 17,
/// <summary>
/// Erases exactly the trash entries of a confirmed plan, refused when the trash changed since (owner review 2, R6); stored
/// as 18.
/// </summary>
EmptyTrash = 18,
}
/// <summary>Why a photo left a sequence (its tombstone, owner review 2, R6); stored as a number.</summary>
public enum RemovalReason
{
/// <summary>Deleted by a user into the archive's trash.</summary>
Deleted = 0,
/// <summary>Replaced by a converted file (a later phase): the tombstone names the replacement's SHA-256.</summary>
Replaced = 1,
}
public enum JobState
@@ -152,8 +152,11 @@ public sealed class ArchiveIndexer
if (positions is not null) existing = positions.ApplyTo(existing);
}
var row = SequenceMapping.ToSequence(manifest, archive.Id, sequence, existing);
// A hero frame the sidecar no longer lists (a photo deleted) names nothing: the project's hero waits for its photo.
if (row.HeroFrameId is { } hero && !manifest.Frames.Any(f => f.Id == hero)) row = row with { HeroFrameId = null };
await tx.Sequences.UpsertAsync(row with { Status = row.Status & ~clear }, cancellationToken).ConfigureAwait(false);
await tx.Frames.SyncSequenceAsync(manifest.Id, SequenceMapping.ToFrames(manifest, sequence), cancellationToken).ConfigureAwait(false);
await tx.FrameRemovals.SyncSequenceAsync(manifest.Id, SequenceMapping.ToRemovals(manifest), cancellationToken).ConfigureAwait(false);
foreach (var key in manifest.SessionKeys)
await tx.Sequences.MapSessionKeyAsync(archive.Id, key, manifest.Id, cancellationToken).ConfigureAwait(false);
await tx.CommitAsync(cancellationToken).ConfigureAwait(false);
+21 -1
View File
@@ -358,13 +358,17 @@ public sealed class ArchiveIngest : IArchiveIngest
updated = SequenceFrames.Update(updated, repaired.ToDictionary(id => id,
_ => (Func<SidecarFrame, SidecarFrame>)(f => f with { Flags = f.Flags & ~(FrameFlags.Missing | FrameFlags.Corrupt) })));
}
// A photo deleted from the sequence and filed into it again (a card imported again, a sync download) is back: its
// tombstone goes, so sync offers it like any other.
var back = Readded(updated, moved);
if (back) updated = updated with { Removed = Remaining(updated.Removed!, moved) };
if (moved.Count > 0 || repaired.Count > 0) ManifestStore.WriteSequence(sequence.ManifestPath, updated);
IngestJournal.Delete(sequence);
var byId = updated.Frames.ToDictionary(f => f.Id);
if (moved.Count > 0 || repaired.Count > 0)
{
await IndexAsync(archive, sequence, updated, inStep, photosChanged, before, moved.ConvertAll(m => byId[m.Id]), repaired.Count > 0,
await IndexAsync(archive, sequence, updated, inStep, photosChanged, before, moved.ConvertAll(m => byId[m.Id]), repaired.Count > 0 || back,
cancellationToken).ConfigureAwait(false);
}
foreach (var (p, frame) in accepted.Where(a => byId.ContainsKey(a.Frame.Id)))
@@ -382,6 +386,22 @@ public sealed class ArchiveIngest : IArchiveIngest
}
}
/// <summary>Whether a frame filed by the batch holds content the sequence's tombstones name.</summary>
private static bool Readded(SequenceManifest manifest, List<SidecarFrame> moved)
{
if (manifest.Removed is not { Count: > 0 } removed || moved.Count == 0) return false;
var tombstoned = removed.Select(r => r.Sha256).ToHashSet(StringComparer.OrdinalIgnoreCase);
return moved.Any(f => f.Sha256 is { } sha && tombstoned.Contains(sha));
}
/// <summary>The tombstones without those of the content filed again; null when none is left.</summary>
private static IReadOnlyList<RemovedPhoto>? Remaining(IReadOnlyList<RemovedPhoto> removed, List<SidecarFrame> moved)
{
var filed = moved.Where(f => f.Sha256 is not null).Select(f => f.Sha256!).ToHashSet(StringComparer.OrdinalIgnoreCase);
var left = removed.Where(r => !filed.Contains(r.Sha256)).ToList();
return left.Count == 0 ? null : left;
}
/// <summary>Final file names, in capture order: the plan's name or the template with <c>{n}</c> = index + 1, made unique.</summary>
private static Dictionary<Guid, SidecarFrame> Name(Archive archive, SequencePaths sequence, SequenceManifest manifest,
List<(Pending Pending, SidecarFrame Frame)> accepted)
+50 -4
View File
@@ -40,6 +40,7 @@ public sealed class ArchiveService : IJobHandlerProvider
Images = new ArchiveImages(db, _writer, Indexer);
Imports = new ImportService(db, Ingest);
Trash = new SequenceTrash(db, _writer, Indexer);
FrameTrash = new FrameTrash(db, _writer, Indexer, Ingest, Trash);
var shared = Housekeeping.CleanupSteps.Shared(Trash);
CleanupSteps = [.. shared, .. _writer.Options.CleanupSteps.Where(s => !shared.Any(o => o.Name == s.Name))];
}
@@ -55,9 +56,12 @@ public sealed class ArchiveService : IJobHandlerProvider
public ArchiveImages Images { get; }
public ImportService Imports { get; }
/// <summary>Deletes sequences into the archive's trash and restores them.</summary>
/// <summary>Deletes sequences into the archive's trash and restores them; lists and empties the trash.</summary>
public SequenceTrash Trash { get; }
/// <summary>Deletes photos of a sequence into the archive's trash and restores them (owner review 2, R6).</summary>
public FrameTrash FrameTrash { get; }
/// <summary>
/// The steps this host's <see cref="JobKind.Cleanup"/> jobs can run: the shared ones (<see cref="Housekeeping.CleanupSteps.Shared"/>),
/// then the host's (<see cref="ArchiveServiceOptions.CleanupSteps"/>). A schedule host lists their names.
@@ -69,7 +73,8 @@ public sealed class ArchiveService : IJobHandlerProvider
/// <summary>
/// Handlers for scan, rebuild, verify, import, thumbs, proxies and maintenance jobs, the read-only duplicates and orphans
/// reports, index backups and clean-ups (owner review 2, R10); register them on the <see cref="JobEngine"/>.
/// reports, index backups and clean-ups (owner review 2, R10) and the emptying of the trash (R6); register them on the
/// <see cref="JobEngine"/>.
/// </summary>
public IReadOnlyList<IJobHandler> CreateJobHandlers() =>
[
@@ -84,6 +89,7 @@ public sealed class ArchiveService : IJobHandlerProvider
new OrphansJobHandler(_db, Maintenance),
new Housekeeping.BackupJobHandler(_db, _writer.Options.BackupDirectory),
new Housekeeping.CleanupJobHandler(_db, CleanupSteps, TimeProvider.System),
new EmptyTrashJobHandler(_db, Trash),
];
/// <summary>Makes <paramref name="root"/> a new, empty archive and indexes it.</summary>
@@ -362,9 +368,30 @@ public sealed class ArchiveService : IJobHandlerProvider
public async Task<IReadOnlyList<TrashEntry>> ListTrashAsync(Guid archiveId, CancellationToken cancellationToken = default) =>
await Trash.ListAsync(await LoadArchiveAsync(archiveId, cancellationToken).ConfigureAwait(false), cancellationToken).ConfigureAwait(false);
/// <summary>Moves a sequence back from the trash and indexes it again.</summary>
/// <summary>Moves a sequence back from the trash and indexes it again (an entry of deleted photos: see <see cref="RestoreEntryAsync"/>).</summary>
public async Task<Sequence> RestoreAsync(Guid archiveId, string entryName, CancellationToken cancellationToken = default) =>
await Trash.RestoreAsync(await LoadArchiveAsync(archiveId, cancellationToken).ConfigureAwait(false), entryName, cancellationToken).ConfigureAwait(false);
(await RestoreEntryAsync(archiveId, entryName, null, cancellationToken).ConfigureAwait(false)).Sequence;
/// <summary>
/// Brings a trash entry back: a sequence into its folder (<see cref="SequenceTrash.RestoreAsync"/>), or photos into
/// their sequence (<see cref="Archiving.FrameTrash.RestoreAsync"/>, which queues on <paramref name="followUps"/> what the
/// frames need again).
/// </summary>
/// <exception cref="DirectoryNotFoundException">The entry is not in the trash.</exception>
/// <exception cref="SequenceInLibraryException">A sequence entry whose sequence is in the index already.</exception>
public async Task<TrashRestoreResult> RestoreEntryAsync(Guid archiveId, string entryName, JobEngine? followUps = null,
CancellationToken cancellationToken = default)
{
var archive = await LoadArchiveAsync(archiveId, cancellationToken).ConfigureAwait(false);
if (!Archiving.FrameTrash.IsPhotoEntry(entryName))
return new TrashRestoreResult(TrashKind.Sequence, await Trash.RestoreAsync(archive, entryName, cancellationToken).ConfigureAwait(false), 0, [], false);
var photos = await FrameTrash.RestoreAsync(archive, entryName, followUps, cancellationToken).ConfigureAwait(false);
return new TrashRestoreResult(TrashKind.Photos, photos.Sequence, photos.Restored, photos.Kept, photos.NewSequence);
}
/// <summary>The dry run of emptying an archive's trash: <paramref name="names"/> of the entries (null: all), measured, with the plan's fingerprint.</summary>
public async Task<TrashEmptyPlan> PlanEmptyTrashAsync(Guid archiveId, IReadOnlyList<string>? names = null, CancellationToken cancellationToken = default) =>
await Trash.PlanEmptyAsync(await LoadArchiveAsync(archiveId, cancellationToken).ConfigureAwait(false), names, cancellationToken).ConfigureAwait(false);
/// <summary>Plans a card or folder import into an archive (nothing is copied).</summary>
public async Task<ImportPlan> PlanImportAsync(Guid archiveId, string sourceRoot, ImportOptions options, Guid? cardId = null, IProgress<int>? progress = null,
@@ -440,6 +467,12 @@ public sealed class ArchiveService : IJobHandlerProvider
private static ArgumentException Unknown(Guid sequenceId) => new($"Unknown sequence {sequenceId}.", nameof(sequenceId));
}
/// <summary>
/// What a restore from the trash did: the kind of entry, the sequence restored (or the one the photos went back into),
/// and for photos how many came back, the files kept in the trash and whether they went into a new sequence.
/// </summary>
public sealed record TrashRestoreResult(TrashKind Kind, Sequence Sequence, int Photos, IReadOnlyList<string> Kept, bool NewSequence);
/// <summary>The fields of a sequence the user describes it with (sidecar and index).</summary>
public sealed record SequenceInfo(string? Title, IReadOnlyList<string> Tags, int Rating, string? Notes);
@@ -518,6 +551,19 @@ public static class ArchiveJobs
new(JobKind.Cleanup, archive is null ? [] : [archive.Id], archive?.Name,
Json(new CleanupParameters(steps, olderThanDays), ArchiveJobJson.Default.CleanupParameters), JobPriority.Normal);
/// <summary>
/// <see cref="JobKind.EmptyTrash"/>: erases exactly the entries of a confirmed <paramref name="plan"/>, refused when the
/// trash no longer matches its fingerprint (owner review 2, R6).
/// </summary>
public static ArchiveJobRequest EmptyTrash(Archive archive, TrashEmptyPlan plan)
{
ArgumentNullException.ThrowIfNull(archive);
ArgumentNullException.ThrowIfNull(plan);
var items = plan.Entries.OrderBy(e => e.Name, StringComparer.Ordinal).Select(e => new EmptyTrashItem(e.Name, SequenceTrash.EntryFingerprint(e))).ToList();
return new(JobKind.EmptyTrash, [archive.Id], archive.Name, Json(new EmptyTrashParameters(items, plan.Fingerprint), ArchiveJobJson.Default.EmptyTrashParameters),
JobPriority.User);
}
public static Task<Job> EnqueueAsync(this JobEngine engine, ArchiveJobRequest request, Guid? createdBy = null, CancellationToken cancellationToken = default) =>
engine.EnqueueAsync(request.Kind, request.Targets, request.Label, request.ParametersJson, request.Priority, createdBy, cancellationToken);
+686
View File
@@ -0,0 +1,686 @@
using System.Globalization;
using System.Text.Json;
using Chronos.Core.Archiving;
using Chronos.Core.Diagnostics;
using Chronos.Core.IO;
using Chronos.Core.Metadata;
using Chronos.Core.Model;
using Chronos.Data.Archiving.Jobs;
using Chronos.Data.Jobs;
using ProcessingJobs = Chronos.Data.Processing.ProcessingJobs;
using ProcessingFrames = Chronos.Data.Processing.SequenceFrames;
using SequenceFrames = Chronos.Core.Archiving.SequenceFrames;
namespace Chronos.Data.Archiving;
/// <summary>Why photos cannot leave their sequence; nothing is touched.</summary>
public enum FrameDeleteBlock
{
/// <summary>The archive is offline or read-only.</summary>
Offline,
/// <summary>A running or paused job works on the sequence or its whole archive (<see cref="SequenceTrash"/>'s rule).</summary>
Busy,
/// <summary>The sequence's frames changed since the caller read them (another version, or a frame named is gone).</summary>
FramesChanged,
/// <summary>Every frame of the sequence would go: delete the sequence instead, which keeps its settings and renders.</summary>
WouldEmpty,
}
/// <summary>The dry run of deleting photos of one sequence into the archive's trash (owner review 2, R6).</summary>
/// <param name="Frames">The frames that go, in index order.</param>
/// <param name="Bytes">Their originals' size.</param>
/// <param name="Blocked">Why they cannot go; empty when they can.</param>
public sealed record FrameDeletePlan(Sequence Sequence, IReadOnlyList<Frame> Frames, long Bytes, IReadOnlyList<FrameDeleteBlock> Blocked)
{
public bool CanDelete => Blocked.Count == 0 && Frames.Count > 0;
}
/// <summary>What a photo deletion did: the trash entry made and the sequence as it is now.</summary>
public sealed record FrameDeleteResult(TrashEntry Entry, Sequence Sequence);
/// <summary>
/// What a restore of photos did: the sequence they went back into (a new one when theirs is gone), how many came back, and
/// the files kept in the trash because they no longer match their record (or their photo is in the sequence again).
/// </summary>
public sealed record PhotoRestoreResult(Sequence Sequence, int Restored, IReadOnlyList<string> Kept, bool NewSequence);
/// <summary>
/// Deleting photos of a sequence without erasing anything (owner review 2, R6; BRIEF §0.2 rule 8): <see cref="PlanDeleteAsync"/>
/// is the dry run; <see cref="DeleteAsync"/> moves each original (and its XMP companion) under the sequence's lease into
/// <c>.chronos/trash/photos/&lt;sequence&gt;-&lt;time&gt;/originals</c> next to a record of the frames, writes the sidecar
/// without them (frames version + 1) and with their tombstones, and indexes it: what covered the frames is cleared and
/// queued again, and the project's positions follow their photos (those on the photos that went wait as pending frames).
/// A journal undoes a deletion a crash interrupted. <see cref="RestoreAsync"/> re-hashes each file against its record and
/// moves the matching ones back with the same frame records, dropping their tombstones; into a new sequence through the
/// archive ingest when theirs is gone.
/// </summary>
public sealed class FrameTrash
{
/// <summary>Folder of the photo entries inside the trash.</summary>
public const string FolderName = "photos";
/// <summary>What <see cref="TrashEntry.Name"/> of a photo entry starts with (the folder name follows).</summary>
public const string EntryPrefix = "photos-";
private const string RecordExtension = ".json";
/// <summary>Statuses the frames removed or put back no longer let stand: the passes make them again.</summary>
private const SequenceStatus Uncovered = SequenceStatus.AnalyzedLight | SequenceStatus.AnalyzedDeep | SequenceStatus.ThumbsBuilt | SequenceStatus.ProxiesBuilt;
private readonly ChronosDatabase _db;
private readonly SequenceWriter _writer;
private readonly ArchiveIndexer _indexer;
private readonly ArchiveIngest _ingest;
private readonly SequenceTrash _trash;
internal FrameTrash(ChronosDatabase db, SequenceWriter writer, ArchiveIndexer indexer, ArchiveIngest ingest, SequenceTrash trash)
{
_db = db;
_writer = writer;
_indexer = indexer;
_ingest = ingest;
_trash = trash;
}
/// <summary>The folder of the photo entries of an archive's trash.</summary>
public static string Directory(ArchivePaths archive) => Path.Combine(SequenceTrash.TrashDirectory(archive), FolderName);
/// <summary>Whether <paramref name="name"/> names a photo entry (<see cref="EntryPrefix"/>).</summary>
public static bool IsPhotoEntry(string name) => name.StartsWith(EntryPrefix, StringComparison.Ordinal);
/// <summary>
/// The dry run: the frames of <paramref name="frameIds"/> that go and their size, or why they cannot go. With
/// <paramref name="expectedFramesVersion"/>, frames that changed since the caller read that version block the plan.
/// </summary>
/// <exception cref="ArgumentException">The sequence is not in the index, or no frame is named.</exception>
public async Task<FrameDeletePlan> PlanDeleteAsync(Guid sequenceId, IReadOnlyList<Guid> frameIds, int? expectedFramesVersion = null,
CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(frameIds);
if (frameIds.Count == 0) throw new ArgumentException("No frame is named.", nameof(frameIds));
var sequence = await _db.Sequences.GetAsync(sequenceId, cancellationToken).ConfigureAwait(false)
?? throw new ArgumentException($"Unknown sequence {sequenceId}.", nameof(sequenceId));
var archive = await _db.Archives.GetAsync(sequence.ArchiveId, cancellationToken).ConfigureAwait(false)
?? throw new ArgumentException($"Unknown archive {sequence.ArchiveId}.", nameof(sequenceId));
var all = await ProcessingFrames.ListAsync(_db, sequenceId, cancellationToken).ConfigureAwait(false);
var wanted = frameIds.ToHashSet();
var frames = all.Where(f => wanted.Contains(f.Id)).ToList();
var blocked = new List<FrameDeleteBlock>();
if (ArchiveVolume.Probe(new ArchivePaths(archive.RootPath)) != ArchiveAvailability.Online) blocked.Add(FrameDeleteBlock.Offline);
var busy = await _trash.BusyAsync(cancellationToken).ConfigureAwait(false);
if (busy.Contains(sequence.Id) || busy.Contains(sequence.ArchiveId)) blocked.Add(FrameDeleteBlock.Busy);
if (frames.Count != wanted.Count || (expectedFramesVersion is { } expected && expected != sequence.FramesVersion)) blocked.Add(FrameDeleteBlock.FramesChanged);
if (frames.Count > 0 && frames.Count == all.Count) blocked.Add(FrameDeleteBlock.WouldEmpty);
return new FrameDeletePlan(sequence, frames, frames.Sum(f => f.Size), blocked);
}
/// <summary>
/// Moves exactly the planned photos into the trash and takes them out of the sequence, then queues on
/// <paramref name="followUps"/> (null: nothing) what the frames need again: the light analysis, a background deep analysis
/// when the sequence had one, and the thumbnails and proxies the host keeps (always on a server; on a desktop those it
/// had). A kind the engine does not run, or one already queued for the sequence, is left out.
/// </summary>
/// <param name="deletedBy">Who asked (user name or client), kept in the trash record.</param>
/// <exception cref="StaleDeletePlanException">The plan is blocked, or the sequence changed since it was made; nothing moved.</exception>
/// <exception cref="ArchiveOfflineException">The archive went offline; nothing moved.</exception>
/// <exception cref="IOException">A file is open elsewhere; nothing moved.</exception>
public async Task<FrameDeleteResult> DeleteAsync(FrameDeletePlan plan, string? deletedBy = null, JobEngine? followUps = null,
CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(plan);
var id = plan.Sequence.Id;
if (!plan.CanDelete) throw new StaleDeletePlanException(id);
var revised = await PlanDeleteAsync(id, [.. plan.Frames.Select(f => f.Id)], plan.Sequence.FramesVersion, cancellationToken).ConfigureAwait(false);
var archive = await ArchiveOfAsync(revised.Sequence, cancellationToken).ConfigureAwait(false);
var paths = new ArchivePaths(archive.RootPath);
// Offline since the plan: said as such (ArchiveOfflineException), not as a stale plan.
ArchiveVolume.EnsureWritable(paths);
if (!revised.CanDelete || revised.Sequence.Folder != plan.Sequence.Folder) throw new StaleDeletePlanException(id);
var folder = paths.Sequence(revised.Sequence.Folder);
var had = revised.Sequence.Status;
TrashEntry entry;
SequenceManifest updated;
await using (await _writer.LockAsync(paths, id, cancellationToken).ConfigureAwait(false))
{
var manifest = SequenceWriter.ReadLocked(folder, id);
var plannedPaths = plan.Frames.ToDictionary(f => f.Id, f => f.RelativePath);
var planned = plannedPaths.Keys.ToHashSet();
var removed = manifest.Frames.Where(f => planned.Contains(f.Id)).ToList();
if (manifest.Versions.Frames != plan.Sequence.FramesVersion || removed.Count != planned.Count
|| removed.Any(f => plannedPaths[f.Id] != folder.ArchiveRelative(f.File)))
throw new StaleDeletePlanException(id);
var kept = manifest.Frames.Where(f => !planned.Contains(f.Id)).ToList();
if (kept.Count == 0) throw new StaleDeletePlanException(id);
var inStep = await _indexer.InStepAsync(archive, folder, manifest, cancellationToken).ConfigureAwait(false);
var photosChanged = !inStep && await _indexer.PhotosChangedAsync(archive, folder, cancellationToken).ConfigureAwait(false);
var trash = Directory(paths);
System.IO.Directory.CreateDirectory(trash);
var name = EntryName(trash, id);
var entryFolder = Path.Combine(trash, name);
var recordPath = entryFolder + RecordExtension;
var photos = await TrashedAsync(folder, removed, kept, cancellationToken).ConfigureAwait(false);
var now = DateTimeOffset.UtcNow;
var record = new PhotoTrashRecord(TrashKind.Photos, id, revised.Sequence.DisplayTitle, revised.Sequence.Folder, photos,
photos.Sum(p => p.Files.Sum(f => f.Size)), now, deletedBy);
var moves = photos.SelectMany(p => p.Files).Select(f => new FrameRemovalMove(f.From, paths.ToRelative(Path.Combine(entryFolder, f.Path)))).ToList();
cancellationToken.ThrowIfCancellationRequested();
// The journal first: a crash from here until the sidecar is written puts every file back (ReadLocked).
FrameRemovalJournal.Write(folder, new FrameRemovalJournal(paths.ToRelative(recordPath), paths.ToRelative(entryFolder), [.. planned], moves));
AtomicFile.WriteAllText(recordPath, JsonSerializer.Serialize(record, TrashJson.Default.PhotoTrashRecord));
System.IO.Directory.CreateDirectory(entryFolder);
var done = new List<FrameRemovalMove>();
try
{
foreach (var move in moves)
{
var target = paths.FromRelative(move.To);
System.IO.Directory.CreateDirectory(Path.GetDirectoryName(target)!);
File.Move(folder.File(move.From), target, overwrite: false);
done.Add(move);
}
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
// A file is open elsewhere: the ones that moved go back, and nothing changed.
Undo(paths, folder, done, recordPath, entryFolder);
throw;
}
updated = SequenceFrames.WithFrames(manifest, kept) with
{
Status = manifest.Status & ~Uncovered,
Removed = WithTombstones(manifest.Removed, removed, kept, now),
};
ManifestStore.WriteSequence(folder.ManifestPath, updated);
FrameRemovalJournal.Delete(folder);
Log.Info("archive", $"{revised.Sequence.DisplayTitle}: {removed.Count} photo(s) moved to the trash ({name}, {record.Bytes} bytes)");
// The files have moved: the index follows to the end, whatever the caller cancels meanwhile.
await _indexer.SyncAsync(archive, folder, updated, null, Uncovered, photosChanged, CancellationToken.None).ConfigureAwait(false);
ShiftProxies(folder, manifest.Frames, updated.Frames, removed);
entry = Entry(archive.Id, name, record);
}
var sequence = await _db.Sequences.GetAsync(id, CancellationToken.None).ConfigureAwait(false) ?? revised.Sequence;
await QueueAsync(followUps, archive, sequence, had, CancellationToken.None).ConfigureAwait(false);
return new FrameDeleteResult(entry, sequence);
}
/// <summary>
/// Brings the photos of a trash entry back. Each file is re-hashed against its record: a file that no longer matches
/// (or is missing) stays in the trash with its photo, which is reported, as is a photo whose content the sequence holds
/// again. The others move back to their places (a name taken since gets <c>_2</c>, <c>_3</c>…) with the same frame
/// records, inserted by capture time, and their tombstones go. When the sequence is gone (deleted, or never indexed in
/// this archive), they go into a new sequence through the archive ingest, titled and placed as theirs was. The entry
/// goes once nothing is left in it.
/// </summary>
/// <param name="name">The entry's <see cref="TrashEntry.Name"/> (<c>photos-…</c>).</param>
/// <exception cref="DirectoryNotFoundException">The entry is not in the trash.</exception>
public async Task<PhotoRestoreResult> RestoreAsync(Archive archive, string name, JobEngine? followUps = null, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(archive);
ArgumentException.ThrowIfNullOrWhiteSpace(name);
var paths = new ArchivePaths(archive.RootPath);
ArchiveVolume.EnsureWritable(paths);
var folderName = DiskName(name);
var entryFolder = Path.Combine(Directory(paths), folderName);
var recordPath = entryFolder + RecordExtension;
using (await LeaseFile.AcquireAsync(paths.LocksDirectory, SequenceTrash.EntryLease(name), _writer.Options.Owner, _writer.Options.LeaseWait,
_writer.Options.LeaseDuration, cancellationToken).ConfigureAwait(false))
{
if (!System.IO.Directory.Exists(entryFolder) || ReadRecord(recordPath) is not { } record)
throw new DirectoryNotFoundException($"{name} is not in the trash of {archive.Name}.");
var (intact, kept) = await CheckAsync(entryFolder, record, cancellationToken).ConfigureAwait(false);
// Nothing that matches its record: the entry stays as it is.
if (intact.Count == 0) throw new InvalidDataException(Core.Localization.Loc.L("Archive.Error.TrashMismatch"));
var sequence = await _db.Sequences.GetAsync(record.SequenceId, cancellationToken).ConfigureAwait(false);
var had = sequence?.Status ?? SequenceStatus.None;
PhotoRestoreResult result;
List<TrashedPhoto> left;
if (sequence is not null && sequence.ArchiveId == archive.Id)
(result, left) = await IntoSequenceAsync(archive, paths, sequence, entryFolder, intact, cancellationToken).ConfigureAwait(false);
else
(result, left) = await IntoNewSequenceAsync(archive, paths, record, entryFolder, intact, cancellationToken).ConfigureAwait(false);
var stays = kept.Concat(left).ToList();
FrameRemovalJournal.RemoveEmpty(entryFolder);
// Only an entry left without a file goes; anything still in its folder keeps it listed, never erased by a restore.
if (stays.Count == 0 && !System.IO.Directory.Exists(entryFolder))
{
File.Delete(recordPath);
}
else
{
AtomicFile.WriteAllText(recordPath, JsonSerializer.Serialize(record with
{
Photos = stays,
Bytes = stays.Sum(p => p.Files.Sum(f => f.Size)),
}, TrashJson.Default.PhotoTrashRecord));
}
Log.Info("archive", $"{record.SequenceTitle}: {result.Restored} photo(s) restored from the trash, {stays.Count} kept there");
var all = result with { Kept = [.. stays.Select(p => p.Frame.Original ?? Path.GetFileName(p.Frame.File))] };
if (result.Restored > 0) await QueueAsync(followUps, archive, all.Sequence, had, CancellationToken.None).ConfigureAwait(false);
return all;
}
}
/// <summary>The photo entries of an archive's trash (records whose folder is gone are left out).</summary>
internal static IReadOnlyList<TrashEntry> List(Archive archive, CancellationToken cancellationToken)
{
var folder = Directory(new ArchivePaths(archive.RootPath));
if (!System.IO.Directory.Exists(folder)) return [];
var entries = new List<TrashEntry>();
foreach (var path in System.IO.Directory.EnumerateFiles(folder, "*" + RecordExtension))
{
cancellationToken.ThrowIfCancellationRequested();
var name = Path.GetFileNameWithoutExtension(path);
if (!System.IO.Directory.Exists(Path.Combine(folder, name)) || ReadRecord(path) is not { } record) continue;
entries.Add(Entry(archive.Id, name, record));
}
return entries;
}
/// <summary>The record of a photo entry; null when it is not one or cannot be read.</summary>
internal static PhotoTrashRecord? ReadEntry(ArchivePaths paths, string name) =>
IsPhotoEntry(name) ? ReadRecord(Path.Combine(Directory(paths), DiskName(name)) + RecordExtension) : null;
/// <summary>The entry's folder and record on disk.</summary>
internal static (string Folder, string Record) Location(ArchivePaths paths, string name)
{
var folder = Path.Combine(Directory(paths), DiskName(name));
return (folder, folder + RecordExtension);
}
/// <summary>The folder name of an entry: its name without <see cref="EntryPrefix"/>, never a path.</summary>
private static string DiskName(string name) => Path.GetFileName(IsPhotoEntry(name) ? name[EntryPrefix.Length..] : name);
private async Task<Archive> ArchiveOfAsync(Sequence sequence, CancellationToken cancellationToken) =>
await _db.Archives.GetAsync(sequence.ArchiveId, cancellationToken).ConfigureAwait(false) ?? throw new StaleDeletePlanException(sequence.Id);
/// <summary>
/// The files of each frame that goes: its original (with the hash it was archived with, or one computed now for a frame
/// never hashed) under the entry's <c>originals/</c>, and an XMP companion (<c>IMG_0001.xmp</c>, <c>IMG_0001.CR2.xmp</c>)
/// no frame that stays shares. A frame whose original is missing goes without a file.
/// </summary>
private static async Task<List<TrashedPhoto>> TrashedAsync(SequencePaths folder, List<SidecarFrame> removed, List<SidecarFrame> kept,
CancellationToken cancellationToken)
{
var used = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
var claimed = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
var keptStems = kept.Select(f => Stem(f.File)).ToHashSet(StringComparer.OrdinalIgnoreCase);
var photos = new List<TrashedPhoto>(removed.Count);
foreach (var frame in removed)
{
cancellationToken.ThrowIfCancellationRequested();
var files = new List<TrashedFile>();
var original = new FileInfo(folder.File(frame.File));
var sha = frame.Sha256;
if (original.Exists)
{
sha ??= await ContentHash.Sha256Async(original.FullName, null, cancellationToken).ConfigureAwait(false);
files.Add(new TrashedFile(frame.File, Unique(used, "originals/" + Inside(frame.File)), original.Length, sha));
if (!keptStems.Contains(Stem(frame.File)))
{
foreach (var companion in Companions(folder, frame.File).Where(claimed.Add))
{
var path = folder.File(companion);
files.Add(new TrashedFile(companion, Unique(used, "originals/" + Inside(companion)), new FileInfo(path).Length,
await ContentHash.Sha256Async(path, null, cancellationToken).ConfigureAwait(false)));
}
}
}
photos.Add(new TrashedPhoto(frame with { Sha256 = sha }, files, null));
}
return photos;
}
/// <summary>The XMP files beside a photo that name it (<c>stem.xmp</c> or <c>file.xmp</c>), relative to the sequence folder.</summary>
private static IEnumerable<string> Companions(SequencePaths folder, string file)
{
var directory = Path.GetDirectoryName(folder.File(file))!;
if (!System.IO.Directory.Exists(directory)) yield break;
var prefix = file.Contains('/') ? file[..(file.LastIndexOf('/') + 1)] : string.Empty;
var name = Path.GetFileName(file);
var stem = Path.GetFileNameWithoutExtension(name);
foreach (var candidate in System.IO.Directory.EnumerateFiles(directory, "*.xmp"))
{
var candidateName = Path.GetFileName(candidate);
if (!Path.GetExtension(candidateName).Equals(".xmp", StringComparison.OrdinalIgnoreCase)) continue;
var named = Path.GetFileNameWithoutExtension(candidateName);
if (named.Equals(stem, StringComparison.OrdinalIgnoreCase) || named.Equals(name, StringComparison.OrdinalIgnoreCase)) yield return prefix + candidateName;
}
}
/// <summary>A photo's stem with its folder (<c>originals/IMG_0001</c>): its companions share it.</summary>
private static string Stem(string file)
{
var slash = file.LastIndexOf('/');
var name = file[(slash + 1)..];
var dot = name.IndexOf('.', StringComparison.Ordinal);
return file[..(slash + 1)] + (dot > 0 ? name[..dot] : name);
}
/// <summary>A sequence-relative path without its leading <c>originals/</c>.</summary>
private static string Inside(string file) =>
file.StartsWith(ArchiveLayout.OriginalsFolder + "/", StringComparison.OrdinalIgnoreCase) ? file[(ArchiveLayout.OriginalsFolder.Length + 1)..] : file;
/// <summary><paramref name="path"/>, or with <c>_2</c>, <c>_3</c>… before its extension when another file of the entry has it.</summary>
private static string Unique(HashSet<string> used, string path)
{
var candidate = path;
for (var n = 2; !used.Add(candidate); n++)
candidate = string.Create(CultureInfo.InvariantCulture, $"{Path.ChangeExtension(path, null)}_{n}{Path.GetExtension(path)}");
return candidate;
}
/// <summary>
/// The tombstones after a deletion: those of before, and one per SHA-256 of the photos that went that no frame staying
/// holds (a duplicate that stays keeps the content in the sequence). A photo without a hash leaves none.
/// </summary>
private static List<RemovedPhoto>? WithTombstones(IReadOnlyList<RemovedPhoto>? before, List<SidecarFrame> removed, List<SidecarFrame> kept,
DateTimeOffset at)
{
var held = kept.Where(f => f.Sha256 is not null).Select(f => f.Sha256!).ToHashSet(StringComparer.OrdinalIgnoreCase);
var tombstones = new Dictionary<string, RemovedPhoto>(StringComparer.OrdinalIgnoreCase);
foreach (var tombstone in before ?? []) tombstones[tombstone.Sha256] = tombstone;
foreach (var sha in removed.Select(f => f.Sha256).OfType<string>())
{
if (!held.Contains(sha)) tombstones[sha] = new RemovedPhoto { Sha256 = sha.ToLowerInvariant(), Reason = RemovalReason.Deleted, At = at };
}
return tombstones.Count == 0 ? null : [.. tombstones.Values];
}
/// <summary>Puts back the files of a deletion that failed half-way, and removes its record, folder and journal.</summary>
private static void Undo(ArchivePaths paths, SequencePaths folder, List<FrameRemovalMove> done, string recordPath, string entryFolder)
{
foreach (var move in done)
{
try
{
File.Move(paths.FromRelative(move.To), folder.File(move.From), overwrite: false);
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
Log.Error("archive", $"{move.To} could not move back to {folder.ArchiveRelative(move.From)}; the journal puts it back later", ex);
return;
}
}
File.Delete(recordPath);
FrameRemovalJournal.RemoveEmpty(entryFolder);
FrameRemovalJournal.Delete(folder);
}
/// <summary>
/// Renames the proxies of the frames whose index changed (BRIEF §8.5: proxies are named by frame index) so they stay
/// theirs, and deletes those of the frames removed: the passes queued afterwards reuse them instead of making the
/// sequence's proxies again. Proxies are regenerable and carry their frame's capture time, so one left in the wrong
/// place (a failure here, logged) is made again.
/// </summary>
internal static void ShiftProxies(SequencePaths folder, IReadOnlyList<SidecarFrame> before, IReadOnlyList<SidecarFrame> after,
IReadOnlyList<SidecarFrame> removed)
{
if (!System.IO.Directory.Exists(folder.Proxies)) return;
try
{
foreach (var frame in removed)
{
var stale = folder.Proxy(frame.I);
if (File.Exists(stale)) File.Delete(stale);
}
var now = after.ToDictionary(f => f.Id, f => f.I);
var moves = before.Where(f => now.TryGetValue(f.Id, out var to) && to != f.I).Select(f => (From: f.I, To: now[f.Id])).ToList();
if (moves.Count == 0) return;
// Downwards from the lowest when frames left, upwards from the highest when frames came: each target is free by then.
var ordered = moves[0].To < moves[0].From ? moves.OrderBy(m => m.From) : moves.OrderByDescending(m => m.From);
foreach (var (from, to) in ordered)
{
var source = folder.Proxy(from);
var target = folder.Proxy(to);
if (File.Exists(source)) File.Move(source, target, overwrite: true);
else if (File.Exists(target)) File.Delete(target);
}
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
Log.Warning("archive", $"Proxies of {folder.RelativeFolder} did not follow their frames; the proxy pass makes them again", ex);
}
}
/// <summary>Queues what the frames need again (see <see cref="DeleteAsync"/>), as an orphans fix does.</summary>
private async Task QueueAsync(JobEngine? followUps, Archive archive, Sequence sequence, SequenceStatus had, CancellationToken cancellationToken)
{
if (followUps is null) return;
var label = sequence.DisplayTitle;
List<ArchiveJobRequest> requests = [ProcessingJobs.LightAnalysis([sequence.Id], label)];
if ((had & SequenceStatus.AnalyzedDeep) != 0) requests.Add(ProcessingJobs.DeepAnalysis([sequence.Id], label) with { Priority = JobPriority.Background });
var images = _writer.Options.ImagesAfterScan;
if (images || (had & SequenceStatus.ThumbsBuilt) != 0) requests.Add(ArchiveJobs.Thumbnails([sequence.Id], label));
if ((images || (had & SequenceStatus.ProxiesBuilt) != 0) && archive.ProxyLongEdge != ProxyPolicy.Disabled)
requests.Add(ArchiveJobs.Proxies([sequence.Id], label));
foreach (var request in requests)
{
if (!followUps.Runs(request.Kind) || await _db.Jobs.IsQueuedAsync(request.Kind, sequence.Id, cancellationToken).ConfigureAwait(false)) continue;
await followUps.EnqueueAsync(request, cancellationToken: cancellationToken).ConfigureAwait(false);
}
}
/// <summary>Each photo whose files all match their record (size and SHA-256), and those that stay in the trash.</summary>
private static async Task<(List<TrashedPhoto> Intact, List<TrashedPhoto> Kept)> CheckAsync(string entryFolder, PhotoTrashRecord record,
CancellationToken cancellationToken)
{
var intact = new List<TrashedPhoto>();
var kept = new List<TrashedPhoto>();
foreach (var photo in record.Photos)
{
var ok = photo.Files.Count > 0;
foreach (var file in photo.Files)
{
var path = Path.Combine(entryFolder, file.Path);
var info = new FileInfo(path);
if (!info.Exists || info.Length != file.Size
|| (file.Sha256 is { } sha && !string.Equals(sha, await ContentHash.Sha256Async(path, null, cancellationToken).ConfigureAwait(false),
StringComparison.OrdinalIgnoreCase)))
{
Log.Warning("archive", $"{file.Path} of trash entry {Path.GetFileName(entryFolder)} does not match its record; it stays in the trash");
ok = false;
break;
}
}
(ok ? intact : kept).Add(photo);
}
return (intact, kept);
}
/// <summary>The photos back into their sequence, under its lease; returns what came back and the photos left in the trash.</summary>
private async Task<(PhotoRestoreResult, List<TrashedPhoto>)> IntoSequenceAsync(Archive archive, ArchivePaths paths, Sequence sequence, string entryFolder,
List<TrashedPhoto> intact, CancellationToken cancellationToken)
{
var folder = paths.Sequence(sequence.Folder);
var left = new List<TrashedPhoto>();
await using (await _writer.LockAsync(paths, sequence.Id, cancellationToken).ConfigureAwait(false))
{
var manifest = SequenceWriter.ReadLocked(folder, sequence.Id);
var inStep = await _indexer.InStepAsync(archive, folder, manifest, cancellationToken).ConfigureAwait(false);
var photosChanged = !inStep && await _indexer.PhotosChangedAsync(archive, folder, cancellationToken).ConfigureAwait(false);
var ids = manifest.Frames.Select(f => f.Id).ToHashSet();
var held = manifest.Frames.Where(f => f.Sha256 is not null).Select(f => f.Sha256!).ToHashSet(StringComparer.OrdinalIgnoreCase);
var taken = manifest.Frames.Select(f => f.File).ToHashSet(StringComparer.OrdinalIgnoreCase);
var back = new List<SidecarFrame>();
foreach (var photo in intact)
{
cancellationToken.ThrowIfCancellationRequested();
// In the sequence again (restored before, or filed again since): the trash keeps its copy.
if (ids.Contains(photo.Frame.Id) || (photo.Frame.Sha256 is { } sha && held.Contains(sha)))
{
left.Add(photo);
continue;
}
var original = photo.Files[0];
var tails = photo.Files.Skip(1).Select(c => CompanionTail(c.From, original.From)).ToList();
var file = FreeName(folder, original.From, tails, taken);
var stem = file[..^Path.GetExtension(file).Length];
try
{
MoveIn(Path.Combine(entryFolder, original.Path), folder.File(file));
// A companion follows its photo's name: IMG_0001_2.xmp beside IMG_0001_2.JPG.
for (var c = 0; c < tails.Count; c++) MoveIn(Path.Combine(entryFolder, photo.Files[c + 1].Path), folder.File(stem + tails[c]));
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
Log.Warning("archive", $"{original.Path} could not move back into {folder.RelativeFolder}; it stays in the trash", ex);
left.Add(photo);
continue;
}
taken.Add(file);
back.Add(photo.Frame with { File = file, Flags = photo.Frame.Flags & ~FrameFlags.Missing });
}
if (back.Count == 0) return (new PhotoRestoreResult(sequence, 0, [], false), left);
var restored = back.Where(f => f.Sha256 is not null).Select(f => f.Sha256!).ToHashSet(StringComparer.OrdinalIgnoreCase);
var tombstones = manifest.Removed?.Where(r => !restored.Contains(r.Sha256)).ToList();
var updated = SequenceFrames.Insert(manifest, back) with
{
Status = manifest.Status & ~Uncovered,
Removed = tombstones is { Count: > 0 } ? tombstones : null,
};
ManifestStore.WriteSequence(folder.ManifestPath, updated);
await _indexer.SyncAsync(archive, folder, updated, null, Uncovered, photosChanged, CancellationToken.None).ConfigureAwait(false);
ShiftProxies(folder, manifest.Frames, updated.Frames, []);
}
var row = await _db.Sequences.GetAsync(sequence.Id, CancellationToken.None).ConfigureAwait(false) ?? sequence;
return (new PhotoRestoreResult(row, intact.Count - left.Count, [], false), left);
}
/// <summary>
/// The photos of a sequence that is gone into a new one, through the archive ingest (staging, naming templates, a
/// sidecar of its own), one photo at a time so one that fails stays in the trash alone: titled as theirs was, placed at
/// its folder when that is free, and keyed by its id, so every entry of that sequence comes back into the same new
/// one. Companions follow their photo.
/// </summary>
private async Task<(PhotoRestoreResult, List<TrashedPhoto>)> IntoNewSequenceAsync(Archive archive, ArchivePaths paths, PhotoTrashRecord record,
string entryFolder, List<TrashedPhoto> intact, CancellationToken cancellationToken)
{
var left = new List<TrashedPhoto>();
var staging = Path.Combine(paths.StagingDirectory, $"trash-{record.SequenceId:N}");
System.IO.Directory.CreateDirectory(staging);
var target = new SequenceTarget($"trash:{record.SequenceId:N}", record.SequenceTitle, record.SequenceFolder.Length > 0 ? record.SequenceFolder : null, null);
Guid? sequenceId = null;
var restored = 0;
foreach (var photo in intact)
{
cancellationToken.ThrowIfCancellationRequested();
var original = photo.Files[0];
var inEntry = Path.Combine(entryFolder, original.Path);
var path = Path.Combine(staging, Guid.NewGuid().ToString("N") + Path.GetExtension(original.From));
File.Move(inEntry, path, overwrite: false);
IngestResult result;
try
{
var frame = new StagedFrame(path, photo.Frame.Original ?? Path.GetFileName(original.From), original.Size, photo.Frame.Sha256 ?? original.Sha256!,
MetadataReader.Read(path, cancellationToken), photo.Frame.CapturedAt, Path.GetFileName(original.From));
result = (await _ingest.CommitSessionAsync(archive, target, [frame], cancellationToken).ConfigureAwait(false))[0];
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or InvalidDataException)
{
Log.Warning("archive", $"{original.Path} could not be restored into a new sequence; it stays in the trash", ex);
if (File.Exists(path)) File.Move(path, inEntry, overwrite: false);
left.Add(photo);
continue;
}
sequenceId ??= result.SequenceId;
if (File.Exists(path))
{
// A duplicate of a photo the new sequence holds stays in staging: it returns to the entry.
File.Move(path, inEntry, overwrite: false);
left.Add(photo);
continue;
}
restored++;
var placed = paths.FromRelative(result.RelativePath);
foreach (var companion in photo.Files.Skip(1))
{
// Beside its photo; a name taken there (another file of that name) keeps the companion in the trash.
var beside = placed[..^Path.GetExtension(placed).Length] + CompanionTail(companion.From, original.From);
if (!File.Exists(beside)) MoveIn(Path.Combine(entryFolder, companion.Path), beside);
}
}
SequenceTrash.Erase(staging);
if (sequenceId is not { } id) throw new IOException(Core.Localization.Loc.L("Archive.Error.TrashMismatch"));
var sequence = await _db.Sequences.GetAsync(id, CancellationToken.None).ConfigureAwait(false)
?? throw new InvalidOperationException($"The photos of {record.SequenceTitle} were restored but their sequence is not indexed.");
return (new PhotoRestoreResult(sequence, restored, [], true), left);
}
/// <summary>What a companion's name adds to its photo's stem (<c>.xmp</c>, <c>.CR2.xmp</c>).</summary>
private static string CompanionTail(string companion, string photo)
{
var stem = Path.GetFileNameWithoutExtension(Path.GetFileName(photo));
var name = Path.GetFileName(companion);
return name.StartsWith(stem, StringComparison.OrdinalIgnoreCase) ? name[stem.Length..] : Path.GetExtension(name);
}
/// <summary>
/// <paramref name="file"/> when it is free in the sidecar and on disk with its companions (<paramref name="tails"/> after
/// its stem), else <c>stem_2.ext</c>, <c>stem_3.ext</c>…
/// </summary>
private static string FreeName(SequencePaths folder, string file, IReadOnlyList<string> tails, HashSet<string> taken)
{
var extension = Path.GetExtension(file);
var stem = file[..^extension.Length];
var candidate = file;
for (var n = 2; taken.Contains(candidate) || File.Exists(folder.File(candidate)) || tails.Any(t => File.Exists(folder.File(Stem(candidate, extension) + t))); n++)
candidate = string.Create(CultureInfo.InvariantCulture, $"{stem}_{n}{extension}");
return candidate;
static string Stem(string name, string extension) => name[..^extension.Length];
}
private static void MoveIn(string from, string to)
{
System.IO.Directory.CreateDirectory(Path.GetDirectoryName(to)!);
File.Move(from, to, overwrite: false);
}
/// <summary>The sequence id and the time, with a number when an entry of that second exists already.</summary>
private static string EntryName(string trash, Guid sequenceId)
{
var stamp = DateTime.UtcNow.ToString("yyyyMMddHHmmss", CultureInfo.InvariantCulture);
var name = $"{sequenceId:N}-{stamp}";
for (var n = 2; System.IO.Directory.Exists(Path.Combine(trash, name)) || File.Exists(Path.Combine(trash, name) + RecordExtension); n++)
name = string.Create(CultureInfo.InvariantCulture, $"{sequenceId:N}-{stamp}-{n}");
return name;
}
private static TrashEntry Entry(Guid archiveId, string folderName, PhotoTrashRecord record) =>
new(archiveId, EntryPrefix + folderName, record.SequenceId, record.SequenceTitle, record.SequenceFolder, record.Photos.Count,
record.Photos.Sum(p => p.Files.Count), record.Bytes, record.DeletedAt, record.DeletedBy, record.Kind);
private static PhotoTrashRecord? ReadRecord(string path)
{
try
{
return File.Exists(path) ? JsonSerializer.Deserialize(File.ReadAllText(path), TrashJson.Default.PhotoTrashRecord) : null;
}
catch (Exception ex) when (ex is JsonException or IOException or UnauthorizedAccessException)
{
Log.Warning("archive", $"Trash record {path} is unreadable", ex);
return null;
}
}
}
/// <summary>
/// What the trash keeps next to deleted photos (owner review 2, R6): the frame records, where each file was and is, the
/// sequence they came from. <see cref="TrashKind.Replaced"/> entries (a later conversion) name each photo's replacement.
/// </summary>
internal sealed record PhotoTrashRecord(TrashKind Kind, Guid SequenceId, string SequenceTitle, string SequenceFolder, IReadOnlyList<TrashedPhoto> Photos,
long Bytes, DateTimeOffset DeletedAt, string? DeletedBy);
/// <summary>One photo of a photo entry: its frame record, its files (the original first) and, when replaced, the replacement's SHA-256.</summary>
internal sealed record TrashedPhoto(SidecarFrame Frame, IReadOnlyList<TrashedFile> Files, string? Replacement);
/// <param name="From">Where the file was, relative to the sequence folder.</param>
/// <param name="Path">Where it is, relative to the entry's folder.</param>
/// <param name="Sha256">Its SHA-256 when it left the sequence; a restore re-hashes the file against it.</param>
internal sealed record TrashedFile(string From, string Path, long Size, string? Sha256);
@@ -31,6 +31,15 @@ public sealed record BackupParameters(int Keep);
/// </summary>
public sealed record CleanupParameters(IReadOnlyList<string> Steps, int OlderThanDays);
/// <summary>
/// <see cref="JobKind.EmptyTrash"/>: target = the archive; the entries of the confirmed plan, each with its own fingerprint
/// (<see cref="SequenceTrash.EntryFingerprint"/>), and the plan's fingerprint (<see cref="TrashEmptyPlan.Fingerprint"/>).
/// </summary>
public sealed record EmptyTrashParameters(IReadOnlyList<EmptyTrashItem> Entries, string Fingerprint);
/// <summary>One entry of a confirmed emptying: its name and fingerprint when it was planned.</summary>
public sealed record EmptyTrashItem(string Name, string Fingerprint);
/// <summary>A job ready to enqueue.</summary>
public sealed record ArchiveJobRequest(JobKind Kind, IReadOnlyList<Guid> Targets, string? Label, string? ParametersJson, JobPriority Priority);
@@ -42,6 +51,7 @@ public sealed record ArchiveJobRequest(JobKind Kind, IReadOnlyList<Guid> Targets
[JsonSerializable(typeof(ImportJobParameters))]
[JsonSerializable(typeof(BackupParameters))]
[JsonSerializable(typeof(CleanupParameters))]
[JsonSerializable(typeof(EmptyTrashParameters))]
[JsonSerializable(typeof(CardProfile))]
internal sealed partial class ArchiveJobJson : JsonSerializerContext
{
@@ -0,0 +1,37 @@
using System.Globalization;
using Chronos.Core.Localization;
using Chronos.Core.Model;
using Chronos.Data.Housekeeping;
using Chronos.Data.Jobs;
namespace Chronos.Data.Archiving.Jobs;
/// <summary>
/// <see cref="JobKind.EmptyTrash"/> (owner review 2, R6): erases exactly the entries of a confirmed plan
/// (<see cref="EmptyTrashParameters"/>) and fails, erasing nothing, when the trash no longer matches the plan's
/// fingerprint (<see cref="StaleTrashPlanException"/>). Checkpoint: <c>done:&lt;n&gt;</c>, the entries done; a resumed job
/// checks each remaining entry against its own fingerprint. Entries kept (a replacement that no longer matches, an entry
/// changed since) are logged and fail the job at the end, after the others are erased.
/// </summary>
internal sealed class EmptyTrashJobHandler(ChronosDatabase db, SequenceTrash trash) : IJobHandler
{
private const string Prefix = "done:";
public JobKind Kind => JobKind.EmptyTrash;
public async Task RunAsync(JobContext context)
{
var archive = await JobArchive.LoadAsync(db, context.Job, context.Cancellation).ConfigureAwait(false);
var parameters = ArchiveJobJson.Read(context.Job, ArchiveJobJson.Default.EmptyTrashParameters, new EmptyTrashParameters([], string.Empty));
var start = context.Checkpoint is { } checkpoint && checkpoint.StartsWith(Prefix, StringComparison.Ordinal)
&& int.TryParse(checkpoint.AsSpan(Prefix.Length), NumberStyles.None, CultureInfo.InvariantCulture, out var done)
? Math.Min(done, parameters.Entries.Count)
: 0;
await context.ReportAsync(start, parameters.Entries.Count).ConfigureAwait(false);
var result = await trash.EmptyAsync(archive, parameters, start,
(n, total) => context.CheckpointAsync(Prefix + n.ToString(CultureInfo.InvariantCulture), n, total), context.Cancellation).ConfigureAwait(false);
context.Log(Loc.L("Archive.Log.TrashEmptied", result.Erased, CleanupSteps.Megabytes(result.Bytes)));
foreach (var title in result.Refused) context.Log(Loc.L("Archive.Log.TrashKept", title));
if (result.Refused.Count > 0) throw new InvalidOperationException(Loc.Plural("Archive.Error.TrashKept", result.Refused.Count));
}
}
+34 -4
View File
@@ -12,15 +12,19 @@ namespace Chronos.Data.Archiving;
/// Keeps a project's frame positions (exclusions, hero, virtual-camera keyframes; BRIEF §5, §7) on their frames when the
/// index takes a sidecar in which frames were inserted before others: an upload or import of earlier photos, photos a
/// scan found, a sync download, a journalled batch completed after a crash. Positions name the frames in the indexed
/// order (what editors, renders and the API read), so the old order comes from the index rows. Only an insertion is
/// followed: when an indexed frame is no longer in the sidecar, positions cannot be mapped and stay as they are. The move
/// is part of the frames change, not a settings edit, so the settings version stays. Settings synced from another copy
/// order (what editors, renders and the API read), so the old order comes from the index rows. An insertion moves them
/// by position; when an indexed frame is no longer in the sidecar (photos deleted, owner review 2, R6) they are translated
/// by photo, those on the photos that went waiting as pending frames. The move is part of the frames change, not a
/// settings edit, so the settings version stays. Settings synced from another copy
/// arrive translated to this copy's frames (<see cref="ProjectSettings.TranslateFramePositions"/>); their positions on
/// photos that were not here yet (<see cref="ProjectSettings.PendingFrames"/>) take their place when the photos arrive,
/// wherever they land.
/// </summary>
internal static class ProjectPositions
{
/// <summary>Index frames read at once when positions are translated by photo.</summary>
private const int PageSize = 1000;
/// <summary>
/// The index's copy of the project (the one in force) with its positions moved to the order of
/// <paramref name="manifest"/> and its pending frames placed; null when nothing changes. A <c>project.json</c> the
@@ -45,9 +49,35 @@ internal static class ProjectPositions
}
if (index != row.Index) moves[row.Index] = index;
}
return Placed(project, moves is null ? null : ProjectSettings.MoveFramePositions(project, moves), manifest);
if (moves is null) return await RemovedAsync(db, indexed.Id, project, manifest, cancellationToken).ConfigureAwait(false);
return Placed(project, ProjectSettings.MoveFramePositions(project, moves), manifest);
}
/// <summary>
/// Indexed frames are no longer in the sidecar (photos deleted into the trash, here or by another process sharing the
/// archive, owner review 2, R6): the positions are translated by photo from the index's frames to the sidecar's
/// (<see cref="ProjectSettings.TranslateFramePositions"/>), so those on the photos that went wait as pending frames
/// and take their place again when the photos come back (a restore), and the others follow their photos.
/// </summary>
private static async Task<PositionMove?> RemovedAsync(ChronosRepositories db, Guid sequenceId, string project, SequenceManifest manifest,
CancellationToken cancellationToken)
{
var before = new List<(int, FramePhoto)>();
for (var after = -1; ;)
{
var page = await db.Frames.ListBySequenceAsync(sequenceId, after, PageSize, cancellationToken).ConfigureAwait(false);
before.AddRange(page.Select(f => (f.Index, new FramePhoto(f.Sha256, f.Size, f.CapturedAt))));
if (page.Count < PageSize) break;
after = page[^1].Index;
}
var to = new FramePhotos(manifest.Frames.Select(f => (f.I, new FramePhoto(f.Sha256, f.Size, f.CapturedAt))));
var translated = ProjectSettings.TranslateFramePositions(project, new FramePhotos(before), to);
if (translated is null) return null;
var hero = HeroOf(project) is null && HeroOf(translated) is { } index ? manifest.Frames.FirstOrDefault(f => f.I == index)?.Id : null;
return new PositionMove(project, translated, hero);
}
/// <summary>
/// The index's copy of the project with the pending frames whose photos <paramref name="manifest"/> now holds put in
/// place: frames appended in step with the index, which moves no position. Null when nothing changes.
@@ -41,6 +41,13 @@ internal static class SequenceMapping
public static List<Frame> ToFrames(SequenceManifest manifest, SequencePaths sequence) =>
manifest.Frames.Select(f => ToFrame(f, manifest.Id, sequence)).ToList();
/// <summary>The sidecar's tombstones as index rows (owner review 2, R6).</summary>
public static List<FrameRemoval> ToRemovals(SequenceManifest manifest) =>
manifest.Removed is not { Count: > 0 } removed
? []
: removed.Where(r => !string.IsNullOrEmpty(r.Sha256))
.Select(r => new FrameRemoval(manifest.Id, r.Sha256.ToLowerInvariant(), r.Replacement?.ToLowerInvariant(), r.Reason, r.At)).ToList();
/// <param name="existing">The current row; its render, analysis, sync and ownership fields are kept, and so are its
/// title, tags, rating and notes when its settings version is ahead of the sidecar's (edited in the index only).</param>
public static Sequence ToSequence(SequenceManifest m, Guid archiveId, SequencePaths sequence, Sequence? existing)
@@ -0,0 +1,205 @@
using System.Globalization;
using System.Security.Cryptography;
using System.Text;
using Chronos.Core.Archiving;
using Chronos.Core.Diagnostics;
using Chronos.Core.Localization;
using Chronos.Core.Model;
using Chronos.Data.Archiving.Jobs;
namespace Chronos.Data.Archiving;
/// <summary>
/// The dry run of emptying an archive's trash (owner review 2, R6; BRIEF §0.2 rule 8): the entries that would be erased,
/// with the files and bytes their folders hold now, and a fingerprint of exactly those entries. Erasing
/// (<see cref="JobKind.EmptyTrash"/>) refuses a plan whose fingerprint no longer matches the trash.
/// </summary>
/// <param name="Unknown">Names asked for that the trash does not hold.</param>
public sealed record TrashEmptyPlan(Guid ArchiveId, IReadOnlyList<TrashEntry> Entries, long Bytes, string Fingerprint, IReadOnlyList<string> Unknown)
{
public int Files => Entries.Sum(e => e.Files);
public int Photos => Entries.Sum(e => e.Frames);
}
/// <summary>What an emptying erased, and the entries it kept (a replacement that no longer matches, an entry changed since the plan).</summary>
public sealed record TrashEmptyResult(int Erased, long Bytes, IReadOnlyList<string> Refused);
/// <summary>The trash changed after the emptying was planned (an entry came, went or changed); nothing was erased.</summary>
public sealed class StaleTrashPlanException() : InvalidOperationException(Loc.L("Archive.Error.TrashChanged"));
public sealed partial class SequenceTrash
{
/// <summary>Folder inside the trash that entries move into before they are erased, so a listing never sees half an entry.</summary>
private const string ErasingFolder = ".erasing";
/// <summary>The lease that keeps a restore and an erasing of one entry apart.</summary>
internal static string EntryLease(string name) => "trash-" + name;
/// <summary>
/// The dry run of emptying the trash: <paramref name="names"/> of the entries to erase (null: every entry), each
/// measured on disk, and the plan's fingerprint (<see cref="Fingerprint"/>). Reads only.
/// </summary>
public async Task<TrashEmptyPlan> PlanEmptyAsync(Archive archive, IReadOnlyList<string>? names = null, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(archive);
var paths = new ArchivePaths(archive.RootPath);
var listed = await ListAsync(archive, cancellationToken).ConfigureAwait(false);
var wanted = names?.ToHashSet(StringComparer.Ordinal);
var chosen = wanted is null ? listed : [.. listed.Where(e => wanted.Contains(e.Name))];
var measured = await Task.Run(() => chosen.Select(e =>
{
var (folder, _) = Location(paths, e);
var (files, bytes) = Directory.Exists(folder) ? Measure(folder, cancellationToken) : (0, 0L);
return e with { Files = files, Bytes = bytes };
}).ToList(), cancellationToken).ConfigureAwait(false);
var unknown = names?.Where(n => !listed.Any(e => e.Name == n)).Distinct(StringComparer.Ordinal).ToList() ?? [];
return new TrashEmptyPlan(archive.Id, measured, measured.Sum(e => e.Bytes), Fingerprint(measured), unknown);
}
/// <summary>
/// Fingerprint of a set of entries: SHA-256 of each one's <see cref="EntryFingerprint"/> in name order. The same
/// entries with the same files on disk give the same fingerprint whatever order they were listed in.
/// </summary>
public static string Fingerprint(IEnumerable<TrashEntry> entries)
{
ArgumentNullException.ThrowIfNull(entries);
var text = new StringBuilder();
foreach (var entry in entries.OrderBy(e => e.Name, StringComparer.Ordinal)) text.Append(EntryFingerprint(entry)).Append('\n');
return Convert.ToHexStringLower(SHA256.HashData(Encoding.UTF8.GetBytes(text.ToString())));
}
/// <summary>What identifies one entry as planned: its kind, name, files and bytes on disk, and when it was deleted.</summary>
public static string EntryFingerprint(TrashEntry entry)
{
ArgumentNullException.ThrowIfNull(entry);
return string.Create(CultureInfo.InvariantCulture, $"{entry.Kind}|{entry.Name}|{entry.Files}|{entry.Bytes}|{entry.DeletedAt.UtcTicks}");
}
/// <summary>
/// Erases the entries of <paramref name="parameters"/> from <paramref name="start"/> on, one at a time under its lease,
/// each moved aside whole and then deleted (a listing never shows half an entry). From the start, the trash must still
/// hold exactly the planned entries (<see cref="EmptyTrashParameters.Fingerprint"/>), else nothing is erased; a run
/// resumed after a stop checks each remaining entry against its own fingerprint, keeps one that changed and skips one
/// that is gone. A <see cref="TrashKind.Replaced"/> entry is erased only while every replacement still matches its
/// SHA-256 in the sequence (re-hashed now); otherwise it is kept. Leftovers of an interrupted erase are deleted first.
/// </summary>
/// <param name="progress">Entries done so far (the resume point) and their number.</param>
/// <exception cref="StaleTrashPlanException">The trash no longer holds exactly the planned entries; nothing was erased.</exception>
internal async Task<TrashEmptyResult> EmptyAsync(Archive archive, EmptyTrashParameters parameters, int start, Func<int, int, ValueTask> progress,
CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(archive);
ArgumentNullException.ThrowIfNull(parameters);
var paths = new ArchivePaths(archive.RootPath);
ArchiveVolume.EnsureWritable(paths);
SweepErasing(paths);
var items = parameters.Entries;
var current = await PlanEmptyAsync(archive, [.. items.Skip(start).Select(i => i.Name)], cancellationToken).ConfigureAwait(false);
if (start == 0 && (current.Fingerprint != parameters.Fingerprint || current.Unknown.Count > 0)) throw new StaleTrashPlanException();
var refused = new List<string>();
int erased = 0;
long bytes = 0;
for (var i = start; i < items.Count; i++)
{
cancellationToken.ThrowIfCancellationRequested();
var item = items[i];
if (current.Entries.FirstOrDefault(e => e.Name == item.Name) is { } entry)
{
if (EntryFingerprint(entry) != item.Fingerprint)
{
refused.Add(entry.Title);
Log.Warning("archive", $"Trash entry {entry.Name} changed since the emptying was planned; it is kept");
}
else if (await EraseAsync(paths, entry, cancellationToken).ConfigureAwait(false))
{
erased++;
bytes += entry.Bytes;
}
else
{
refused.Add(entry.Title);
}
}
await progress(i + 1, items.Count).ConfigureAwait(false);
}
return new TrashEmptyResult(erased, bytes, refused);
}
/// <summary>Erases one entry under its lease; false (logged) when a replaced entry's replacements no longer match, or it changed.</summary>
private async Task<bool> EraseAsync(ArchivePaths paths, TrashEntry entry, CancellationToken cancellationToken)
{
using (await LeaseFile.AcquireAsync(paths.LocksDirectory, EntryLease(entry.Name), _writer.Options.Owner, _writer.Options.LeaseWait,
_writer.Options.LeaseDuration, cancellationToken).ConfigureAwait(false))
{
var (folder, record) = Location(paths, entry);
if (!Directory.Exists(folder) || !File.Exists(record)) return false;
if (entry.Kind == TrashKind.Replaced && !await ReplacementsHoldAsync(paths, entry, cancellationToken).ConfigureAwait(false))
{
Log.Warning("archive", $"Trash entry {entry.Name} is kept: a replacement of its originals no longer matches");
return false;
}
var aside = Path.Combine(TrashDirectory(paths), ErasingFolder, Guid.NewGuid().ToString("N"));
Directory.CreateDirectory(Path.GetDirectoryName(aside)!);
Directory.Move(folder, aside);
File.Delete(record);
Erase(aside);
Log.Info("archive", $"Trash entry {entry.Name} ({entry.Title}, {entry.Files} files, {entry.Bytes} bytes) erased");
return true;
}
}
/// <summary>
/// Whether every photo of a replaced entry still has its replacement in the sequence: a frame holding the replacement's
/// SHA-256 whose file hashes to it now. The originals are erased only then (a later phase's conversion).
/// </summary>
private async Task<bool> ReplacementsHoldAsync(ArchivePaths paths, TrashEntry entry, CancellationToken cancellationToken)
{
if (FrameTrash.ReadEntry(paths, entry.Name) is not { } record) return false;
foreach (var photo in record.Photos)
{
if (photo.Replacement is not { } replacement) return false;
var holders = (await _db.Frames.FindBySha256Async(replacement.ToLowerInvariant(), cancellationToken).ConfigureAwait(false))
.Where(f => f.SequenceId == record.SequenceId).ToList();
var verified = false;
foreach (var frame in holders)
{
var file = paths.FromRelative(frame.RelativePath);
if (!File.Exists(file)) continue;
if (string.Equals(await ContentHash.Sha256Async(file, null, cancellationToken).ConfigureAwait(false), replacement, StringComparison.OrdinalIgnoreCase))
{
verified = true;
break;
}
}
if (!verified) return false;
}
return true;
}
/// <summary>The folder and record of an entry of either kind.</summary>
private static (string Folder, string Record) Location(ArchivePaths paths, TrashEntry entry)
{
if (FrameTrash.IsPhotoEntry(entry.Name)) return FrameTrash.Location(paths, entry.Name);
var folder = Path.Combine(TrashDirectory(paths), Path.GetFileName(entry.Name));
return (folder, folder + RecordExtension);
}
/// <summary>Deletes what an interrupted erase left aside: those entries were confirmed and had left the listing already.</summary>
private static void SweepErasing(ArchivePaths paths)
{
var erasing = Path.Combine(TrashDirectory(paths), ErasingFolder);
if (!Directory.Exists(erasing)) return;
foreach (var leftover in Directory.EnumerateDirectories(erasing)) Erase(leftover);
}
/// <summary>Deletes a folder and everything in it, read-only files included; one that is gone already is fine.</summary>
internal static void Erase(string folder)
{
if (!Directory.Exists(folder)) return;
foreach (var file in new DirectoryInfo(folder).EnumerateFiles("*", new EnumerationOptions { RecurseSubdirectories = true, AttributesToSkip = 0 }))
{
if ((file.Attributes & FileAttributes.ReadOnly) != 0) file.Attributes &= ~FileAttributes.ReadOnly;
}
Directory.Delete(folder, recursive: true);
}
}
+33 -7
View File
@@ -48,11 +48,27 @@ public sealed record DeletePlan(IReadOnlyList<DeleteStep> Steps, IReadOnlyList<B
public int Renders => Steps.Sum(s => s.Sequence.RenderedCount);
}
/// <summary>A deleted sequence kept in an archive's trash, as <see cref="SequenceTrash.ListAsync"/> lists it.</summary>
/// <param name="Name">Folder name inside the trash; pass it to <see cref="SequenceTrash.RestoreAsync"/>.</param>
/// <summary>What a trash entry holds (owner review 2, R6).</summary>
public enum TrashKind
{
/// <summary>A whole sequence folder (<see cref="SequenceTrash"/>).</summary>
Sequence,
/// <summary>Photos deleted from a sequence (<see cref="FrameTrash"/>).</summary>
Photos,
/// <summary>Originals a conversion replaced (a later phase): erased only while their replacements still match.</summary>
Replaced,
}
/// <summary>An entry of an archive's trash, as <see cref="SequenceTrash.ListAsync"/> lists it: a deleted sequence or deleted photos.</summary>
/// <param name="Name">The entry's name: the folder name inside the trash for a sequence, <c>photos-</c> and the folder name for
/// photos; pass it to <see cref="ArchiveService.RestoreEntryAsync"/>.</param>
/// <param name="Title">The sequence's title (the one the photos came from).</param>
/// <param name="Folder">Archive-relative folder the sequence came from (and returns to when it is free).</param>
/// <param name="Frames">Photos the entry holds.</param>
public sealed record TrashEntry(Guid ArchiveId, string Name, Guid SequenceId, string Title, string Folder, int Frames, int Files, long Bytes,
DateTimeOffset DeletedAt, string? DeletedBy);
DateTimeOffset DeletedAt, string? DeletedBy, TrashKind Kind = TrashKind.Sequence);
/// <summary>The plan no longer matches the index or the jobs (a sequence moved, vanished or got busy since it was made); nothing was deleted.</summary>
public sealed class StaleDeletePlanException(Guid sequenceId) : InvalidOperationException(Loc.L("Archive.Error.StalePlan"))
@@ -75,7 +91,7 @@ public sealed class SequenceInLibraryException(Guid sequenceId, string title) :
/// moved, its index step runs to the end whatever the caller cancels; when that step fails, the folder moves back, so
/// the trash and the index never disagree.
/// </summary>
public sealed class SequenceTrash
public sealed partial class SequenceTrash
{
/// <summary>Folder of the trash inside the archive's <c>.chronos</c>.</summary>
public const string FolderName = "trash";
@@ -168,7 +184,10 @@ public sealed class SequenceTrash
return entries;
}
/// <summary>The sequences in an archive's trash, newest first; records whose folder is gone are left out.</summary>
/// <summary>
/// The entries of an archive's trash, newest first: deleted sequences (<c>trash/*.json</c>) and deleted photos
/// (<c>trash/photos/*.json</c>); records whose folder is gone are left out.
/// </summary>
public async Task<IReadOnlyList<TrashEntry>> ListAsync(Archive archive, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(archive);
@@ -184,7 +203,8 @@ public sealed class SequenceTrash
if (!Directory.Exists(Path.Combine(trash, name)) || ReadRecord(path) is not { } record) continue;
entries.Add(Entry(archive.Id, name, record));
}
return entries.OrderByDescending(e => e.DeletedAt).ToList();
entries.AddRange(FrameTrash.List(archive, cancellationToken));
return entries.OrderByDescending(e => e.DeletedAt).ThenBy(e => e.Name, StringComparer.Ordinal).ToList();
}, cancellationToken).ConfigureAwait(false);
}
@@ -215,8 +235,12 @@ public sealed class SequenceTrash
.ToHashSet(StringComparer.OrdinalIgnoreCase);
var folder = FreeFolder(paths, record.Sequence.Folder, locations);
var sequence = paths.Sequence(folder);
// The entry's lease keeps an emptying of the trash out meanwhile.
using (await LeaseFile.AcquireAsync(paths.LocksDirectory, EntryLease(Path.GetFileName(name)), _writer.Options.Owner, _writer.Options.LeaseWait,
_writer.Options.LeaseDuration, cancellationToken).ConfigureAwait(false))
await using (await _writer.LockAsync(paths, id, cancellationToken).ConfigureAwait(false))
{
if (!Directory.Exists(source)) throw new DirectoryNotFoundException($"{name} is not in the trash of {archive.Name}.");
cancellationToken.ThrowIfCancellationRequested();
Directory.CreateDirectory(Path.GetDirectoryName(sequence.Folder)!);
Directory.Move(source, sequence.Folder);
@@ -269,6 +293,7 @@ public sealed class SequenceTrash
await using var tx = await _db.BeginAsync(cancellationToken).ConfigureAwait(false);
await tx.Sequences.UpsertAsync(SequenceMapping.ToSequence(manifest, archive.Id, sequence, kept), cancellationToken).ConfigureAwait(false);
await tx.Frames.SyncSequenceAsync(deleted.Id, SequenceMapping.ToFrames(manifest, sequence), cancellationToken).ConfigureAwait(false);
await tx.FrameRemovals.SyncSequenceAsync(deleted.Id, SequenceMapping.ToRemovals(manifest), cancellationToken).ConfigureAwait(false);
foreach (var key in manifest.SessionKeys) await tx.Sequences.MapSessionKeyAsync(archive.Id, key, deleted.Id, cancellationToken).ConfigureAwait(false);
foreach (var render in record.Renders)
{
@@ -426,7 +451,7 @@ public sealed class SequenceTrash
/// archives of archive-wide ones (scan, rebuild, import, maintenance, a verify of everything). An import's folders
/// are known only to its plan, so it holds its whole archive; a verify of named sequences holds only those.
/// </summary>
private async Task<IReadOnlySet<Guid>> BusyAsync(CancellationToken cancellationToken)
internal async Task<IReadOnlySet<Guid>> BusyAsync(CancellationToken cancellationToken)
{
var busy = new HashSet<Guid>();
foreach (var state in new[] { JobState.Running, JobState.Paused })
@@ -551,4 +576,5 @@ internal sealed record TrashRecord(Sequence Sequence, IReadOnlyList<Render> Rend
[JsonSourceGenerationOptions(PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase, DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
UseStringEnumConverter = true, IgnoreReadOnlyProperties = true)]
[JsonSerializable(typeof(TrashRecord))]
[JsonSerializable(typeof(PhotoTrashRecord))]
internal sealed partial class TrashJson : JsonSerializerContext;
+3 -1
View File
@@ -88,7 +88,8 @@ internal sealed class SequenceWriter(ArchiveServiceOptions options)
}
/// <summary>
/// Reads the sidecar of a locked sequence, completing an interrupted ingest batch first.
/// Reads the sidecar of a locked sequence, undoing an interrupted photo deletion (<see cref="FrameRemovalJournal"/>) and
/// completing an interrupted ingest batch first.
/// </summary>
/// <exception cref="InvalidDataException">The sidecar in the folder belongs to another sequence.</exception>
public static SequenceManifest ReadLocked(SequencePaths sequence, Guid sequenceId)
@@ -96,6 +97,7 @@ internal sealed class SequenceWriter(ArchiveServiceOptions options)
var manifest = ManifestStore.ReadSequence(sequence.ManifestPath);
if (manifest.Id != sequenceId)
throw new InvalidDataException($"{sequence.ManifestPath} belongs to sequence {manifest.Id}, not {sequenceId}.");
FrameRemovalJournal.Recover(sequence, manifest);
if (IngestJournal.Recover(sequence, manifest) is { } recovered)
{
ManifestStore.WriteSequence(sequence.ManifestPath, recovered);
+4
View File
@@ -23,6 +23,7 @@ public abstract class ChronosRepositories
Audit = new AuditRepository(access);
Uploads = new UploadRepository(access);
Schedules = new ScheduleRepository(access);
FrameRemovals = new FrameRemovalRepository(access);
}
public ArchiveRepository Archives { get; }
@@ -41,6 +42,9 @@ public abstract class ChronosRepositories
/// <summary>Scheduled jobs (owner review 2, R10).</summary>
public ScheduleRepository Schedules { get; }
/// <summary>Tombstones of the photos removed from sequences (owner review 2, R6).</summary>
public FrameRemovalRepository FrameRemovals { get; }
}
/// <summary>
@@ -0,0 +1,14 @@
-- Tombstones of photos removed from sequences (owner review 2, R6): a photo deleted into the archive's trash (reason 0)
-- or replaced by a converted file (reason 1, with the replacement's SHA-256). The sidecar's "removed" list is the truth;
-- the index writes these rows from it, so a rebuild keeps them. Sync never offers a tombstoned photo back, and an upload
-- of that content into the sequence is answered as already there. Mirrors sqlite/0008_frame_removals.sql.
CREATE TABLE frame_removals (
sequence_id uuid NOT NULL REFERENCES sequences (id) ON DELETE CASCADE,
sha256 text NOT NULL,
replacement_sha256 text,
reason integer NOT NULL DEFAULT 0,
removed_at timestamptz NOT NULL,
PRIMARY KEY (sequence_id, sha256)
);
CREATE INDEX ix_frame_removals_sha256 ON frame_removals (sha256);
@@ -0,0 +1,14 @@
-- Tombstones of photos removed from sequences (owner review 2, R6): a photo deleted into the archive's trash (reason 0)
-- or replaced by a converted file (reason 1, with the replacement's SHA-256). The sidecar's "removed" list is the truth;
-- the index writes these rows from it, so a rebuild keeps them. Sync never offers a tombstoned photo back, and an upload
-- of that content into the sequence is answered as already there. Mirrors postgres/0008_frame_removals.sql.
CREATE TABLE frame_removals (
sequence_id TEXT NOT NULL REFERENCES sequences (id) ON DELETE CASCADE,
sha256 TEXT NOT NULL,
replacement_sha256 TEXT,
reason INTEGER NOT NULL DEFAULT 0,
removed_at TEXT NOT NULL,
PRIMARY KEY (sequence_id, sha256)
);
CREATE INDEX ix_frame_removals_sha256 ON frame_removals (sha256);
@@ -0,0 +1,71 @@
using System.Globalization;
using System.Text;
using Chronos.Core.Model;
namespace Chronos.Data.Repositories;
/// <summary>
/// Tombstones of the photos removed from sequences (owner review 2, R6; <c>frame_removals</c>), by sequence and SHA-256.
/// The sidecar's <c>removed</c> list is the truth: the index writes a sequence's rows from it whenever it takes the
/// sidecar (<see cref="SyncSequenceAsync"/>), so a rebuild keeps them, and they go with the sequence's row.
/// </summary>
public sealed class FrameRemovalRepository
{
private const string Columns = "sequence_id, sha256, replacement_sha256, reason, removed_at";
/// <summary>Rows one INSERT writes (five parameters each, well inside both engines' limits).</summary>
private const int InsertRows = 200;
private readonly DbAccess _db;
internal FrameRemovalRepository(DbAccess db) => _db = db;
/// <summary>The tombstones of a sequence, oldest first.</summary>
public Task<IReadOnlyList<FrameRemoval>> ListBySequenceAsync(Guid sequenceId, CancellationToken cancellationToken = default) =>
_db.ListAsync($"SELECT {Columns} FROM frame_removals WHERE sequence_id = $1 ORDER BY removed_at, sha256", [sequenceId], Map, cancellationToken);
/// <summary>Whether <paramref name="sha256"/> was removed from the sequence (any reason).</summary>
public async Task<bool> IsRemovedAsync(Guid sequenceId, string sha256, CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrEmpty(sha256);
return await _db.ScalarAsync<long>("SELECT count(*) FROM frame_removals WHERE sequence_id = $1 AND sha256 = $2",
[sequenceId, sha256.ToLowerInvariant()], cancellationToken).ConfigureAwait(false) > 0;
}
/// <summary>
/// Replaces the tombstones of a sequence with <paramref name="removals"/> (the sidecar's list; one per SHA-256, the
/// last one given wins). Returns how many rows the sequence has now.
/// </summary>
public Task<int> SyncSequenceAsync(Guid sequenceId, IReadOnlyList<FrameRemoval> removals, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(removals);
var rows = removals.Where(r => !string.IsNullOrEmpty(r.Sha256)).GroupBy(r => r.Sha256.ToLowerInvariant(), StringComparer.Ordinal)
.Select(g => g.Last() with { Sha256 = g.Key, SequenceId = sequenceId }).ToList();
return _db.InTransactionAsync(async session =>
{
await session.ExecuteAsync("DELETE FROM frame_removals WHERE sequence_id = $1", [sequenceId], cancellationToken).ConfigureAwait(false);
for (var start = 0; start < rows.Count; start += InsertRows)
{
var count = Math.Min(InsertRows, rows.Count - start);
var sql = new StringBuilder($"INSERT INTO frame_removals ({Columns}) VALUES ");
var args = new DbParam[count * 5];
for (var i = 0; i < count; i++)
{
var row = rows[start + i];
sql.Append(i > 0 ? ", (" : "(").Append(string.Join(", ", Enumerable.Range(i * 5 + 1, 5).Select(n => "$" + n.ToString(CultureInfo.InvariantCulture))))
.Append(')');
args[i * 5] = row.SequenceId;
args[i * 5 + 1] = row.Sha256;
args[i * 5 + 2] = row.ReplacementSha256?.ToLowerInvariant();
args[i * 5 + 3] = (int)row.Reason;
args[i * 5 + 4] = row.RemovedAt;
}
await session.ExecuteAsync(sql.ToString(), args, cancellationToken).ConfigureAwait(false);
}
return rows.Count;
}, cancellationToken);
}
private static FrameRemoval Map(DbRow r) =>
new(r.GetGuid(0), r.GetString(1), r.GetStringOrNull(2), (RemovalReason)r.GetInt32(3), r.GetDateTimeOffset(4));
}
+14 -3
View File
@@ -24,6 +24,9 @@ internal sealed class SyncPositions(ChronosDatabase db, ISyncRemote remote, Guid
/// <summary>The manifest's frames by hash, and with their capture times once fetched.</summary>
private FramePhotos? _hashed, _timed;
/// <summary>The photos either copy removed: a replaced photo is translated as its replacement (owner review 2, R6).</summary>
private SyncTombstones? _tombstones;
/// <summary>The server's frames version the translation to the server's frames is made against.</summary>
public int ServerFrames => manifest.FramesVersion;
@@ -73,7 +76,10 @@ internal sealed class SyncPositions(ChronosDatabase db, ISyncRemote remote, Guid
var frames = await SequenceFrames.ListAsync(db, localId, cancellationToken).ConfigureAwait(false);
if (await db.Sequences.GetAsync(localId, cancellationToken).ConfigureAwait(false) is not { } after) return null;
if (after.FramesVersion == before.FramesVersion)
return new LocalState(after, new FramePhotos(frames.Select(f => (f.Index, new FramePhoto(f.Sha256, f.Size, f.CapturedAt)))));
{
var tombstones = await TombstonesAsync(cancellationToken).ConfigureAwait(false);
return new LocalState(after, new FramePhotos(frames.Select(f => (f.Index, tombstones.Canonical(new FramePhoto(f.Sha256, f.Size, f.CapturedAt))))));
}
}
return null;
}
@@ -87,20 +93,25 @@ internal sealed class SyncPositions(ChronosDatabase db, ISyncRemote remote, Guid
/// </summary>
private async Task<FramePhotos?> ServerAsync(FramePhotos local, CancellationToken cancellationToken)
{
var tombstones = await TombstonesAsync(cancellationToken).ConfigureAwait(false);
if (_hashed is null)
{
if (!OneList(manifest.Frames)) return null;
_hashed = new FramePhotos(manifest.Frames.Select(f => (f.Index, new FramePhoto(f.Sha256, f.Size, null))));
_hashed = new FramePhotos(manifest.Frames.Select(f => (f.Index, tombstones.Canonical(new FramePhoto(f.Sha256, f.Size, null)))));
}
if (_hashed.SameAs(local)) return _hashed;
if (_timed is null)
{
var captured = (await remote.ListFramesAsync(manifest.SequenceId, cancellationToken).ConfigureAwait(false)).ToDictionary(f => f.Id, f => f.CapturedAt);
_timed = new FramePhotos(manifest.Frames.Select(f => (f.Index, new FramePhoto(f.Sha256, f.Size, captured.GetValueOrDefault(f.Id)))));
_timed = new FramePhotos(manifest.Frames.Select(f => (f.Index, tombstones.Canonical(new FramePhoto(f.Sha256, f.Size, captured.GetValueOrDefault(f.Id))))));
}
return _timed;
}
/// <summary>This copy's tombstones and the server's, read once.</summary>
private async Task<SyncTombstones> TombstonesAsync(CancellationToken cancellationToken) =>
_tombstones ??= new SyncTombstones(await db.FrameRemovals.ListBySequenceAsync(localId, cancellationToken).ConfigureAwait(false), manifest.Removed);
/// <summary>Each frame once, at the indices 0…n-1 a sequence's frames always take.</summary>
private static bool OneList(IReadOnlyList<ManifestFrame> frames)
{
+16
View File
@@ -164,6 +164,22 @@ public static class SyncRules
];
}
/// <summary>
/// <see cref="CompareOriginals(IReadOnlyCollection{FramePhoto}, IReadOnlyCollection{FramePhoto})"/> honouring the photos
/// either side removed (owner review 2, R6): a photo the server deleted is not missing there, one this PC deleted is not
/// missing here, and a photo replaced by a converted file on either side is compared as its replacement.
/// </summary>
public static (int MissingRemote, int MissingLocal) CompareOriginals(IReadOnlyCollection<FramePhoto> local, IReadOnlyCollection<FramePhoto> remote,
SyncTombstones tombstones)
{
ArgumentNullException.ThrowIfNull(local);
ArgumentNullException.ThrowIfNull(remote);
ArgumentNullException.ThrowIfNull(tombstones);
if (tombstones.IsEmpty) return CompareOriginals(local, remote);
return CompareOriginals([.. local.Where(p => !tombstones.DeletedRemotely(p.Sha256)).Select(tombstones.Canonical)],
[.. remote.Where(p => !tombstones.DeletedLocally(p.Sha256)).Select(tombstones.Canonical)]);
}
/// <summary>
/// Frames whose proxy the server asks for (it cannot decode their originals, BRIEF §2.3) that this PC holds and its
/// <paramref name="decoder"/> reads: an upload makes and sends them. A local original not hashed yet (a folder adopted
+3 -1
View File
@@ -528,8 +528,10 @@ public sealed class SyncService
var captured = manifest.Frames.Any(f => f.Sha256.Length == 0) || present.Any(f => f.Sha256 is null)
? (await remote.ListFramesAsync(link.RemoteId, cancellationToken).ConfigureAwait(false)).ToDictionary(f => f.Id, f => f.CapturedAt)
: null;
// Photos either side removed are not offered back; one replaced by a converted file counts as its replacement (owner review 2, R6).
var tombstones = new SyncTombstones(await _db.FrameRemovals.ListBySequenceAsync(local.Id, cancellationToken).ConfigureAwait(false), manifest.Removed);
var (missingRemote, missingLocal) = SyncRules.CompareOriginals([.. present.Select(f => new FramePhoto(f.Sha256, f.Size, f.CapturedAt))],
[.. manifest.Frames.Select(f => new FramePhoto(f.Sha256, f.Size, captured?.GetValueOrDefault(f.Id)))]);
[.. manifest.Frames.Select(f => new FramePhoto(f.Sha256, f.Size, captured?.GetValueOrDefault(f.Id)))], tombstones);
// Proxies the server asks for that this PC can make from its originals travel with an upload too (BRIEF §2.3).
var proxies = policy.CanEdit && manifest.Proxies is not null ? SyncRules.ProxiesToSend(manifest, present, Options.Archive.Decoder, captured) : 0;
// Those change without the server's frame counters moving: while some may be made here, the server's next
+81
View File
@@ -0,0 +1,81 @@
using Chronos.Core.Model;
using Chronos.Core.Project;
using Chronos.Protocol;
namespace Chronos.Data.Sync;
/// <summary>
/// The photos each copy of a linked pair removed (owner review 2, R6; BRIEF §14.5): a photo one side deleted is not
/// offered to it again (no upload of what the server deleted, no download of what this PC deleted; local files are never
/// deleted by sync), and a photo one side replaced by a converted file is the same photo as its replacement on either side
/// (an alias: both are compared, matched and translated as the replacement). No I/O.
/// </summary>
public sealed class SyncTombstones
{
/// <summary>Links a replacement chain may have at most (a conversion of a conversion…); longer ones stop there.</summary>
private const int MaxAliasSteps = 8;
private readonly HashSet<string> _deletedLocally;
private readonly HashSet<string> _deletedRemotely;
private readonly Dictionary<string, string> _aliases;
public SyncTombstones(IEnumerable<FrameRemoval> local, IEnumerable<RemovedPhotoDto>? remote)
{
ArgumentNullException.ThrowIfNull(local);
_deletedLocally = new HashSet<string>(StringComparer.Ordinal);
_deletedRemotely = new HashSet<string>(StringComparer.Ordinal);
_aliases = new Dictionary<string, string>(StringComparer.Ordinal);
foreach (var removal in local) Add(removal.Sha256, removal.ReplacementSha256, removal.Reason == RemovalReason.Replaced, _deletedLocally);
foreach (var removal in remote ?? [])
Add(removal.Sha256, removal.Replacement, string.Equals(removal.Reason, Wire(RemovalReason.Replaced), StringComparison.OrdinalIgnoreCase), _deletedRemotely);
}
/// <summary>No photo removed on either side.</summary>
public static SyncTombstones None { get; } = new([], null);
/// <summary>Whether either side removed or replaced anything.</summary>
public bool IsEmpty => _deletedLocally.Count == 0 && _deletedRemotely.Count == 0 && _aliases.Count == 0;
/// <summary>The wire name of a removal reason (<c>deleted</c>, <c>replaced</c>).</summary>
public static string Wire(RemovalReason reason) => reason.ToString().ToLowerInvariant();
/// <summary>The tombstones of a sequence as the manifest carries them.</summary>
public static IReadOnlyList<RemovedPhotoDto> ToDto(IEnumerable<FrameRemoval> removals) =>
[.. removals.Select(r => new RemovedPhotoDto(r.Sha256, r.ReplacementSha256, Wire(r.Reason), r.RemovedAt))];
/// <summary>
/// The SHA-256 a photo is compared as: the end of its replacement chain on either side, lower-case; null or empty
/// (not hashed yet) stays as it is.
/// </summary>
public string? Canonical(string? sha256)
{
if (string.IsNullOrEmpty(sha256)) return sha256;
var current = sha256.ToLowerInvariant();
for (var step = 0; step < MaxAliasSteps && _aliases.TryGetValue(current, out var next); step++) current = next;
return current;
}
/// <summary>The photo as it is compared (<see cref="Canonical(string?)"/>).</summary>
public FramePhoto Canonical(FramePhoto photo)
{
ArgumentNullException.ThrowIfNull(photo);
return photo.IsHashed && _aliases.Count > 0 ? photo with { Sha256 = Canonical(photo.Sha256) } : photo;
}
/// <summary>This PC deleted the photo (or what replaced it): the server's copy is not downloaded.</summary>
public bool DeletedLocally(string? sha256) => Deleted(sha256, _deletedLocally);
/// <summary>The server deleted the photo (or what replaced it): this PC's copy is not uploaded.</summary>
public bool DeletedRemotely(string? sha256) => Deleted(sha256, _deletedRemotely);
private bool Deleted(string? sha256, HashSet<string> deleted) =>
!string.IsNullOrEmpty(sha256) && deleted.Count > 0 && (deleted.Contains(sha256.ToLowerInvariant()) || deleted.Contains(Canonical(sha256)!));
private void Add(string sha256, string? replacement, bool replaced, HashSet<string> deleted)
{
if (string.IsNullOrEmpty(sha256)) return;
var sha = sha256.ToLowerInvariant();
if (replaced && !string.IsNullOrEmpty(replacement)) _aliases[sha] = replacement.ToLowerInvariant();
else deleted.Add(sha);
}
}
+9 -3
View File
@@ -48,13 +48,15 @@ internal sealed class SyncTransfers(SyncService sync)
var frames = (await SequenceFrames.ListAsync(db, local.Id, ct).ConfigureAwait(false)).Where(f => (f.Flags & FrameFlags.Missing) == 0).ToList();
var manifest = remoteId is { } known ? await remote.GetManifestAsync(known, ct).ConfigureAwait(false) : null;
var present = manifest?.Frames.Where(f => f.Sha256.Length > 0).Select(f => f.Sha256.ToLowerInvariant()).ToHashSet(StringComparer.Ordinal) ?? [];
// A photo the server deleted is not sent back; one it replaced by a converted file is there as its replacement (owner review 2, R6).
var tombstones = new SyncTombstones(await db.FrameRemovals.ListBySequenceAsync(local.Id, ct).ConfigureAwait(false), manifest?.Removed);
var present = manifest?.Frames.Where(f => f.Sha256.Length > 0).Select(f => tombstones.Canonical(f.Sha256)!).ToHashSet(StringComparer.Ordinal) ?? [];
// Originals the server holds without a hash (adopted there, not verified) are the ones of their size and capture
// time: the server could not recognise them by content, so they are not sent again.
var adopted = manifest is not null && manifest.Frames.Any(f => f.Sha256.Length == 0)
? await UnhashedAsync(remote, manifest, ct).ConfigureAwait(false)
: new FramePhotos([]);
var pending = new Queue<Frame>(frames.Where(f => (f.Sha256 is null || !present.Contains(f.Sha256.ToLowerInvariant()))
var pending = new Queue<Frame>(frames.Where(f => (f.Sha256 is null || (!present.Contains(tombstones.Canonical(f.Sha256)!) && !tombstones.DeletedRemotely(f.Sha256)))
&& adopted.IndexOf(new FramePhoto(f.Sha256, f.Size, f.CapturedAt)) is null));
var progress = new TransferProgress(context, frames.Count, frames.Count - pending.Count);
await progress.ReportAsync().ConfigureAwait(false);
@@ -205,12 +207,16 @@ internal sealed class SyncTransfers(SyncService sync)
var manifest = await remote.GetManifestAsync(remoteId, ct).ConfigureAwait(false);
var captured = (await remote.ListFramesAsync(remoteId, ct).ConfigureAwait(false)).ToDictionary(f => f.Id, f => f.CapturedAt);
var (have, restore, photos) = local is null ? ([], [], new FramePhotos([])) : await LocalContentAsync(archive, local, ct).ConfigureAwait(false);
// A photo this PC deleted is not fetched again; one replaced here by a converted file is here as its replacement (owner review 2, R6).
var tombstones = local is null ? SyncTombstones.None
: new SyncTombstones(await db.FrameRemovals.ListBySequenceAsync(local.Id, ct).ConfigureAwait(false), manifest.Removed);
if (!tombstones.IsEmpty) have = [.. have.Select(h => tombstones.Canonical(h)!)];
// An original the server has not hashed yet (adopted there, not verified) is here when an original of its size and
// capture time is; each one is fetched on its own and hashed here.
var wanted = manifest.Frames
.Where(f => f.Sha256.Length == 0
? photos.IndexOf(new FramePhoto(null, f.Size, captured.GetValueOrDefault(f.Id))) is null
: !have.Contains(f.Sha256.ToLowerInvariant()))
: !have.Contains(tombstones.Canonical(f.Sha256)!) && !tombstones.DeletedLocally(f.Sha256))
.DistinctBy(f => f.Sha256.Length == 0 ? f.Id.ToString("N") : f.Sha256.ToLowerInvariant()).ToList();
EnsureSpace(paths.Root, wanted.Sum(f => f.Size));
+67
View File
@@ -0,0 +1,67 @@
namespace Chronos.Protocol;
/// <summary>
/// <c>POST /api/v1/sequences/{id}/frames/delete</c> (owner review 2, R6): the photos to move into the archive's trash.
/// <see cref="ExpectedFramesVersion"/> is the frames version the ids were read at: a sequence whose frames changed since
/// answers 409 <c>frames_conflict</c>. <see cref="DryRun"/> only reports what would happen.
/// </summary>
public sealed record FrameDeleteRequest(IReadOnlyList<Guid> FrameIds, int? ExpectedFramesVersion, bool DryRun);
/// <summary>
/// What a photo deletion does (dry run) or did: the photos and bytes that go, the sequence's frames version (as it is on a
/// dry run, after the deletion otherwise) and the trash entry made (null on a dry run).
/// </summary>
public sealed record FrameDeleteResultDto(Guid SequenceId, int Photos, long Bytes, int FramesVersion, int FrameCount, string? TrashEntry, bool DryRun);
/// <summary>
/// An entry of the archive's trash (<c>GET /api/v1/archive/trash</c>): a deleted sequence (<see cref="Kind"/> <c>sequence</c>)
/// or photos deleted from one (<c>photos</c>; <c>replaced</c> for originals a conversion replaced). <see cref="Name"/>
/// identifies it in restore and empty requests.
/// </summary>
public sealed record TrashEntryDto(string Name, string Kind, Guid SequenceId, string Title, string Folder, int Photos, int Files, long Bytes,
string? DeletedBy, DateTimeOffset DeletedAt);
/// <summary><c>GET /api/v1/archive/trash</c>: the entries, newest first, and their total size.</summary>
public sealed record TrashListDto(IReadOnlyList<TrashEntryDto> Entries, long Bytes);
/// <summary>
/// <c>POST /api/v1/archive/trash/{name}/restore</c>: the sequence the entry went back into (a new one when the photos'
/// sequence is gone: <see cref="NewSequence"/>), how many photos came back, and the files kept in the trash because they
/// no longer match their record.
/// </summary>
public sealed record TrashRestoreDto(string Name, string Kind, Guid SequenceId, int Photos, IReadOnlyList<string> Kept, bool NewSequence);
/// <summary>
/// <c>POST /api/v1/archive/trash/empty</c>: erase the entries <see cref="Names"/> (null: every entry). A dry run answers the
/// plan with its <see cref="TrashEmptyPlanDto.Fingerprint"/>; the erase itself must send that fingerprint back (409
/// <c>trash_changed</c> when the trash changed since) and runs as an <c>empty-trash</c> job (202).
/// </summary>
public sealed record TrashEmptyRequest(IReadOnlyList<string>? Names, bool DryRun, string? Fingerprint);
/// <summary>The dry run of emptying the trash: the entries measured on disk, their totals, the fingerprint, and names not in the trash.</summary>
public sealed record TrashEmptyPlanDto(IReadOnlyList<TrashEntryDto> Entries, int Photos, int Files, long Bytes, string Fingerprint, IReadOnlyList<string> Unknown);
/// <summary><c>POST /api/v1/sequences/bulk/delete</c>: move these sequences into the trash (or, with <see cref="DryRun"/>, say what would).</summary>
public sealed record BulkDeleteRequest(IReadOnlyList<Guid> Ids, bool DryRun);
/// <summary><c>POST /api/v1/sequences/bulk/retag</c>: add and remove tags on these sequences (the same edit as a PATCH of their tags).</summary>
public sealed record BulkRetagRequest(IReadOnlyList<Guid> Ids, IReadOnlyList<string>? Add, IReadOnlyList<string>? Remove);
/// <summary>
/// <c>POST /api/v1/sequences/bulk/jobs</c>: queue one job of <see cref="Kind"/> (<c>analyse</c>, <c>deep</c>, <c>proxies</c>,
/// <c>thumbnails</c>, <c>verify</c>) over these sequences.
/// </summary>
public sealed record BulkJobsRequest(IReadOnlyList<Guid> Ids, string Kind);
/// <summary>
/// One sequence of a bulk operation: <see cref="Outcome"/> <c>ok</c>, or <c>skipped</c> with the error code that says why
/// (<see cref="Reason"/>, e.g. <c>not_found</c>, <c>forbidden</c>, <c>sequence_busy</c>) and its localised sentence.
/// </summary>
/// <param name="TrashFolder">For a delete: where the folder goes (or went) in the trash; null when it has no folder.</param>
public sealed record BulkItemResult(Guid Id, string Outcome, string? Reason = null, string? Message = null, string? TrashFolder = null);
/// <summary>
/// The result of a bulk operation: one item per id asked for, the counts, the photos and bytes a delete moves (or moved),
/// and the job a bulk job request queued (null when nothing was queued).
/// </summary>
public sealed record BulkResultDto(IReadOnlyList<BulkItemResult> Items, int Ok, int Skipped, bool DryRun, int Photos = 0, long Bytes = 0, JobDto? Job = null);
+13
View File
@@ -73,4 +73,17 @@ namespace Chronos.Protocol;
[JsonSerializable(typeof(CreateScheduleRequest))]
[JsonSerializable(typeof(UpdateScheduleRequest))]
[JsonSerializable(typeof(ScheduleRunDto))]
[JsonSerializable(typeof(RemovedPhotoDto))]
[JsonSerializable(typeof(FrameDeleteRequest))]
[JsonSerializable(typeof(FrameDeleteResultDto))]
[JsonSerializable(typeof(TrashEntryDto))]
[JsonSerializable(typeof(TrashListDto))]
[JsonSerializable(typeof(TrashRestoreDto))]
[JsonSerializable(typeof(TrashEmptyRequest))]
[JsonSerializable(typeof(TrashEmptyPlanDto))]
[JsonSerializable(typeof(BulkDeleteRequest))]
[JsonSerializable(typeof(BulkRetagRequest))]
[JsonSerializable(typeof(BulkJobsRequest))]
[JsonSerializable(typeof(BulkItemResult))]
[JsonSerializable(typeof(BulkResultDto))]
public sealed partial class ProtocolJson : JsonSerializerContext;
+12 -1
View File
@@ -75,6 +75,10 @@ public sealed record FrameDto(
/// <summary><c>GET /api/v1/sequences/{id}/manifest</c>: what a client needs to diff a sequence (BRIEF §14.5).</summary>
/// <param name="Proxies">How to make the proxies the server asks for (<see cref="ManifestFrame.WantsProxy"/>); null when it makes none.</param>
/// <param name="Removed">
/// Tombstones of the photos removed from the sequence (owner review 2, R6): a client never offers them back, and counts a
/// replaced photo and its replacement as one; null from servers before owner review 2 and when there are none.
/// </param>
public sealed record SequenceManifest(
Guid SequenceId,
int FramesVersion,
@@ -83,7 +87,14 @@ public sealed record SequenceManifest(
IReadOnlyList<string> Tags,
IReadOnlyList<ManifestFrame> Frames,
IReadOnlyList<RenderDto> Renders,
ProxySpec? Proxies = null);
ProxySpec? Proxies = null,
IReadOnlyList<RemovedPhotoDto>? Removed = null);
/// <summary>
/// A photo removed from a sequence (owner review 2, R6): its lower-case hex SHA-256, the replacement's when a conversion
/// replaced it, why (<c>deleted</c>, <c>replaced</c>) and when.
/// </summary>
public sealed record RemovedPhotoDto(string Sha256, string? Replacement, string Reason, DateTimeOffset At);
/// <summary>One frame of a manifest: its content, whether the server has a proxy and thumbnail of it, and its flags.</summary>
/// <param name="Sha256">Lower-case hex SHA-256; empty while the server has not hashed it (a folder adopted and not verified yet).</param>
+8
View File
@@ -114,6 +114,14 @@ public static class JobEndpoints
return ApiErrors.BadRequest(context, "invalid_parameters", "parametersJson");
}
if (kind == JobKind.Verify)
{
// A verify of sequences targets their archive and names them in its parameters, as its handler reads it.
var verify = LibraryEndpoints.SequenceJob(kind, archive.Current, targets, label);
targets = verify.Targets;
queue = [(label, verify.ParametersJson)];
}
Job? first = null;
foreach (var (jobLabel, parameters) in queue)
{
+377
View File
@@ -0,0 +1,377 @@
using System.Globalization;
using Microsoft.AspNetCore.Mvc;
using Chronos.Core.Archiving;
using Chronos.Core.Imaging.Decoders;
using Chronos.Core.Model;
using Chronos.Data;
using Chronos.Data.Archiving;
using Chronos.Data.Archiving.Jobs;
using Chronos.Data.Jobs;
using Chronos.Data.Processing;
using Chronos.Data.Repositories;
using Chronos.Protocol;
using Chronos.Server.Events;
using Chronos.Server.Jobs;
using Chronos.Server.Security;
using Chronos.Server.Storage;
namespace Chronos.Server.Api;
/// <summary>
/// The library the admin panel manages without editing (owner review 2, R6): a sequence's cover thumbnail, deleting photos
/// of a sequence into the archive's trash (<see cref="FrameTrash"/>), and mass operations on many sequences at once —
/// delete into the trash, retag, queue analysis, proxies, thumbnails or a verify — each answering per sequence whether it
/// was done or skipped, and why.
/// </summary>
public static class LibraryEndpoints
{
/// <summary>Sequences one bulk request may name.</summary>
public const int MaxBulkIds = 1000;
/// <summary>Photos one delete request may name.</summary>
public const int MaxFrameIds = 100_000;
private const string Ok = "ok";
private const string Skipped = "skipped";
private const string Regenerable = "private, max-age=3600";
public static void Map(RouteGroupBuilder api)
{
var sequences = api.MapGroup("/sequences");
sequences.MapGet("/{id:guid}/thumb", ThumbAsync).Requires(Permissions.Read);
sequences.MapPost("/{id:guid}/frames/delete", DeleteFramesAsync).Requires(Permissions.Delete);
sequences.MapPost("/bulk/delete", BulkDeleteAsync).Requires(Permissions.Delete);
sequences.MapPost("/bulk/retag", BulkRetagAsync).Requires(Permissions.Edit);
sequences.MapPost("/bulk/jobs", BulkJobsAsync).Requires(Permissions.Edit);
}
/// <summary>The archive engine behind these operations: the host's, or one of its own (a test host that replaces the archive module).</summary>
internal static ArchiveService Engine(ArchiveService? archives, ChronosDatabase db) =>
archives ?? new ArchiveService(db, new ArchiveServiceOptions { Decoder = DecoderFactory.CreateServer(), Owner = ArchiveModule.Owner, ImagesAfterScan = true });
/// <summary>
/// The thumbnail of the sequence's hero frame, else of its first frame, streamed like <c>/frames/{id}/thumb</c> with an
/// ETag (304 when unchanged); 404 <c>not_found</c> (<c>thumbnail</c>) when the sequence has no frame or that thumbnail
/// is not built yet.
/// </summary>
private static async Task<IResult> ThumbAsync(HttpContext context, Guid id, ChronosDatabase db, ServerArchive archive, IFrameAssetLocator locator)
{
var ct = context.RequestAborted;
if (await db.Sequences.GetAsync(id, ct) is not { } sequence) return ApiErrors.NotFound(context, "sequence");
var frame = sequence.HeroFrameId is { } hero && await db.Frames.GetAsync(hero, ct) is { } chosen && chosen.SequenceId == id
? chosen
: (await db.Frames.ListBySequenceAsync(id, limit: 1, cancellationToken: ct)).FirstOrDefault();
if (frame is null) return ApiErrors.NotFound(context, "thumbnail");
var location = await locator.LocateAsync(archive.Current, sequence, frame, FrameAsset.Thumbnail, ct);
if (location is null || !ArchivePathGuard.IsInside(archive.Root, location.Path)) return ApiErrors.NotFound(context, "thumbnail");
var file = new FileInfo(location.Path);
if (!file.Exists || file.LinkTarget is not null) return ApiErrors.NotFound(context, "thumbnail");
var etag = string.Create(CultureInfo.InvariantCulture, $"s-{frame.Id:N}-{file.LastWriteTimeUtc.Ticks}-{location.Offset}");
return FileResponses.Serve(context, file, location.Offset, location.Length, location.ContentType, Regenerable, etag)
?? ApiErrors.NotFound(context, "thumbnail");
}
/// <summary>
/// Moves photos of a sequence into the archive's trash (<see cref="FrameTrash"/>; BRIEF §0.2 rule 8: nothing is erased),
/// or with <c>dryRun</c> says what would go. Owners and admins only, like deleting the sequence. 400 <c>bad_request</c>
/// (<c>frameIds</c>); 409 <c>frames_conflict</c> (the frames changed since <c>expectedFramesVersion</c>, or a photo named
/// is not the sequence's; details: the frames version now), <c>sequence_busy</c>, <c>would_empty</c> (every photo would
/// go: delete the sequence instead), <c>folder_in_use</c>; 503 <c>archive_offline</c>. Publishes <c>sequenceChanged</c>
/// <c>frames</c>; the thumbnails, proxies and analyses follow as jobs.
/// </summary>
private static async Task<IResult> DeleteFramesAsync(HttpContext context, Caller caller, Guid id, FrameDeleteRequest request, ChronosDatabase db,
[FromServices] ArchiveService? archives, JobEngineHost host, AuditLog audit, EventHub events, IHostApplicationLifetime lifetime)
{
var ct = context.RequestAborted;
if (request.FrameIds is not { Count: > 0 and <= MaxFrameIds } ids) return ApiErrors.BadRequest(context, "bad_request", "frameIds");
if (await db.Sequences.GetAsync(id, ct) is not { } sequence) return ApiErrors.NotFound(context, "sequence");
if (!caller.IsAdmin && sequence.OwnerUserId != caller.Id) return ApiErrors.Forbidden(context);
var engine = Engine(archives, db).FrameTrash;
FrameDeletePlan plan;
try
{
plan = await engine.PlanDeleteAsync(id, ids, request.ExpectedFramesVersion, ct);
}
catch (ArgumentException)
{
return ApiErrors.NotFound(context, "sequence");
}
if (plan.Blocked.Count > 0) return Refused(context, plan.Blocked[0], plan.Sequence);
if (request.DryRun)
return Results.Json(new FrameDeleteResultDto(id, plan.Frames.Count, plan.Bytes, plan.Sequence.FramesVersion, plan.Sequence.FrameCount, null, true),
ProtocolJson.Default.FrameDeleteResultDto);
FrameDeleteResult result;
try
{
// Once confirmed the deletion runs to the end even if the client goes away meanwhile.
result = await engine.DeleteAsync(plan, caller.User.Username, host.Engine, lifetime.ApplicationStopping);
}
catch (ArchiveOfflineException)
{
return Refused(context, FrameDeleteBlock.Offline, plan.Sequence);
}
catch (Exception ex) when (ex is StaleDeletePlanException or LeaseUnavailableException)
{
var again = await engine.PlanDeleteAsync(id, ids, plan.Sequence.FramesVersion, ct);
return Refused(context, again.Blocked.Count > 0 ? again.Blocked[0] : ex is LeaseUnavailableException ? FrameDeleteBlock.Busy : FrameDeleteBlock.FramesChanged,
again.Sequence);
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
// A photo is open in another program: nothing moved.
return ApiErrors.Conflict(context, "folder_in_use");
}
await audit.WriteAsync(caller, context, "frames_deleted", "sequence", id.ToString(), $"{plan.Frames.Count} · {result.Entry.Name}");
events.Publish(new SequenceChangedEvent(id, "frames", result.Sequence.FramesVersion, result.Sequence.SettingsVersion));
return Results.Json(new FrameDeleteResultDto(id, plan.Frames.Count, plan.Bytes, result.Sequence.FramesVersion, result.Sequence.FrameCount,
result.Entry.Name, false), ProtocolJson.Default.FrameDeleteResultDto);
}
private static IResult Refused(HttpContext context, FrameDeleteBlock reason, Sequence sequence) => reason switch
{
FrameDeleteBlock.Offline => ApiErrors.Result(context, StatusCodes.Status503ServiceUnavailable, "archive_offline"),
FrameDeleteBlock.Busy => ApiErrors.Conflict(context, "sequence_busy"),
FrameDeleteBlock.WouldEmpty => ApiErrors.Conflict(context, "would_empty"),
_ => ApiErrors.Conflict(context, "frames_conflict", sequence.FramesVersion.ToString(CultureInfo.InvariantCulture)),
};
/// <summary>
/// Moves many sequences into the trash as <c>DELETE /sequences/{id}</c> does each (or, with <c>dryRun</c>, plans them):
/// one item per id, <c>ok</c> with the trash folder, or <c>skipped</c> with the code that says why (<c>not_found</c>,
/// <c>forbidden</c> for a sequence of another user, <c>sequence_busy</c>, <c>archive_root</c>, <c>contains_sequences</c>,
/// <c>sidecar_mismatch</c>, <c>folder_missing</c>, <c>archive_offline</c>, <c>folder_in_use</c>). Each sequence moves whole
/// or not at all. 400 <c>bad_request</c> (<c>ids</c>).
/// </summary>
private static async Task<IResult> BulkDeleteAsync(HttpContext context, Caller caller, BulkDeleteRequest request, ChronosDatabase db,
[FromServices] ArchiveService? archives, AuditLog audit, EventHub events, IHostApplicationLifetime lifetime)
{
var ct = context.RequestAborted;
if (Ids(request.Ids) is not { } ids) return ApiErrors.BadRequest(context, "bad_request", "ids");
var trash = Engine(archives, db).Trash;
var items = new Dictionary<Guid, BulkItemResult>();
var allowed = new List<Sequence>();
foreach (var id in ids)
{
if (await db.Sequences.GetAsync(id, ct) is not { } sequence) items[id] = Skip(context, id, "not_found");
else if (!caller.IsAdmin && sequence.OwnerUserId != caller.Id) items[id] = Skip(context, id, "forbidden");
else allowed.Add(sequence);
}
var steps = new List<DeleteStep>();
foreach (var (sequence, reason, step) in await PlanEachAsync(trash, allowed, ct))
{
if (step is not null) steps.Add(step);
else items[sequence.Id] = Skip(context, sequence.Id, reason!);
}
var photos = 0;
long bytes = 0;
foreach (var step in steps)
{
var sequence = step.Sequence;
if (!request.DryRun)
{
try
{
await trash.DeleteAsync(new DeletePlan([step], []), caller.User.Username, null, lifetime.ApplicationStopping);
}
catch (ArchiveOfflineException)
{
items[sequence.Id] = Skip(context, sequence.Id, "archive_offline");
continue;
}
catch (Exception ex) when (ex is StaleDeletePlanException or LeaseUnavailableException)
{
items[sequence.Id] = Skip(context, sequence.Id, "sequence_busy");
continue;
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
items[sequence.Id] = Skip(context, sequence.Id, "folder_in_use");
continue;
}
await audit.WriteAsync(caller, context, "sequence_deleted", "sequence", sequence.Id.ToString(), sequence.DisplayTitle);
events.Publish(new SequenceChangedEvent(sequence.Id, "deleted", sequence.FramesVersion, sequence.SettingsVersion));
}
items[sequence.Id] = new BulkItemResult(sequence.Id, Ok, TrashFolder: step.TrashFolder);
photos += sequence.FrameCount;
bytes += step.Bytes;
}
return Results.Json(Result(ids, items, request.DryRun, photos, bytes), ProtocolJson.Default.BulkResultDto);
}
/// <summary>Each sequence's delete plan: its step, or the code of why it cannot go.</summary>
private static async Task<List<(Sequence Sequence, string? Reason, DeleteStep? Step)>> PlanEachAsync(SequenceTrash trash, List<Sequence> sequences,
CancellationToken ct)
{
var planned = new List<(Sequence, string?, DeleteStep?)>();
if (sequences.Count == 0) return planned;
DeletePlan plan;
try
{
plan = await trash.PlanAsync([.. sequences.Select(s => s.Id)], null, ct);
}
catch (ArgumentException) when (sequences.Count == 1)
{
return [(sequences[0], "not_found", null)];
}
catch (ArgumentException)
{
// One left the index meanwhile: planned one by one, so the others still go.
foreach (var sequence in sequences) planned.AddRange(await PlanEachAsync(trash, [sequence], ct));
return planned;
}
catch (ArchiveOfflineException)
{
return [.. sequences.Select(s => (s, (string?)"archive_offline", (DeleteStep?)null))];
}
foreach (var step in plan.Steps) planned.Add((step.Sequence, null, step));
foreach (var blocked in plan.Blocked) planned.Add((blocked.Sequence, BlockCode(blocked.Reason), null));
return planned;
}
/// <summary>The error code of a sequence that cannot go to the trash, as <c>DELETE /sequences/{id}</c> answers it.</summary>
internal static string BlockCode(DeleteBlock reason) => reason switch
{
DeleteBlock.Offline => "archive_offline",
DeleteBlock.Busy => "sequence_busy",
DeleteBlock.ArchiveRoot => "archive_root",
DeleteBlock.ContainsSequences => "contains_sequences",
DeleteBlock.ForeignSidecar => "sidecar_mismatch",
_ => "folder_missing",
};
/// <summary>
/// Adds and removes tags on many sequences, each with the same edit as a PATCH of its tags (case-insensitive, the first
/// spelling kept; settings version + 1 when they change). Skipped: <c>not_found</c>, <c>bad_request</c> (more than 50
/// tags then). 400 <c>bad_request</c> (<c>ids</c>, <c>tags</c>). Publishes <c>sequenceChanged</c> <c>settings</c>.
/// </summary>
private static async Task<IResult> BulkRetagAsync(HttpContext context, Caller caller, BulkRetagRequest request, ChronosDatabase db, AuditLog audit,
EventHub events)
{
var ct = context.RequestAborted;
if (Ids(request.Ids) is not { } ids) return ApiErrors.BadRequest(context, "bad_request", "ids");
if (Tags(request.Add) is not { } add || Tags(request.Remove) is not { } remove || add.Count + remove.Count == 0)
return ApiErrors.BadRequest(context, "bad_request", "tags");
var items = new Dictionary<Guid, BulkItemResult>();
foreach (var id in ids)
{
items[id] = await RetagAsync(context, caller, db, audit, events, id, add, remove, ct);
}
return Results.Json(Result(ids, items, false, 0, 0), ProtocolJson.Default.BulkResultDto);
}
private static async Task<BulkItemResult> RetagAsync(HttpContext context, Caller caller, ChronosDatabase db, AuditLog audit, EventHub events, Guid id,
IReadOnlyList<string> add, IReadOnlyList<string> remove, CancellationToken ct)
{
for (var attempt = 0; attempt < 3; attempt++)
{
if (await db.Sequences.GetAsync(id, ct) is not { } sequence) return Skip(context, id, "not_found");
var tags = sequence.Tags.Where(t => !remove.Contains(t, StringComparer.OrdinalIgnoreCase)).ToList();
tags.AddRange(add.Where(t => !tags.Contains(t, StringComparer.OrdinalIgnoreCase)));
if (tags.Count > SequenceEndpoints.MaxTags) return Skip(context, id, "bad_request");
if (tags.ToHashSet(StringComparer.OrdinalIgnoreCase).SetEquals(sequence.Tags)) return new BulkItemResult(id, Ok);
var result = await db.Sequences.EditAsync(id, new SequenceEdit { Tags = tags }, sequence.SettingsVersion, ct);
if (result.Outcome == SequenceEditOutcome.NotFound) return Skip(context, id, "not_found");
// Edited meanwhile: the tags apply again on top of that edit.
if (result.Outcome == SequenceEditOutcome.Conflict) continue;
var updated = result.Sequence!;
await audit.WriteAsync(caller, context, "sequence_updated", "sequence", id.ToString(), "tags");
events.Publish(new SequenceChangedEvent(id, "settings", updated.FramesVersion, updated.SettingsVersion));
return new BulkItemResult(id, Ok);
}
return Skip(context, id, "settings_conflict");
}
/// <summary>
/// Queues one job over many sequences: <c>analyse</c> (light analysis), <c>deep</c> (deep analysis), <c>proxies</c>,
/// <c>thumbnails</c> or <c>verify</c> (a verify of those sequences in their archive), at user priority. Sequences not in
/// the index are skipped (<c>not_found</c>). 202 with the job when one was queued; 400 <c>invalid_job_kind</c>,
/// <c>bad_request</c> (<c>ids</c>); 409 <c>job_unavailable</c> when this server cannot run that kind.
/// </summary>
private static async Task<IResult> BulkJobsAsync(HttpContext context, Caller caller, BulkJobsRequest request, ChronosDatabase db, ServerArchive archive,
JobEngineHost host, AuditLog audit)
{
var ct = context.RequestAborted;
if (Ids(request.Ids) is not { } ids) return ApiErrors.BadRequest(context, "bad_request", "ids");
if (BulkKind(request.Kind) is not { } kind) return ApiErrors.BadRequest(context, "invalid_job_kind");
if (!host.CanRun(kind)) return ApiErrors.Conflict(context, "job_unavailable", JobWire.Kind(kind));
var items = new Dictionary<Guid, BulkItemResult>();
var found = new List<Sequence>();
foreach (var id in ids)
{
if (await db.Sequences.GetAsync(id, ct) is { } sequence && sequence.ArchiveId == archive.Id)
{
found.Add(sequence);
items[id] = new BulkItemResult(id, Ok);
}
else
{
items[id] = Skip(context, id, "not_found");
}
}
if (found.Count == 0) return Results.Json(Result(ids, items, false, 0, 0), ProtocolJson.Default.BulkResultDto);
var targets = found.Select(s => s.Id).ToList();
var label = Label(found);
var job = await host.Engine.EnqueueAsync(SequenceJob(kind, archive.Current, targets, label) with { Priority = JobPriority.User }, caller.Id, ct);
await audit.WriteAsync(caller, context, "job_created", "job", job.Id.ToString(), $"{JobWire.Kind(kind)} · {label}");
return Results.Json(Result(ids, items, false, 0, 0) with { Job = Dto.Job(job) }, ProtocolJson.Default.BulkResultDto,
statusCode: StatusCodes.Status202Accepted);
}
/// <summary>The kinds a bulk job request names.</summary>
private static JobKind? BulkKind(string? kind) => kind?.Trim().ToLowerInvariant() switch
{
"analyse" => JobKind.LightAnalyse,
"deep" => JobKind.DeepAnalyse,
"proxies" => JobKind.Proxies,
"thumbnails" => JobKind.Thumbs,
"verify" => JobKind.Verify,
_ => null,
};
/// <summary>
/// The job of a sequence kind over <paramref name="sequenceIds"/>. A verify targets the archive and names the sequences
/// in its parameters (<see cref="ArchiveJobs.Verify"/>), as its handler reads it.
/// </summary>
internal static ArchiveJobRequest SequenceJob(JobKind kind, Archive archive, IReadOnlyList<Guid> sequenceIds, string label) => kind switch
{
JobKind.Verify => ArchiveJobs.Verify(archive, sequenceIds) with { Label = label },
JobKind.LightAnalyse => ProcessingJobs.LightAnalysis(sequenceIds, label),
JobKind.DeepAnalyse => ProcessingJobs.DeepAnalysis(sequenceIds, label),
JobKind.Proxies => ArchiveJobs.Proxies(sequenceIds, label),
JobKind.Thumbs => ArchiveJobs.Thumbnails(sequenceIds, label),
_ => throw new ArgumentOutOfRangeException(nameof(kind), kind, "Not a sequence job."),
};
/// <summary>"Harbour sunset", or the first three titles and "…".</summary>
internal static string Label(IReadOnlyList<Sequence> sequences) =>
sequences.Count == 1 ? sequences[0].DisplayTitle
: string.Join(", ", sequences.Take(3).Select(s => s.DisplayTitle)) + (sequences.Count > 3 ? " …" : string.Empty);
/// <summary>The ids of a bulk request, once each in their order; null when there are none or too many.</summary>
private static List<Guid>? Ids(IReadOnlyList<Guid>? ids) =>
ids is { Count: > 0 and <= MaxBulkIds } ? [.. ids.Distinct()] : null;
/// <summary>Tags as a request names them, trimmed, once each; null when one is blank or too long.</summary>
private static List<string>? Tags(IReadOnlyList<string>? tags)
{
if (tags is null) return [];
var list = new List<string>();
foreach (var tag in tags)
{
var trimmed = tag?.Trim();
if (string.IsNullOrEmpty(trimmed) || trimmed.Length > TagRepository.MaxLength) return null;
if (!list.Contains(trimmed, StringComparer.OrdinalIgnoreCase)) list.Add(trimmed);
}
return list.Count > SequenceEndpoints.MaxTags ? null : list;
}
private static BulkItemResult Skip(HttpContext context, Guid id, string code) => new(id, Skipped, code, ApiErrors.Message(context, code));
private static BulkResultDto Result(List<Guid> ids, Dictionary<Guid, BulkItemResult> items, bool dryRun, int photos, long bytes)
{
var ordered = ids.Select(id => items[id]).ToList();
var ok = ordered.Count(i => i.Outcome == Ok);
return new BulkResultDto(ordered, ok, ordered.Count - ok, dryRun, photos, bytes);
}
}
+5 -2
View File
@@ -25,7 +25,8 @@ public static class SequenceEndpoints
{
private const int MaxTitle = 200;
private const int MaxNotes = 2000;
private const int MaxTags = 50;
/// <summary>Tags a sequence may carry.</summary>
internal const int MaxTags = 50;
/// <summary>Listings of a sequence's frames read without a lock before one is read under its row lock (<see cref="SnapshotAsync"/>).</summary>
private const int SnapshotAttempts = 3;
@@ -238,9 +239,11 @@ public static class SequenceEndpoints
if (context.Request.Headers.IfNoneMatch == etag) return Results.StatusCode(StatusCodes.Status304NotModified);
var renders = await db.Renders.ListBySequenceAsync(id, ct);
// Tombstones change only with the frames (a deletion bumps the frames version), so the ETag covers them.
var removed = await db.FrameRemovals.ListBySequenceAsync(id, ct);
context.Response.Headers.ETag = etag;
return Results.Json(new SequenceManifest(id, sequence.FramesVersion, sequence.SettingsVersion, sequence.SettingsJson, sequence.Tags, frames,
[.. renders.Select(Dto.Render)], spec), ProtocolJson.Default.SequenceManifest);
[.. renders.Select(Dto.Render)], spec, removed.Count == 0 ? null : Data.Sync.SyncTombstones.ToDto(removed)), ProtocolJson.Default.SequenceManifest);
}
/// <summary>
+122
View File
@@ -0,0 +1,122 @@
using System.Text.RegularExpressions;
using Microsoft.AspNetCore.Mvc;
using Chronos.Core.Archiving;
using Chronos.Core.Model;
using Chronos.Data;
using Chronos.Data.Archiving;
using Chronos.Data.Jobs;
using Chronos.Protocol;
using Chronos.Server.Events;
using Chronos.Server.Jobs;
using Chronos.Server.Security;
using Chronos.Server.Storage;
namespace Chronos.Server.Api;
/// <summary>
/// <c>/api/v1/archive/trash</c> (owner review 2, R6; administrators of the archive): the entries of the archive's trash —
/// deleted sequences and photos deleted from sequences —, restoring one, and emptying it: a dry run that measures the
/// entries and fingerprints them, then an <c>empty-trash</c> job that erases exactly those entries and refuses when the
/// trash changed since (BRIEF §0.2 rule 8: the only erase of user files, always an explicit, confirmed action).
/// </summary>
public static partial class TrashEndpoints
{
public static void Map(RouteGroupBuilder api)
{
var trash = api.MapGroup("/archive/trash").Requires(Permissions.ManageArchive);
trash.MapGet("", ListAsync);
trash.MapPost("/{name}/restore", RestoreAsync);
trash.MapPost("/empty", EmptyAsync);
}
/// <summary>Whether <paramref name="name"/> has the shape of an entry name (a sequence id, a time, <c>photos-</c>): never a path.</summary>
public static bool IsEntryName(string? name) => name is not null && EntryName().IsMatch(name);
public static TrashEntryDto Entry(TrashEntry e) =>
new(e.Name, e.Kind.ToString().ToLowerInvariant(), e.SequenceId, e.Title, e.Folder, e.Frames, e.Files, e.Bytes, e.DeletedBy, e.DeletedAt);
/// <summary>The entries newest first with their total size; an archive offline lists none.</summary>
private static async Task<IResult> ListAsync(HttpContext context, ChronosDatabase db, ServerArchive archive, [FromServices] ArchiveService? archives)
{
var entries = await LibraryEndpoints.Engine(archives, db).Trash.ListAsync(archive.Current, context.RequestAborted);
return Results.Json(new TrashListDto([.. entries.Select(Entry)], entries.Sum(e => e.Bytes)), ProtocolJson.Default.TrashListDto);
}
/// <summary>
/// Brings an entry back: a sequence into its folder and the library, photos into their sequence (a new one when it is
/// gone), each photo re-hashed against its record first. 404 <c>not_found</c> (<c>trash_entry</c>); 409
/// <c>sequence_in_library</c> (that sequence is in the library already), <c>sequence_busy</c>, <c>folder_in_use</c>; 422
/// <c>trash_mismatch</c> (no file matches its record any more); 503 <c>archive_offline</c>. Publishes
/// <c>sequenceChanged</c>: <c>created</c> for a sequence, <c>frames</c> for photos.
/// </summary>
private static async Task<IResult> RestoreAsync(HttpContext context, Caller caller, string name, ChronosDatabase db, ServerArchive archive,
[FromServices] ArchiveService? archives, JobEngineHost host, AuditLog audit, EventHub events, IHostApplicationLifetime lifetime)
{
if (!IsEntryName(name)) return ApiErrors.NotFound(context, "trash_entry");
TrashRestoreResult result;
try
{
result = await LibraryEndpoints.Engine(archives, db).RestoreEntryAsync(archive.Id, name, host.Engine, lifetime.ApplicationStopping);
}
catch (DirectoryNotFoundException)
{
return ApiErrors.NotFound(context, "trash_entry");
}
catch (SequenceInLibraryException)
{
return ApiErrors.Conflict(context, "sequence_in_library");
}
catch (ArchiveOfflineException)
{
return ApiErrors.Result(context, StatusCodes.Status503ServiceUnavailable, "archive_offline");
}
catch (LeaseUnavailableException)
{
return ApiErrors.Conflict(context, "sequence_busy");
}
catch (InvalidDataException)
{
return ApiErrors.Result(context, StatusCodes.Status422UnprocessableEntity, "trash_mismatch");
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
return ApiErrors.Conflict(context, "folder_in_use");
}
var sequence = result.Sequence;
await audit.WriteAsync(caller, context, "trash_restored", "sequence", sequence.Id.ToString(), $"{name} · {sequence.DisplayTitle}");
events.Publish(new SequenceChangedEvent(sequence.Id, result.Kind == TrashKind.Sequence || result.NewSequence ? "created" : "frames",
sequence.FramesVersion, sequence.SettingsVersion));
return Results.Json(new TrashRestoreDto(name, result.Kind.ToString().ToLowerInvariant(), sequence.Id, result.Kind == TrashKind.Sequence ? sequence.FrameCount : result.Photos,
result.Kept, result.NewSequence), ProtocolJson.Default.TrashRestoreDto);
}
/// <summary>
/// Empties the trash, or the entries <c>names</c> names. With <c>dryRun</c> the plan: the entries measured on disk, their
/// totals and a fingerprint, nothing touched. Otherwise the request carries that fingerprint, and the server queues an
/// <c>empty-trash</c> job (202) that erases exactly those entries; the job fails, erasing nothing, when the trash changed
/// before it ran. 400 <c>bad_request</c> (<c>names</c>, <c>fingerprint</c>); 409 <c>trash_changed</c> (details: the
/// fingerprint now), <c>job_unavailable</c>.
/// </summary>
private static async Task<IResult> EmptyAsync(HttpContext context, Caller caller, TrashEmptyRequest request, ChronosDatabase db, ServerArchive archive,
[FromServices] ArchiveService? archives, JobEngineHost host, AuditLog audit)
{
var ct = context.RequestAborted;
if (request.Names is { } names && (names.Count == 0 || names.Any(n => !IsEntryName(n)))) return ApiErrors.BadRequest(context, "bad_request", "names");
var plan = await LibraryEndpoints.Engine(archives, db).PlanEmptyTrashAsync(archive.Id, request.Names, ct);
if (request.DryRun)
{
return Results.Json(new TrashEmptyPlanDto([.. plan.Entries.Select(Entry)], plan.Photos, plan.Files, plan.Bytes, plan.Fingerprint, plan.Unknown),
ProtocolJson.Default.TrashEmptyPlanDto);
}
if (string.IsNullOrWhiteSpace(request.Fingerprint)) return ApiErrors.BadRequest(context, "bad_request", "fingerprint");
if (plan.Fingerprint != request.Fingerprint || plan.Unknown.Count > 0) return ApiErrors.Conflict(context, "trash_changed", plan.Fingerprint);
if (!host.CanRun(JobKind.EmptyTrash)) return ApiErrors.Conflict(context, "job_unavailable", JobWire.Kind(JobKind.EmptyTrash));
var job = await host.Engine.EnqueueAsync(ArchiveJobs.EmptyTrash(archive.Current, plan), caller.Id, ct);
await audit.WriteAsync(caller, context, "trash_emptied", "job", job.Id.ToString(),
$"{plan.Entries.Count} · {plan.Bytes} · {string.Join(", ", plan.Entries.Take(5).Select(e => e.Name))}");
return Results.Json(Dto.Job(job), ProtocolJson.Default.JobDto, statusCode: StatusCodes.Status202Accepted);
}
[GeneratedRegex("^(photos-)?[0-9a-f]{32}(-[0-9]{14}(-[0-9]{1,6})?)?$", RegexOptions.CultureInvariant)]
private static partial Regex EntryName();
}
@@ -26,6 +26,7 @@ public sealed class ServerSettingsStore(ServerOptions options, ServerLogging log
[
JobKind.Render, JobKind.Import, JobKind.Scan, JobKind.Rebuild, JobKind.Verify, JobKind.LightAnalyse, JobKind.DeepAnalyse, JobKind.Thumbs,
JobKind.Proxies, JobKind.Maintenance, JobKind.Duplicates, JobKind.Orphans, JobKind.Adopt, JobKind.Move, JobKind.Backup, JobKind.Cleanup,
JobKind.EmptyTrash,
];
private readonly Lock _gate = new();
+2
View File
@@ -173,12 +173,14 @@ public static class ServerHost
AuthEndpoints.Map(api);
UserEndpoints.Map(api);
SequenceEndpoints.Map(api);
LibraryEndpoints.Map(api);
FileEndpoints.Map(api);
UploadEndpoints.Map(api);
JobEndpoints.Map(api);
ScheduleEndpoints.Map(api);
NodeEndpoints.Map(api);
ArchiveEndpoints.Map(api);
TrashEndpoints.Map(api);
AdminEndpoints.Map(api);
DiagnosticsEndpoints.Map(api);
EventEndpoints.Map(api);
@@ -56,6 +56,14 @@ public static class UploadRules
return intact.FirstOrDefault(f => f.SequenceId == target) ?? intact.FirstOrDefault();
}
/// <summary>
/// Whether <paramref name="target"/> tombstoned <paramref name="sha256"/> (owner review 2, R6): a photo deleted from that
/// sequence (or replaced by a converted file) that an upload must not bring back. The upload is answered as
/// <c>alreadyExists</c> without a frame, so a client neither sends the file nor treats the answer as a failure.
/// </summary>
public static async Task<bool> RemovedFromTargetAsync(ChronosDatabase db, string sha256, Guid? target, CancellationToken cancellationToken) =>
target is { } sequence && await db.FrameRemovals.IsRemovedAsync(sequence, sha256, cancellationToken).ConfigureAwait(false);
/// <summary>
/// The sequence an upload files into: its own, or the one its session key names (the user's own key without one);
/// null while that sequence does not exist yet.
+5 -1
View File
@@ -60,7 +60,11 @@ public sealed class UploadService(
var sha = request.Sha256.ToLowerInvariant();
var sessionKey = string.IsNullOrWhiteSpace(request.SessionKey) ? null : request.SessionKey.Trim();
var copies = await db.Frames.FindBySha256Async(sha, cancellationToken);
var target = copies.Count == 0 ? null : await UploadRules.TargetAsync(db, archive.Id, request.SequenceId, sessionKey, caller.Id, cancellationToken);
var target = await UploadRules.TargetAsync(db, archive.Id, request.SequenceId, sessionKey, caller.Id, cancellationToken);
// A photo deleted from the target sequence is not filed into it again: an old client syncing a copy that still holds
// it, or a watch folder sending it again, hears that the server has it (owner review 2, R6). A restore brings it back.
if (await UploadRules.RemovedFromTargetAsync(db, sha, target, cancellationToken))
return new StartUploadResponse(null, options.ChunkSizeBytes, [], AlreadyExists: true, null);
// Only a copy that is still whole stands for the upload: a client may delete its file on this answer.
if (UploadRules.StandIn(archive.Root, copies, target) is { } known)
return new StartUploadResponse(null, options.ChunkSizeBytes, [], AlreadyExists: true, known.Id);
@@ -0,0 +1,123 @@
using System.Globalization;
using System.IO;
using Chronos.App.Archiving;
using Chronos.App.Infrastructure;
using Chronos.Core.Localization;
using Chronos.Core.Model;
using Chronos.Data;
using Chronos.Data.Archiving;
using Chronos.Data.Archiving.Jobs;
using Chronos.Testing;
namespace Chronos.App.Tests.Archiving;
/// <summary>The trash dialog (owner review 2, R6): photo entries next to sequences, and Empty… behind a dry run and a typed confirmation.</summary>
public sealed class TrashDialogTests(TestContext context)
{
private static readonly DateTimeOffset Deleted = new(2026, 10, 27, 9, 30, 0, TimeSpan.Zero);
private static TrashEntry Entry(string title, TrashKind kind, int photos, long bytes) =>
new(Guid.CreateVersion7(), (kind == TrashKind.Sequence ? string.Empty : FrameTrash.EntryPrefix) + Guid.CreateVersion7().ToString("N"),
Guid.CreateVersion7(), title, "2026/" + title, photos, photos, bytes, Deleted, "alberto", kind);
[Test]
public void EveryKindOfEntryHasWordsAndTheRowSaysWhatItHolds()
{
Loc.SetCulture(CultureInfo.GetCultureInfo("en"));
foreach (var kind in Enum.GetValues<TrashKind>())
Assert.NotEqual("Archive.Trash.Kind." + kind, TrashEmptyPlans.KindText(kind), "every kind has words");
var row = new TrashRowViewModel(Entry("Harbour sunset", TrashKind.Photos, 12, 340_000_000), new Chronos.Ui.Mvvm.RelayCommand(() => { }));
Assert.Equal(Loc.L("Archive.Trash.Kind.Photos"), row.KindText);
Assert.True(row.Detail.StartsWith(row.KindText, StringComparison.Ordinal), row.Detail);
Assert.Contains(Loc.Plural("Shell.Status.Photos", 12), row.Detail);
}
[Test]
public void EmptyingAsksToTypeHowManyEntriesAreErased()
{
Loc.SetCulture(CultureInfo.GetCultureInfo("en"));
var entries = new[] { Entry("Harbour sunset", TrashKind.Photos, 12, 340_000_000), Entry("Comet", TrashKind.Sequence, 1_005, 21_000_000_000) };
var plan = new TrashEmptyPlan(Guid.CreateVersion7(), entries, 21_340_000_000, "f1", []);
var request = Assert.NotNull(TrashEmptyPlans.Request(plan));
Assert.True(request.IsDestructive, "the action is outlined, never primary");
Assert.Equal("2", request.TypedConfirmation);
Assert.Equal(Loc.L("Archive.Delete.Type", "2"), request.TypedPrompt);
Assert.Equal(Loc.Plural("Archive.Trash.EmptyHeading", 2), request.Heading);
Assert.Equal(Loc.L("Archive.Trash.EmptyMessage", Loc.Plural("Shell.Status.Photos", 1_017), Loc.Plural("Archive.Delete.Files", 1_017),
Formatting.Bytes(21_340_000_000)), request.Message);
Assert.Equal(Loc.L("Archive.Trash.Erase"), request.ActionText);
Assert.Equal(Loc.L("Archive.Delete.Keep"), request.SafeText);
var lines = request.Detail!.Split(Environment.NewLine);
Assert.Equal(2, lines.Length);
Assert.Contains(Loc.L("Archive.Trash.Kind.Photos"), lines[0]);
Assert.Null(TrashEmptyPlans.Request(plan with { Entries = [] }), "nothing to erase: no confirmation");
}
[Test]
public async Task EmptyQueuesTheJobOfTheConfirmedPlanOnly()
{
Loc.SetCulture(CultureInfo.GetCultureInfo("en"));
var directory = context.CreateTempDirectory("trash-dialog");
await using var db = await ChronosDatabase.OpenAsync(DatabaseOptions.Sqlite(Path.Combine(directory, "index.db")));
var archives = new ArchiveService(db, new ArchiveServiceOptions { Owner = "test", LeaseWait = TimeSpan.FromSeconds(TestTiming.Budget(20)) });
var archive = await archives.CreateAsync(Path.Combine(directory, "archive"), "Main");
// A sequence deleted earlier: its folder and its record in the archive's trash.
var id = Guid.CreateVersion7();
var trash = SequenceTrash.TrashDirectory(new Core.Archiving.ArchivePaths(archive.RootPath));
Directory.CreateDirectory(Path.Combine(trash, id.ToString("N")));
await File.WriteAllBytesAsync(Path.Combine(trash, id.ToString("N"), "IMG_0001.JPG"), [1, 2, 3]);
await File.WriteAllTextAsync(Path.Combine(trash, id.ToString("N") + ".json"),
$$"""{"sequence":{"id":"{{id}}","archiveId":"{{archive.Id}}","title":"Harbour sunset","folder":"2026/Harbour","frameCount":1},"renders":[],"files":1,"bytes":3,"deletedAt":"2026-10-27T09:30:00+00:00","deletedBy":"alberto"}""");
var plain = new TrashDialogViewModel(archives, archive, () => Task.CompletedTask, _ => { });
await plain.LoadAsync();
Assert.Count(1, plain.Rows);
Assert.False(plain.CanEmpty, "a host without a job engine offers no emptying");
ConfirmRequest? asked = null;
var answer = false;
var queued = new TaskCompletionSource<ArchiveJobRequest>(TaskCreationOptions.RunContinuationsAsynchronously);
var closed = false;
var dialog = new TrashDialogViewModel(archives, archive, () => Task.CompletedTask, _ => closed = true, r =>
{
asked = r;
return answer;
}, r =>
{
queued.TrySetResult(r);
return Task.CompletedTask;
});
await dialog.LoadAsync();
Assert.True(dialog.CanEmpty);
Assert.Equal(Loc.Plural("Archive.Trash.Entries", 1) + Loc.Separator + Formatting.Bytes(3), dialog.SummaryText);
// Kept: the confirmation was dismissed, nothing is queued.
dialog.EmptyCommand.Execute(null);
await UntilAsync(() => asked is not null && !dialog.IsBusy);
Assert.Equal("1", asked!.TypedConfirmation);
Assert.False(queued.Task.IsCompleted);
Assert.False(closed);
answer = true;
dialog.EmptyCommand.Execute(null);
var request = await queued.Task.WaitAsync(TimeSpan.FromSeconds(TestTiming.Budget(20)));
Assert.Equal(JobKind.EmptyTrash, request.Kind);
Assert.SequenceEqual([archive.Id], request.Targets);
Assert.Contains(id.ToString("N"), request.ParametersJson);
await UntilAsync(() => closed);
Assert.Count(1, await archives.ListTrashAsync(archive.Id), "the job erases, not the dialog");
}
private static async Task UntilAsync(Func<bool> condition)
{
var until = DateTime.UtcNow + TimeSpan.FromSeconds(TestTiming.Budget(20));
while (!condition())
{
Assert.True(DateTime.UtcNow < until, "timed out");
await Task.Delay(20);
}
}
}
@@ -118,7 +118,7 @@ public sealed class LibraryScreenSmokeTests
await ArchiveScenes.ShowAsync(app.Shell, "trash");
var trash = Assert.NotNull(app.Shell.Archive.Dialog as TrashDialogViewModel, "Maintenance › Trash… opens the trash");
Assert.Equal(2, trash.Rows.Count, "the sample's deleted sequences");
Assert.Equal(2, trash.Rows.Count, "the sample's deleted sequence and photos");
Assert.True(trash.HasRows);
await app.ExpectAsync<ArchiveView>("archive-trash-sample");
trash.CloseCommand.Execute(null);
@@ -58,6 +58,43 @@ public sealed class ManifestStoreTests(TestContext context)
Assert.Contains(field, json);
}
[Test]
public void TombstonesAndConversionSourcesRoundTripWhileASidecarWithoutThemStaysAsItWas()
{
var path = SidecarPath();
var plain = Sample();
ManifestStore.WriteSequence(path, plain);
var before = File.ReadAllText(path);
Assert.False(before.Contains("\"removed\"", StringComparison.Ordinal), "no tombstones, no field");
Assert.False(before.Contains("\"source\"", StringComparison.Ordinal), "an original has no source");
var at = new DateTimeOffset(2026, 10, 27, 10, 0, 0, TimeSpan.Zero);
var changed = plain with
{
Removed =
[
new RemovedPhoto { Sha256 = new string('b', 64), Reason = RemovalReason.Deleted, At = at },
new RemovedPhoto { Sha256 = new string('c', 64), Replacement = new string('d', 64), Reason = RemovalReason.Replaced, At = at },
],
Frames =
[
plain.Frames[0] with
{
Source = new FrameSource { Sha256 = new string('e', 64), Format = ImageFileFormat.Cr2, File = "originals/IMG_0001.CR2", Size = 25_000_000 },
},
],
};
ManifestStore.WriteSequence(path, changed);
var read = ManifestStore.ReadSequence(path);
Assert.Equal(Canonical(changed), Canonical(read));
Assert.Equal(RemovalReason.Replaced, Assert.NotNull(read.Removed)[1].Reason);
Assert.Equal(new string('d', 64), read.Removed[1].Replacement);
Assert.Equal(ImageFileFormat.Cr2, Assert.NotNull(read.Frames[0].Source).Format);
var json = File.ReadAllText(path);
foreach (var field in new[] { "\"removed\":", "\"replacement\":", "\"reason\":\"Replaced\"", "\"at\":", "\"source\":" }) Assert.Contains(field, json);
}
[Test]
public void ASidecarBeingReadIsReplacedOnceTheReaderIsDone()
{
@@ -0,0 +1,456 @@
using System.Security.Cryptography;
using System.Text.Json;
using Chronos.Core.Archiving;
using Chronos.Core.Import;
using Chronos.Core.Model;
using Chronos.Core.Project;
using Chronos.Data.Archiving;
using Chronos.Data.Archiving.Jobs;
using Chronos.Data.Repositories;
using Chronos.Data.Tests.Infrastructure;
using Chronos.Testing;
using static Chronos.Data.Tests.Archiving.ArchiveFixture;
namespace Chronos.Data.Tests.Archiving;
/// <summary>Deleting photos of a sequence into the archive's trash, restoring them and emptying the trash (owner review 2, R6).</summary>
[Category("archive")]
public abstract class FrameTrashTests(TestContext context) : DatabaseTest(context)
{
private async Task<ArchiveFixture> FixtureAsync() => Own(await CreateAsync(Context, database: await NewOptionsAsync(multiConnection: true)));
/// <summary>A sequence of <paramref name="count"/> uploaded photos (hashed, under <c>originals/</c>), ten seconds apart.</summary>
private static async Task<(Sequence Sequence, List<Frame> Frames)> UploadedAsync(ArchiveFixture fx, int count, string session = "upload-1", int seed = 100)
{
Guid? id = null;
for (var i = 0; i < count; i++)
{
var (path, size, sha) = fx.Stage(Start.AddSeconds(10 * i), seed + i);
var result = await fx.Service.Ingest.CommitAsync(new IngestRequest(fx.Archive.Id, id, null, id is null ? session : null, path, $"IMG_{i + 1:0000}.JPG",
size, sha, null, null), fx.Ct);
id = result.SequenceId;
}
return ((await fx.Db.Sequences.GetAsync(id!.Value))!, [.. await fx.Db.Frames.ListBySequenceAsync(id.Value)]);
}
private static async Task<FrameDeleteResult> DeleteAsync(ArchiveFixture fx, Sequence sequence, params Frame[] frames)
{
var plan = await fx.Service.FrameTrash.PlanDeleteAsync(sequence.Id, [.. frames.Select(f => f.Id)], sequence.FramesVersion, fx.Ct);
Assert.True(plan.CanDelete, string.Join(", ", plan.Blocked));
return await fx.Service.FrameTrash.DeleteAsync(plan, "alberto", null, fx.Ct);
}
private static Job Running(JobKind kind, IReadOnlyList<Guid> targets) => new()
{
Id = Guid.CreateVersion7(),
Kind = kind,
TargetIds = targets,
State = JobState.Running,
NodeId = "test#0",
Created = DateTimeOffset.UtcNow,
Started = DateTimeOffset.UtcNow,
};
private static string Sha(string path) => Convert.ToHexStringLower(SHA256.HashData(File.ReadAllBytes(path)));
[Test]
public async Task TheDryRunCountsThePhotosAndChangesNothing()
{
var fx = await FixtureAsync();
var (sequence, frames) = await UploadedAsync(fx, 5);
var before = await Photos.DumpIndexAsync(fx.Db, fx.Archive.Id);
var plan = await fx.Service.FrameTrash.PlanDeleteAsync(sequence.Id, [frames[1].Id, frames[3].Id], sequence.FramesVersion, fx.Ct);
Assert.True(plan.CanDelete);
Assert.SequenceEqual([frames[1].Id, frames[3].Id], plan.Frames.Select(f => f.Id));
Assert.Equal(frames[1].Size + frames[3].Size, plan.Bytes);
Assert.Equal(before, await Photos.DumpIndexAsync(fx.Db, fx.Archive.Id));
Assert.False(Directory.Exists(FrameTrash.Directory(fx.Paths)), "a dry run moves nothing");
foreach (var frame in frames) Assert.FileExists(fx.Paths.FromRelative(frame.RelativePath));
}
[Test]
public async Task APlanSaysWhyPhotosCannotGo()
{
var fx = await FixtureAsync();
var (sequence, frames) = await UploadedAsync(fx, 3);
var trash = fx.Service.FrameTrash;
Assert.SequenceEqual([FrameDeleteBlock.WouldEmpty], (await trash.PlanDeleteAsync(sequence.Id, [.. frames.Select(f => f.Id)], null, fx.Ct)).Blocked);
Assert.SequenceEqual([FrameDeleteBlock.FramesChanged], (await trash.PlanDeleteAsync(sequence.Id, [frames[0].Id], sequence.FramesVersion - 1, fx.Ct)).Blocked);
Assert.SequenceEqual([FrameDeleteBlock.FramesChanged], (await trash.PlanDeleteAsync(sequence.Id, [frames[0].Id, Guid.CreateVersion7()], null, fx.Ct)).Blocked);
await Assert.ThrowsAsync<ArgumentException>(() => trash.PlanDeleteAsync(Guid.CreateVersion7(), [frames[0].Id], null, fx.Ct));
// A job on the sequence, or on its whole archive, holds it.
await fx.Db.Jobs.EnqueueAsync(Running(JobKind.Render, [sequence.Id]));
var busy = await trash.PlanDeleteAsync(sequence.Id, [frames[0].Id], null, fx.Ct);
Assert.SequenceEqual([FrameDeleteBlock.Busy], busy.Blocked);
Assert.False(busy.CanDelete);
await Assert.ThrowsAsync<StaleDeletePlanException>(() => trash.DeleteAsync(busy, null, null, fx.Ct));
Assert.Count(3, await fx.Db.Frames.ListBySequenceAsync(sequence.Id));
}
[Test]
public async Task DeletingMovesThePhotosToTheTrashAndTombstonesThem()
{
var fx = await FixtureAsync();
var (sequence, frames) = await UploadedAsync(fx, 5);
var folder = fx.Paths.Sequence(sequence.Folder);
// Proxies are named by frame index: they follow their frames.
Directory.CreateDirectory(folder.Proxies);
for (var i = 0; i < frames.Count; i++) File.WriteAllBytes(folder.Proxy(i), [(byte)i]);
await fx.Db.Sequences.UpdateStatusAsync(sequence.Id, SequenceStatus.AnalyzedLight | SequenceStatus.ThumbsBuilt, SequenceStatus.None);
var result = await DeleteAsync(fx, sequence, frames[1], frames[3]);
foreach (var gone in new[] { frames[1], frames[3] }) Assert.FileMissing(fx.Paths.FromRelative(gone.RelativePath));
var entry = result.Entry;
Assert.Equal(TrashKind.Photos, entry.Kind);
Assert.True(FrameTrash.IsPhotoEntry(entry.Name));
Assert.Equal(2, entry.Frames);
Assert.Equal(frames[1].Size + frames[3].Size, entry.Bytes);
var entryFolder = Path.Combine(FrameTrash.Directory(fx.Paths), entry.Name[FrameTrash.EntryPrefix.Length..]);
Assert.Equal(frames[1].Sha256, Sha(Path.Combine(entryFolder, "originals", Path.GetFileName(frames[1].RelativePath))));
Assert.FileMissing(folder.RemovalJournalPath);
var manifest = ManifestStore.ReadSequence(folder.ManifestPath);
Assert.SequenceEqual([frames[0].Id, frames[2].Id, frames[4].Id], manifest.Frames.Select(f => f.Id));
Assert.SequenceEqual([0, 1, 2], manifest.Frames.Select(f => f.I));
Assert.Equal(sequence.FramesVersion + 1, manifest.Versions.Frames);
Assert.SequenceEqual([frames[1].Sha256, frames[3].Sha256], Assert.NotNull(manifest.Removed).Select(r => r.Sha256));
Assert.True(manifest.Removed.All(r => r.Reason == RemovalReason.Deleted));
var row = result.Sequence;
Assert.Equal(3, row.FrameCount);
Assert.Equal(manifest.Versions.Frames, row.FramesVersion);
Assert.Equal(SequenceStatus.None, row.Status & (SequenceStatus.AnalyzedLight | SequenceStatus.ThumbsBuilt));
Assert.SequenceEqual([frames[0].Id, frames[2].Id, frames[4].Id], (await fx.Db.Frames.ListBySequenceAsync(sequence.Id)).Select(f => f.Id));
Assert.SequenceEqual([frames[1].Sha256, frames[3].Sha256], (await fx.Db.FrameRemovals.ListBySequenceAsync(sequence.Id)).Select(r => r.Sha256).Order());
Assert.True(await fx.Db.FrameRemovals.IsRemovedAsync(sequence.Id, frames[3].Sha256!.ToUpperInvariant()));
Assert.SequenceEqual<byte>([0, 2, 4], Enumerable.Range(0, 3).Select(i => File.ReadAllBytes(folder.Proxy(i))[0]));
Assert.FileMissing(folder.Proxy(3));
Assert.FileMissing(folder.Proxy(4));
var listed = Assert.Contains(await fx.Service.ListTrashAsync(fx.Archive.Id, fx.Ct), e => e.Name == entry.Name);
Assert.Equal(sequence.DisplayTitle, listed.Title);
Assert.Equal("alberto", listed.DeletedBy);
}
[Test]
public async Task PositionsOnDeletedPhotosWaitAndComeBackWithThem()
{
var fx = await FixtureAsync();
var (sequence, frames) = await UploadedAsync(fx, 5);
var folder = fx.Paths.Sequence(sequence.Folder);
var project = new ProjectSettings { ExcludedFrames = [1, 4], HeroFrame = 3 }.ToJson();
var version = (await fx.Db.Sequences.EditAsync(sequence.Id, new SequenceEdit { SettingsJson = project, HeroFrameId = frames[3].Id }, null, fx.Ct))
.Sequence!.SettingsVersion;
var deleted = await DeleteAsync(fx, (await fx.Db.Sequences.GetAsync(sequence.Id))!, frames[1], frames[3]);
var row = deleted.Sequence;
var parked = ProjectSettings.FromJson(row.SettingsJson!);
Assert.SequenceEqual([2], parked.ExcludedFrames, "the exclusion of the fifth photo follows it");
Assert.Null(parked.HeroFrame);
Assert.Count(2, Assert.NotNull(parked.PendingFrames));
Assert.True(parked.HeroPending, "the hero waits for its photo");
Assert.Null(row.HeroFrameId);
Assert.Equal(version, row.SettingsVersion);
Assert.True(File.Exists(folder.ProjectPath), "project.json mirrors the index");
var restored = await fx.Service.RestoreEntryAsync(fx.Archive.Id, deleted.Entry.Name, null, fx.Ct);
Assert.Equal(TrashKind.Photos, restored.Kind);
Assert.Equal(2, restored.Photos);
Assert.Empty(restored.Kept);
var back = (await fx.Db.Sequences.GetAsync(sequence.Id))!;
var placed = ProjectSettings.FromJson(back.SettingsJson!);
Assert.SequenceEqual([1, 4], placed.ExcludedFrames);
Assert.Equal(3, placed.HeroFrame);
Assert.Null(placed.PendingFrames);
Assert.Equal(frames[3].Id, back.HeroFrameId);
Assert.Equal(version, back.SettingsVersion);
}
[Test]
public async Task ARestorePutsThePhotosBackWithTheirIdsAndDropsTheirTombstones()
{
var fx = await FixtureAsync();
var (sequence, frames) = await UploadedAsync(fx, 4);
var folder = fx.Paths.Sequence(sequence.Folder);
Directory.CreateDirectory(folder.Proxies);
for (var i = 0; i < frames.Count; i++) File.WriteAllBytes(folder.Proxy(i), [(byte)i]);
var deleted = await DeleteAsync(fx, sequence, frames[0], frames[2]);
var restored = await fx.Service.RestoreEntryAsync(fx.Archive.Id, deleted.Entry.Name, null, fx.Ct);
Assert.False(restored.NewSequence);
Assert.Equal(sequence.Id, restored.Sequence.Id);
var now = await fx.Db.Frames.ListBySequenceAsync(sequence.Id);
Assert.SequenceEqual(frames.Select(f => f.Id), now.Select(f => f.Id));
Assert.SequenceEqual(frames.Select(f => f.RelativePath), now.Select(f => f.RelativePath));
Assert.SequenceEqual(frames.Select(f => f.Sha256), now.Select(f => f.Sha256));
foreach (var frame in frames) Assert.Equal(frame.Sha256, Sha(fx.Paths.FromRelative(frame.RelativePath)));
Assert.Null(ManifestStore.ReadSequence(folder.ManifestPath).Removed);
Assert.Empty(await fx.Db.FrameRemovals.ListBySequenceAsync(sequence.Id));
Assert.Equal(sequence.FramesVersion + 2, restored.Sequence.FramesVersion);
Assert.DoesNotContain(deleted.Entry.Name, (await fx.Service.ListTrashAsync(fx.Archive.Id, fx.Ct)).Select(e => e.Name));
Assert.False(Directory.Exists(Path.Combine(FrameTrash.Directory(fx.Paths), deleted.Entry.Name[FrameTrash.EntryPrefix.Length..])));
// The proxies of the frames that stayed are back at their index; those of the photos that went are made again.
Assert.SequenceEqual<byte>([1], File.ReadAllBytes(folder.Proxy(1)));
Assert.SequenceEqual<byte>([3], File.ReadAllBytes(folder.Proxy(3)));
Assert.FileMissing(folder.Proxy(0));
}
[Test]
public async Task ARestoreKeepsAPhotoWhoseFileNoLongerMatchesItsRecord()
{
var fx = await FixtureAsync();
var (sequence, frames) = await UploadedAsync(fx, 4);
var deleted = await DeleteAsync(fx, sequence, frames[1], frames[2]);
var entryFolder = Path.Combine(FrameTrash.Directory(fx.Paths), deleted.Entry.Name[FrameTrash.EntryPrefix.Length..]);
var damaged = Path.Combine(entryFolder, "originals", Path.GetFileName(frames[2].RelativePath));
var bytes = File.ReadAllBytes(damaged);
bytes[^10] ^= 0x01;
File.WriteAllBytes(damaged, bytes);
var restored = await fx.Service.RestoreEntryAsync(fx.Archive.Id, deleted.Entry.Name, null, fx.Ct);
Assert.Equal(1, restored.Photos);
Assert.SequenceEqual([frames[2].OriginalFileName!], restored.Kept);
Assert.SequenceEqual([frames[0].Id, frames[1].Id, frames[3].Id], (await fx.Db.Frames.ListBySequenceAsync(sequence.Id)).Select(f => f.Id));
Assert.SequenceEqual([frames[2].Sha256], (await fx.Db.FrameRemovals.ListBySequenceAsync(sequence.Id)).Select(r => r.Sha256));
var left = Assert.Contains(await fx.Service.ListTrashAsync(fx.Archive.Id, fx.Ct), e => e.Name == deleted.Entry.Name);
Assert.Equal(1, left.Frames);
Assert.FileExists(damaged);
// Nothing left that matches: the entry stays as it is.
await Assert.ThrowsAsync<InvalidDataException>(() => fx.Service.RestoreEntryAsync(fx.Archive.Id, deleted.Entry.Name, null, fx.Ct));
Assert.FileExists(damaged);
}
[Test]
public async Task PhotosOfASequenceThatIsGoneComeBackAsANewOne()
{
var fx = await FixtureAsync();
var (sequence, frames) = await UploadedAsync(fx, 3);
var deleted = await DeleteAsync(fx, sequence, frames[1]);
var whole = await fx.Service.PlanDeleteAsync([sequence.Id], null, fx.Ct);
await fx.Service.DeleteAsync(whole, null, null, fx.Ct);
var restored = await fx.Service.RestoreEntryAsync(fx.Archive.Id, deleted.Entry.Name, null, fx.Ct);
Assert.True(restored.NewSequence);
Assert.NotEqual(sequence.Id, restored.Sequence.Id);
Assert.Equal(sequence.DisplayTitle, restored.Sequence.DisplayTitle);
var frame = (await fx.Db.Frames.ListBySequenceAsync(restored.Sequence.Id)).Single();
Assert.Equal(frames[1].Sha256, frame.Sha256);
Assert.Equal(frames[1].Sha256, Sha(fx.Paths.FromRelative(frame.RelativePath)));
// The whole sequence is still in the trash, restorable on its own.
Assert.Contains(await fx.Service.ListTrashAsync(fx.Archive.Id, fx.Ct), e => e.Kind == TrashKind.Sequence && e.SequenceId == sequence.Id);
}
[Test]
public async Task TombstonesSurviveARebuildFromTheSidecars()
{
var fx = await FixtureAsync();
var (sequence, frames) = await UploadedAsync(fx, 3);
await DeleteAsync(fx, sequence, frames[0]);
await fx.ReopenWithEmptyIndexAsync();
await fx.ScanAsync(ScanMode.Rebuild);
var removals = await fx.Db.FrameRemovals.ListBySequenceAsync(sequence.Id);
Assert.SequenceEqual([frames[0].Sha256], removals.Select(r => r.Sha256));
Assert.Equal(RemovalReason.Deleted, removals[0].Reason);
Assert.Count(2, await fx.Db.Frames.ListBySequenceAsync(sequence.Id));
}
[Test]
public async Task TombstonesAreReplacedWholeAndGoWithTheirSequence()
{
var fx = await FixtureAsync();
var (sequence, _) = await UploadedAsync(fx, 1);
var at = new DateTimeOffset(2026, 10, 27, 10, 0, 0, TimeSpan.Zero);
var many = Enumerable.Range(0, 450).Select(i => new FrameRemoval(sequence.Id, i.ToString("x64", System.Globalization.CultureInfo.InvariantCulture),
i % 2 == 0 ? null : new string('A', 64), i % 2 == 0 ? RemovalReason.Deleted : RemovalReason.Replaced, at)).ToList();
Assert.Equal(450, await fx.Db.FrameRemovals.SyncSequenceAsync(sequence.Id, many, fx.Ct));
var stored = await fx.Db.FrameRemovals.ListBySequenceAsync(sequence.Id, fx.Ct);
Assert.Count(450, stored);
Assert.Equal((new string('a', 64), RemovalReason.Replaced), (stored.Single(r => r.Sha256 == many[1].Sha256).ReplacementSha256, stored[1].Reason));
Assert.Equal(1, await fx.Db.FrameRemovals.SyncSequenceAsync(sequence.Id, [many[3], many[3]], fx.Ct));
Assert.Count(1, await fx.Db.FrameRemovals.ListBySequenceAsync(sequence.Id, fx.Ct));
await fx.Db.Sequences.DeleteAsync(sequence.Id, fx.Ct);
Assert.Empty(await fx.Db.FrameRemovals.ListBySequenceAsync(sequence.Id, fx.Ct));
}
[Test]
public async Task ADeletionACrashInterruptedIsUndoneByTheNextWriter()
{
var fx = await FixtureAsync();
var (sequence, frames) = await UploadedAsync(fx, 3);
var folder = fx.Paths.Sequence(sequence.Folder);
// The journal was written and one of two files moved when the process died; the sidecar still lists both frames.
var entry = Path.Combine(FrameTrash.Directory(fx.Paths), $"{sequence.Id:N}-20261027120000");
var record = entry + ".json";
Directory.CreateDirectory(Path.Combine(entry, "originals"));
File.WriteAllText(record, "{}");
var moves = frames.Take(2).Select(f => new FrameRemovalMove(f.RelativePath[(sequence.Folder.Length + 1)..],
fx.Paths.ToRelative(Path.Combine(entry, "originals", Path.GetFileName(f.RelativePath))))).ToList();
FrameRemovalJournal.Write(folder, new FrameRemovalJournal(fx.Paths.ToRelative(record), fx.Paths.ToRelative(entry), [frames[0].Id, frames[1].Id], moves));
File.Move(fx.Paths.FromRelative(frames[0].RelativePath), fx.Paths.FromRelative(moves[0].To));
// Any writer of the sequence reads its sidecar under the lease first.
await fx.Service.SetSequenceInfoAsync(sequence.Id, "Harbour", [], 0, null, fx.Ct);
foreach (var frame in frames) Assert.Equal(frame.Sha256, Sha(fx.Paths.FromRelative(frame.RelativePath)));
Assert.FileMissing(folder.RemovalJournalPath);
Assert.FileMissing(record);
Assert.False(Directory.Exists(entry));
Assert.Count(3, ManifestStore.ReadSequence(folder.ManifestPath).Frames);
}
[Test]
public async Task ErasingTheCardKeepsTheCardCopyOfADeletedPhoto()
{
var fx = await FixtureAsync();
var card = Context.CreateTempDirectory("card");
Photos.WriteSeries(Path.Combine(card, "DCIM", "100CANON"), Start, 4);
var media = await fx.Service.Imports.RegisterCardAsync(new CardVolume("3F2A9C11-EOS_DIGITAL-127865454592", 0x3F2A9C11, "EOS_DIGITAL", 127_865_454_592), fx.Ct);
var plan = await fx.Service.PlanImportAsync(fx.Archive.Id, card, new ImportOptions(), media.Id, null, fx.Ct);
await fx.Service.Imports.ExecuteAsync(new ImportJobParameters(fx.Archive.Id, plan, media.Id, true, null), Guid.NewGuid(), 0,
_ => ValueTask.CompletedTask, null, fx.Ct);
var id = (await fx.Db.Sequences.FindBySessionKeyAsync(fx.Archive.Id, plan.Sessions.Single().SessionKey))!.Value;
var frames = await fx.Db.Frames.ListBySequenceAsync(id);
await DeleteAsync(fx, (await fx.Db.Sequences.GetAsync(id))!, frames[2]);
var erase = await fx.Service.PlanCardEraseAsync(fx.Archive.Id, media.Id, card, null, fx.Ct);
CardEraser.Execute(erase);
// The deleted photo has no frame any more: nothing in the archive vouches for its card copy, which stays.
Assert.Equal(3, erase.ToDelete.Count());
var left = Directory.EnumerateFiles(card, "*.JPG", SearchOption.AllDirectories).Select(f => Path.GetFileName(f)).ToList();
Assert.SequenceEqual([frames[2].OriginalFileName!], left);
}
[Test]
public async Task EmptyingErasesExactlyThePlannedEntries()
{
var fx = await FixtureAsync();
var (sequence, frames) = await UploadedAsync(fx, 3);
var (other, _) = await UploadedAsync(fx, 2, "upload-2", 200);
var photos = await DeleteAsync(fx, sequence, frames[1]);
await fx.Service.DeleteAsync(await fx.Service.PlanDeleteAsync([other.Id], null, fx.Ct), null, null, fx.Ct);
var engine = await fx.StartEngineAsync();
var plan = await fx.Service.PlanEmptyTrashAsync(fx.Archive.Id, null, fx.Ct);
Assert.Count(2, plan.Entries);
Assert.Equal(3, plan.Photos);
Assert.True(plan.Bytes > 0);
Assert.Equal(plan.Fingerprint, (await fx.Service.PlanEmptyTrashAsync(fx.Archive.Id, null, fx.Ct)).Fingerprint, "the plan reads only");
Assert.SequenceEqual(["nope"], (await fx.Service.PlanEmptyTrashAsync(fx.Archive.Id, [photos.Entry.Name, "nope"], fx.Ct)).Unknown);
var job = await engine.EnqueueAsync(ArchiveJobs.EmptyTrash(fx.Archive, plan), cancellationToken: fx.Ct);
var done = await fx.WaitForAsync(job.Id, JobState.Done);
Assert.Empty(await fx.Service.ListTrashAsync(fx.Archive.Id, fx.Ct));
Assert.Empty(Directory.EnumerateFileSystemEntries(FrameTrash.Directory(fx.Paths)));
Assert.Empty(Directory.EnumerateFiles(SequenceTrash.TrashDirectory(fx.Paths), "*", SearchOption.AllDirectories));
Assert.Contains(Chronos.Core.Localization.Loc.L("Archive.Log.TrashEmptied", 2, Housekeeping.CleanupSteps.Megabytes(plan.Bytes)), done.LogTail);
// Tombstones outlive the trash: the deleted photo is still not offered back.
Assert.True(await fx.Db.FrameRemovals.IsRemovedAsync(sequence.Id, frames[1].Sha256!));
}
[Test]
public async Task AnEmptyingPlannedBeforeTheTrashChangedErasesNothing()
{
var fx = await FixtureAsync();
var (sequence, frames) = await UploadedAsync(fx, 4);
var first = await DeleteAsync(fx, sequence, frames[1]);
var second = await DeleteAsync(fx, (await fx.Db.Sequences.GetAsync(sequence.Id))!, frames[2]);
var plan = await fx.Service.PlanEmptyTrashAsync(fx.Archive.Id, null, fx.Ct);
var engine = await fx.StartEngineAsync();
// A planned entry came back (restored) before the job ran.
await fx.Service.RestoreEntryAsync(fx.Archive.Id, first.Entry.Name, null, fx.Ct);
var job = await engine.EnqueueAsync(ArchiveJobs.EmptyTrash(fx.Archive, plan), cancellationToken: fx.Ct);
var failed = await fx.WaitForAsync(job.Id, JobState.Failed);
Assert.Equal(Chronos.Core.Localization.Loc.L("Archive.Error.TrashChanged"), failed.Error);
Assert.Contains(await fx.Service.ListTrashAsync(fx.Archive.Id, fx.Ct), e => e.Name == second.Entry.Name);
// A planned entry whose folder changed since makes the plan stale too.
var again = await fx.Service.PlanEmptyTrashAsync(fx.Archive.Id, null, fx.Ct);
var folder = Path.Combine(FrameTrash.Directory(fx.Paths), second.Entry.Name[FrameTrash.EntryPrefix.Length..]);
File.WriteAllBytes(Path.Combine(folder, "added.bin"), [1, 2, 3]);
Assert.NotEqual(again.Fingerprint, (await fx.Service.PlanEmptyTrashAsync(fx.Archive.Id, null, fx.Ct)).Fingerprint);
var stale = await engine.EnqueueAsync(ArchiveJobs.EmptyTrash(fx.Archive, again), cancellationToken: fx.Ct);
await fx.WaitForAsync(stale.Id, JobState.Failed);
Assert.Contains(await fx.Service.ListTrashAsync(fx.Archive.Id, fx.Ct), e => e.Name == second.Entry.Name);
Assert.FileExists(Path.Combine(folder, "added.bin"));
}
[Test]
public async Task AReplacedEntryIsErasedOnlyWhileItsReplacementMatches()
{
var fx = await FixtureAsync();
var (sequence, frames) = await UploadedAsync(fx, 2);
var replacement = frames[1];
var good = WriteReplaced(fx, sequence, "20261027100000", replacement.Sha256!);
var bad = WriteReplaced(fx, sequence, "20261027110000", new string('a', 64));
var engine = await fx.StartEngineAsync();
var listed = await fx.Service.ListTrashAsync(fx.Archive.Id, fx.Ct);
Assert.True(listed.All(e => e.Kind == TrashKind.Replaced));
var plan = await fx.Service.PlanEmptyTrashAsync(fx.Archive.Id, null, fx.Ct);
var job = await engine.EnqueueAsync(ArchiveJobs.EmptyTrash(fx.Archive, plan), cancellationToken: fx.Ct);
var failed = await fx.WaitForAsync(job.Id, JobState.Failed);
Assert.Equal(Chronos.Core.Localization.Loc.Plural("Archive.Error.TrashKept", 1), failed.Error);
var left = Assert.Contains(await fx.Service.ListTrashAsync(fx.Archive.Id, fx.Ct), e => e.Name == FrameTrash.EntryPrefix + bad);
Assert.Equal(TrashKind.Replaced, left.Kind);
Assert.DoesNotContain(FrameTrash.EntryPrefix + good, (await fx.Service.ListTrashAsync(fx.Archive.Id, fx.Ct)).Select(e => e.Name));
// A replacement whose file no longer hashes to it keeps its originals too.
var other = WriteReplaced(fx, sequence, "20261027120000", replacement.Sha256!);
var file = fx.Paths.FromRelative(replacement.RelativePath);
var bytes = File.ReadAllBytes(file);
bytes[^10] ^= 0x01;
File.WriteAllBytes(file, bytes);
var again = await fx.Service.PlanEmptyTrashAsync(fx.Archive.Id, [FrameTrash.EntryPrefix + other], fx.Ct);
var second = await engine.EnqueueAsync(ArchiveJobs.EmptyTrash(fx.Archive, again), cancellationToken: fx.Ct);
await fx.WaitForAsync(second.Id, JobState.Failed);
Assert.Contains(await fx.Service.ListTrashAsync(fx.Archive.Id, fx.Ct), e => e.Name == FrameTrash.EntryPrefix + other);
}
/// <summary>An entry of originals a conversion replaced (a later phase), its one photo replaced by <paramref name="replacement"/>.</summary>
private static string WriteReplaced(ArchiveFixture fx, Sequence sequence, string stamp, string replacement)
{
var name = $"{sequence.Id:N}-{stamp}";
var folder = Path.Combine(FrameTrash.Directory(fx.Paths), name);
Directory.CreateDirectory(Path.Combine(folder, "originals"));
var bytes = Photos.Jpeg(Start, 999);
File.WriteAllBytes(Path.Combine(folder, "originals", "IMG_0001.CR2"), bytes);
var sha = Convert.ToHexStringLower(SHA256.HashData(bytes));
var frame = new SidecarFrame { Id = Guid.CreateVersion7(), File = "originals/IMG_0001.CR2", Size = bytes.Length, Sha256 = sha };
var record = new PhotoTrashRecord(TrashKind.Replaced, sequence.Id, sequence.DisplayTitle, sequence.Folder,
[new TrashedPhoto(frame, [new TrashedFile(frame.File, "originals/IMG_0001.CR2", bytes.Length, sha)], replacement)], bytes.Length,
DateTimeOffset.UtcNow, "converter");
File.WriteAllText(folder + ".json", JsonSerializer.Serialize(record, TrashJson.Default.PhotoTrashRecord));
return name;
}
}
public sealed class SqliteFrameTrashTests(TestContext context) : FrameTrashTests(context)
{
protected override Backend Backend => Backend.SqliteFile;
}
[RequiresEnvironment(PgTestServer.Variable)]
public sealed class PostgresFrameTrashTests(TestContext context) : FrameTrashTests(context)
{
protected override Backend Backend => Backend.Postgres;
}
@@ -9,7 +9,7 @@ public abstract class MigrationTests(TestContext context) : RepositoryTest(conte
private static readonly string[] Tables =
[
"archives", "sequences", "frames", "renders", "jobs", "media_cards", "import_history", "users", "tokens",
"render_nodes", "settings", "tags", "sequence_tags", "audit_log", "uploads", "schedules",
"render_nodes", "settings", "tags", "sequence_tags", "audit_log", "uploads", "schedules", "frame_removals",
];
[Test]
@@ -274,6 +274,9 @@ internal sealed class FakeSequence
public List<FakeFrame> Frames { get; } = [];
public List<FakeRender> Renders { get; } = [];
/// <summary>Tombstones of photos removed on the server (owner review 2, R6), as its manifest lists them.</summary>
public List<RemovedPhotoDto> Removed { get; } = [];
/// <summary>Like the server, the capture time comes from the photo's Exif data when none is given.</summary>
public FakeFrame AddFrame(byte[] bytes, string? name = null, DateTime? capturedAt = null) => InsertFrame(Frames.Count, bytes, name, capturedAt);
@@ -295,7 +298,7 @@ internal sealed class FakeSequence
public SequenceManifest Manifest(Func<Guid, bool> proxySent) => new(Id, FramesVersion, SettingsVersion, SettingsJson, Tags,
[.. Frames.Select((f, i) => new ManifestFrame(f.Id, i, f.FileName, f.Bytes.Length, Unhashed ? string.Empty : f.Sha256,
(Status & SequenceStatus.ProxiesBuilt) != 0 || proxySent(f.Id), (Status & SequenceStatus.ThumbsBuilt) != 0, WantsProxy: Undecodable && !proxySent(f.Id)))],
[.. Renders.Select(r => r.Dto())], Undecodable ? new ProxySpec(1280, 90) : null);
[.. Renders.Select(r => r.Dto())], Undecodable ? new ProxySpec(1280, 90) : null, Removed.Count == 0 ? null : [.. Removed]);
public FrameDto FrameDto(FakeFrame frame) => frame.Dto(Id, Frames.IndexOf(frame), Unhashed);
}
@@ -67,6 +67,36 @@ public sealed class SyncRulesTests
Assert.Equal((1, 0), SyncRules.CompareOriginals([null, null], ["bb"]));
}
[Test]
public void PhotosEitherSideRemovedAreNotOfferedBack()
{
static FramePhoto Photo(string sha) => new(sha, 100, null);
var at = new DateTimeOffset(2026, 10, 27, 10, 0, 0, TimeSpan.Zero);
var localId = Guid.CreateVersion7();
// The server deleted "cc", which this PC still holds; this PC deleted "dd", which the server still holds.
var tombstones = new SyncTombstones([new FrameRemoval(localId, "DD", null, RemovalReason.Deleted, at)],
[new RemovedPhotoDto("cc", null, "deleted", at)]);
Assert.Equal((0, 0), SyncRules.CompareOriginals([Photo("aa"), Photo("bb"), Photo("cc")], [Photo("aa"), Photo("bb"), Photo("dd")], tombstones));
Assert.Equal((1, 1), SyncRules.CompareOriginals([Photo("aa"), Photo("cc")], [Photo("aa"), Photo("dd")], SyncTombstones.None));
Assert.True(tombstones.DeletedRemotely("CC"));
Assert.True(tombstones.DeletedLocally("dd"));
Assert.False(tombstones.DeletedLocally("cc"));
// A photo replaced by a converted file on either side is the same photo as its replacement, and so is a chain of them.
var replaced = new SyncTombstones([], [new RemovedPhotoDto("aa", "a2", "replaced", at), new RemovedPhotoDto("a2", "a3", "replaced", at)]);
Assert.Equal("a3", replaced.Canonical("AA"));
Assert.Equal((0, 0), SyncRules.CompareOriginals([Photo("aa"), Photo("bb")], [Photo("a3"), Photo("bb")], replaced));
Assert.False(replaced.DeletedRemotely("aa"), "a replaced photo is there as its replacement");
var replacedHere = new SyncTombstones([new FrameRemoval(localId, "bb", "b2", RemovalReason.Replaced, at)], null);
Assert.Equal((0, 0), SyncRules.CompareOriginals([Photo("b2")], [Photo("bb")], replacedHere));
// A replacement deleted since: neither the photo nor its replacement is offered back.
var gone = new SyncTombstones([], [new RemovedPhotoDto("aa", "a2", "replaced", at), new RemovedPhotoDto("a2", null, "deleted", at)]);
Assert.Equal((0, 0), SyncRules.CompareOriginals([Photo("aa")], [], gone));
Assert.True(SyncTombstones.None.IsEmpty);
Assert.SequenceEqual(["deleted", "replaced"], SyncTombstones.ToDto([new FrameRemoval(localId, "aa", null, RemovalReason.Deleted, at),
new FrameRemoval(localId, "bb", "b2", RemovalReason.Replaced, at)]).Select(r => r.Reason));
}
[Test]
public void FramesEitherSideHasNotHashedAreMatchedBySizeAndCaptureTime()
{
@@ -6,6 +6,7 @@ using Chronos.Core.Model;
using Chronos.Core.Project;
using Chronos.Core.Render;
using Chronos.Data.Sync;
using Chronos.Protocol;
using Chronos.Testing;
namespace Chronos.Data.Tests.Sync;
@@ -116,6 +117,40 @@ public sealed class SyncTransferTests(TestContext context)
Assert.Equal((SyncStatus.InSync, 0), (entry.Status, entry.Link!.MissingRemote));
}
[Test]
public async Task PhotosOneSideDeletedAreNeitherSentNorFetchedAgain()
{
await using var f = await SyncFixture.CreateAsync(context);
var photos = f.NewPhotos(4);
var local = await f.LocalAsync("Comet", photos);
// The server holds the first three and deleted the fourth, which this PC still holds.
var remote = f.Remote.Add("Comet", photos[..3]);
remote.Removed.Add(new RemovedPhotoDto(Sha(photos[3]), null, "deleted", DateTimeOffset.UtcNow));
await f.Sync.LinkAsync(f.Server, remote.Id, local.Id, f.Ct);
await f.RefreshAsync();
Assert.Equal(0, (await f.EntryAsync("Comet")).Link!.MissingRemote);
var engine = await f.StartEngineAsync();
var upload = await f.Sync.EnqueueUploadAsync(engine, f.Server, local);
await f.Archive.WaitForAsync(upload.Id, JobState.Done);
Assert.Equal(3, remote.Frames.Count);
Assert.Count(4, await f.Db.Frames.ListBySequenceAsync(local.Id, cancellationToken: f.Ct));
// This PC deletes the first photo, which the server still holds: it is not fetched back, and nothing local is deleted.
var frames = await f.Db.Frames.ListBySequenceAsync(local.Id, cancellationToken: f.Ct);
var sequence = (await f.Db.Sequences.GetAsync(local.Id, f.Ct))!;
var plan = await f.Archive.Service.FrameTrash.PlanDeleteAsync(local.Id, [frames[0].Id], sequence.FramesVersion, f.Ct);
await f.Archive.Service.FrameTrash.DeleteAsync(plan, null, null, f.Ct);
await f.RefreshAsync();
var entry = await f.EntryAsync("Comet");
Assert.Equal((0, 0), (entry.Link!.MissingRemote, entry.Link.MissingLocal));
var download = await f.Sync.EnqueueDownloadAsync(engine, f.Server, remote.Id, "Comet", SyncDownloadMode.Originals, null);
await f.Archive.WaitForAsync(download.Id, JobState.Done);
var kept = await f.Db.Frames.ListBySequenceAsync(local.Id, cancellationToken: f.Ct);
Assert.SequenceEqual(photos[1..].Select(Sha), kept.Select(k => k.Sha256!));
}
[Test]
public async Task AResumedUploadSendsOnlyWhatTheServerLacks()
{
@@ -0,0 +1,293 @@
using System.Net;
using System.Net.Http.Headers;
using System.Net.Http.Json;
using Microsoft.Extensions.DependencyInjection;
using Chronos.Core.Codecs.Jpeg;
using Chronos.Core.Model;
using Chronos.Protocol;
using Chronos.Server.Storage;
using Chronos.Testing;
namespace Chronos.Server.Tests.Api;
/// <summary>
/// The library the admin panel manages (owner review 2, R6): sequence thumbnails, photos deleted into the trash, the trash
/// listed, restored and emptied, and bulk operations, against the real archive engine.
/// </summary>
[Category("server")]
public sealed class LibraryTests(TestContext context)
{
private static readonly TimeSpan Wait = TimeSpan.FromSeconds(TestTiming.Budget(30));
[Test]
public async Task PhotosAreDeletedIntoTheTrashAndRestoredThroughTheApi()
{
await using var server = await TestServer.StartAsync(context, services: ArchiveModule.Register);
var (editor, _, _) = await server.UserAsync("alberto", "editor");
var (other, _, _) = await server.UserAsync("giulia", "editor");
var admin = await server.AdminAsync();
var photos = Enumerable.Range(0, 3).Select(Photo).ToList();
Guid? id = null;
for (var i = 0; i < photos.Count; i++) id = (await Uploads.UploadAsync(editor, photos[i], $"IMG_{i + 1:0000}.JPG", sessionKey: "roof")).SequenceId;
await UntilIdleAsync(server);
var sequence = (await server.Database.Sequences.GetAsync(id!.Value))!;
var frames = await server.Database.Frames.ListBySequenceAsync(sequence.Id);
var url = $"/api/v1/sequences/{sequence.Id}/frames/delete";
using (var forbidden = await other.PostAsJsonAsync(url, new FrameDeleteRequest([frames[1].Id], null, false), ProtocolJson.Default.FrameDeleteRequest))
await forbidden.ExpectErrorAsync(HttpStatusCode.Forbidden, "forbidden");
using (var none = await editor.PostAsJsonAsync(url, new FrameDeleteRequest([], null, false), ProtocolJson.Default.FrameDeleteRequest))
await none.ExpectErrorAsync(HttpStatusCode.BadRequest, "bad_request");
using (var stale = await editor.PostAsJsonAsync(url, new FrameDeleteRequest([frames[1].Id], sequence.FramesVersion - 1, false),
ProtocolJson.Default.FrameDeleteRequest))
Assert.Equal(sequence.FramesVersion.ToString(System.Globalization.CultureInfo.InvariantCulture),
(await stale.ExpectErrorAsync(HttpStatusCode.Conflict, "frames_conflict")).Details);
using (var all = await editor.PostAsJsonAsync(url, new FrameDeleteRequest([.. frames.Select(f => f.Id)], null, false), ProtocolJson.Default.FrameDeleteRequest))
await all.ExpectErrorAsync(HttpStatusCode.Conflict, "would_empty");
using (var dry = await editor.PostAsJsonAsync(url, new FrameDeleteRequest([frames[1].Id], sequence.FramesVersion, true), ProtocolJson.Default.FrameDeleteRequest))
{
var plan = await dry.ReadAsync(ProtocolJson.Default.FrameDeleteResultDto);
Assert.True(plan.DryRun);
Assert.Equal((1, frames[1].Size, 3), (plan.Photos, plan.Bytes, plan.FrameCount));
Assert.Null(plan.TrashEntry);
}
Assert.Count(3, await server.Database.Frames.ListBySequenceAsync(sequence.Id));
FrameDeleteResultDto done;
using (var deleted = await editor.PostAsJsonAsync(url, new FrameDeleteRequest([frames[1].Id], sequence.FramesVersion, false),
ProtocolJson.Default.FrameDeleteRequest))
done = await deleted.ReadAsync(ProtocolJson.Default.FrameDeleteResultDto);
Assert.False(done.DryRun);
Assert.Equal((2, sequence.FramesVersion + 1), (done.FrameCount, done.FramesVersion));
var name = Assert.NotNull(done.TrashEntry);
Assert.FileMissing(Path.Combine(server.Options.ArchivePath, frames[1].RelativePath));
// The manifest names the deleted photo, so clients do not send it back; an upload of it is answered as already there.
using (var manifest = await editor.GetAsync($"/api/v1/sequences/{sequence.Id}/manifest"))
Assert.SequenceEqual([Uploads.Sha(photos[1])], Assert.NotNull((await manifest.ReadAsync(ProtocolJson.Default.SequenceManifest)).Removed).Select(r => r.Sha256));
var again = await Uploads.StartAsync(editor, photos[1], "IMG_0002.JPG", sequence.Id, expected: HttpStatusCode.OK);
Assert.True(again.AlreadyExists);
Assert.Null(again.FrameId);
Assert.Count(2, await server.Database.Frames.ListBySequenceAsync(sequence.Id));
using (var notAdmin = await editor.GetAsync("/api/v1/archive/trash")) await notAdmin.ExpectErrorAsync(HttpStatusCode.Forbidden, "forbidden");
using (var list = await admin.GetAsync("/api/v1/archive/trash"))
{
var trash = await list.ReadAsync(ProtocolJson.Default.TrashListDto);
var entry = Assert.Contains(trash.Entries, e => e.Name == name);
Assert.Equal(("photos", 1, sequence.Id, "alberto"), (entry.Kind, entry.Photos, entry.SequenceId, entry.DeletedBy));
Assert.Equal(frames[1].Size, trash.Bytes);
}
await UntilIdleAsync(server);
using (var restored = await admin.PostAsync($"/api/v1/archive/trash/{name}/restore", null))
{
var result = await restored.ReadAsync(ProtocolJson.Default.TrashRestoreDto);
Assert.Equal(("photos", sequence.Id, 1, false), (result.Kind, result.SequenceId, result.Photos, result.NewSequence));
Assert.Empty(result.Kept);
}
Assert.SequenceEqual(frames.Select(f => f.Id), (await server.Database.Frames.ListBySequenceAsync(sequence.Id)).Select(f => f.Id));
using (var gone = await admin.PostAsync($"/api/v1/archive/trash/{name}/restore", null)) await gone.ExpectErrorAsync(HttpStatusCode.NotFound, "not_found");
using (var bad = await admin.PostAsync("/api/v1/archive/trash/..%2F..%2Fsecret/restore", null)) Assert.Equal(HttpStatusCode.NotFound, bad.StatusCode);
}
[Test]
public async Task TheTrashIsEmptiedByAJobAfterADryRunAndItsFingerprint()
{
await using var server = await TestServer.StartAsync(context, services: ArchiveModule.Register);
var (editor, _, _) = await server.UserAsync("alberto", "editor");
var admin = await server.AdminAsync();
var a = (await Uploads.UploadAsync(editor, Photo(10), "a.jpg", sessionKey: "a")).SequenceId;
Guid b = default;
for (var i = 0; i < 2; i++) b = (await Uploads.UploadAsync(editor, Photo(20 + i), $"b{i}.jpg", sessionKey: "b")).SequenceId;
await UntilIdleAsync(server);
using (var deleted = await editor.DeleteAsync($"/api/v1/sequences/{a}")) await deleted.ExpectAsync(HttpStatusCode.OK);
var frames = await server.Database.Frames.ListBySequenceAsync(b);
using (var photo = await editor.PostAsJsonAsync($"/api/v1/sequences/{b}/frames/delete", new FrameDeleteRequest([frames[0].Id], null, false),
ProtocolJson.Default.FrameDeleteRequest))
await photo.ExpectAsync(HttpStatusCode.OK);
await UntilIdleAsync(server);
TrashEmptyPlanDto plan;
using (var dry = await admin.PostAsJsonAsync("/api/v1/archive/trash/empty", new TrashEmptyRequest(null, true, null), ProtocolJson.Default.TrashEmptyRequest))
plan = await dry.ReadAsync(ProtocolJson.Default.TrashEmptyPlanDto);
Assert.Equal(2, plan.Entries.Count);
Assert.SequenceEqual(["photos", "sequence"], plan.Entries.Select(e => e.Kind).Order());
Assert.Equal(2, plan.Photos);
using (var noPrint = await admin.PostAsJsonAsync("/api/v1/archive/trash/empty", new TrashEmptyRequest(null, false, null), ProtocolJson.Default.TrashEmptyRequest))
await noPrint.ExpectErrorAsync(HttpStatusCode.BadRequest, "bad_request");
using (var wrong = await admin.PostAsJsonAsync("/api/v1/archive/trash/empty", new TrashEmptyRequest(null, false, "0123"), ProtocolJson.Default.TrashEmptyRequest))
Assert.Equal(plan.Fingerprint, (await wrong.ExpectErrorAsync(HttpStatusCode.Conflict, "trash_changed")).Details);
using (var names = await admin.PostAsJsonAsync("/api/v1/archive/trash/empty", new TrashEmptyRequest(["../x"], true, null), ProtocolJson.Default.TrashEmptyRequest))
await names.ExpectErrorAsync(HttpStatusCode.BadRequest, "bad_request");
JobDto job;
using (var empty = await admin.PostAsJsonAsync("/api/v1/archive/trash/empty", new TrashEmptyRequest(null, false, plan.Fingerprint),
ProtocolJson.Default.TrashEmptyRequest))
{
await empty.ExpectAsync(HttpStatusCode.Accepted);
job = await empty.ReadAsync(ProtocolJson.Default.JobDto);
}
Assert.Equal("empty-trash", job.Kind);
await UntilJobAsync(admin, job.Id, "done");
using (var list = await admin.GetAsync("/api/v1/archive/trash")) Assert.Empty((await list.ReadAsync(ProtocolJson.Default.TrashListDto)).Entries);
Assert.Empty(Directory.EnumerateFiles(server.Options.TrashPath, "*", SearchOption.AllDirectories));
Assert.Count(1, await server.Database.Frames.ListBySequenceAsync(b));
}
[Test]
public async Task BulkOperationsAnswerForEachSequence()
{
await using var server = await TestServer.StartAsync(context, services: ArchiveModule.Register);
var archive = server.Services.GetRequiredService<ServerArchive>();
var (editor, _, _) = await server.UserAsync("alberto", "editor");
var (other, _, _) = await server.UserAsync("giulia", "editor");
var a = (await Uploads.UploadAsync(editor, Photo(30), "a.jpg", sessionKey: "a")).SequenceId;
var b = (await Uploads.UploadAsync(editor, Photo(31), "b.jpg", sessionKey: "b")).SequenceId;
var c = (await Uploads.UploadAsync(other, Photo(32), "c.jpg", sessionKey: "c")).SequenceId;
var unknown = Guid.CreateVersion7();
await UntilIdleAsync(server);
using (var retag = await editor.PostAsJsonAsync("/api/v1/sequences/bulk/retag", new BulkRetagRequest([a, b, unknown], ["Night", "roof"], null),
ProtocolJson.Default.BulkRetagRequest))
{
var result = await retag.ReadAsync(ProtocolJson.Default.BulkResultDto);
Assert.Equal((2, 1), (result.Ok, result.Skipped));
Assert.Equal("not_found", result.Items.Single(i => i.Id == unknown).Reason);
}
using (var untag = await editor.PostAsJsonAsync("/api/v1/sequences/bulk/retag", new BulkRetagRequest([a], ["sky"], ["NIGHT"]),
ProtocolJson.Default.BulkRetagRequest))
await untag.ExpectAsync(HttpStatusCode.OK);
Assert.SequenceEqual(["roof", "sky"], (await server.Database.Sequences.GetAsync(a))!.Tags.Order());
Assert.SequenceEqual(["Night", "roof"], (await server.Database.Sequences.GetAsync(b))!.Tags.Order());
using (var noTags = await editor.PostAsJsonAsync("/api/v1/sequences/bulk/retag", new BulkRetagRequest([a], null, null), ProtocolJson.Default.BulkRetagRequest))
await noTags.ExpectErrorAsync(HttpStatusCode.BadRequest, "bad_request");
// A verify of sequences targets their archive (it once targeted the sequences, which its handler cannot read).
using (var verify = await editor.PostAsJsonAsync("/api/v1/sequences/bulk/jobs", new BulkJobsRequest([a, b, unknown], "verify"),
ProtocolJson.Default.BulkJobsRequest))
{
await verify.ExpectAsync(HttpStatusCode.Accepted);
var result = await verify.ReadAsync(ProtocolJson.Default.BulkResultDto);
var job = Assert.NotNull(result.Job);
Assert.Equal(("verify", 2, 1), (job.Kind, result.Ok, result.Skipped));
Assert.SequenceEqual([archive.Id], job.TargetIds);
await UntilJobAsync(editor, job.Id, "done");
}
using (var single = await editor.PostAsJsonAsync("/api/v1/jobs", new CreateJobRequest("verify", [a], null), ProtocolJson.Default.CreateJobRequest))
{
var job = await single.ReadAsync(ProtocolJson.Default.JobDto);
Assert.SequenceEqual([archive.Id], job.TargetIds);
await UntilJobAsync(editor, job.Id, "done");
}
using (var thumbs = await editor.PostAsJsonAsync("/api/v1/sequences/bulk/jobs", new BulkJobsRequest([a, b], "thumbnails"), ProtocolJson.Default.BulkJobsRequest))
{
var job = Assert.NotNull((await thumbs.ReadAsync(ProtocolJson.Default.BulkResultDto)).Job);
Assert.Equal("thumbs", job.Kind);
Assert.SequenceEqual([a, b], job.TargetIds);
}
using (var bogus = await editor.PostAsJsonAsync("/api/v1/sequences/bulk/jobs", new BulkJobsRequest([a], "render"), ProtocolJson.Default.BulkJobsRequest))
await bogus.ExpectErrorAsync(HttpStatusCode.BadRequest, "invalid_job_kind");
await UntilIdleAsync(server);
using (var dry = await editor.PostAsJsonAsync("/api/v1/sequences/bulk/delete", new BulkDeleteRequest([a, c, unknown], true), ProtocolJson.Default.BulkDeleteRequest))
{
var result = await dry.ReadAsync(ProtocolJson.Default.BulkResultDto);
Assert.True(result.DryRun);
Assert.Equal(".chronos/trash/" + a.ToString("N"), result.Items[0].TrashFolder);
Assert.SequenceEqual(["ok", "skipped", "skipped"], result.Items.Select(i => i.Outcome));
Assert.SequenceEqual<string?>([null, "forbidden", "not_found"], result.Items.Select(i => i.Reason));
Assert.Equal(1, result.Photos);
}
Assert.NotNull(await server.Database.Sequences.GetAsync(a));
using (var deleted = await editor.PostAsJsonAsync("/api/v1/sequences/bulk/delete", new BulkDeleteRequest([a, b], false), ProtocolJson.Default.BulkDeleteRequest))
Assert.Equal(2, (await deleted.ReadAsync(ProtocolJson.Default.BulkResultDto)).Ok);
Assert.Null(await server.Database.Sequences.GetAsync(a));
Assert.Null(await server.Database.Sequences.GetAsync(b));
Assert.NotNull(await server.Database.Sequences.GetAsync(c));
}
[Test]
public async Task ASequenceThumbnailIsItsHeroFramesOrItsFirst()
{
await using var server = await TestServer.StartAsync(context, services: ArchiveModule.Register);
var (editor, _, _) = await server.UserAsync("alberto", "editor");
var first = await Uploads.UploadAsync(editor, Photo(40), "IMG_0001.JPG", sessionKey: "t");
var second = await Uploads.UploadAsync(editor, Photo(41), "IMG_0002.JPG", sessionKey: "t");
var id = first.SequenceId;
var thumbs = new Dictionary<Guid, byte[]>();
foreach (var frame in new[] { first.FrameId, second.FrameId }) thumbs[frame] = await ThumbnailAsync(editor, $"/api/v1/frames/{frame}/thumb");
using (var cover = await editor.GetAsync($"/api/v1/sequences/{id}/thumb"))
{
await cover.ExpectAsync(HttpStatusCode.OK);
var bytes = await cover.Content.ReadAsByteArrayAsync();
Assert.BytesEqual(thumbs[first.FrameId], bytes);
var etag = Assert.NotNull(cover.Headers.ETag);
using var request = new HttpRequestMessage(HttpMethod.Get, $"/api/v1/sequences/{id}/thumb");
request.Headers.IfNoneMatch.Add(etag);
using var cached = await editor.SendAsync(request);
Assert.Equal(HttpStatusCode.NotModified, cached.StatusCode);
}
using (var hero = await editor.PatchAsJsonAsync($"/api/v1/sequences/{id}", new UpdateSequenceRequest(null, null, null, null, null, null, second.FrameId),
ProtocolJson.Default.UpdateSequenceRequest))
await hero.ExpectAsync(HttpStatusCode.OK);
var heroThumb = await ThumbnailAsync(editor, $"/api/v1/sequences/{id}/thumb");
Assert.BytesEqual(thumbs[second.FrameId], heroThumb);
using var created = await editor.PostAsJsonAsync("/api/v1/sequences", new CreateSequenceRequest("Empty", null), ProtocolJson.Default.CreateSequenceRequest);
var empty = await created.ReadAsync(ProtocolJson.Default.SequenceDto);
using (var none = await editor.GetAsync($"/api/v1/sequences/{empty.Id}/thumb")) await none.ExpectErrorAsync(HttpStatusCode.NotFound, "not_found");
using (var unknown = await editor.GetAsync($"/api/v1/sequences/{Guid.CreateVersion7()}/thumb")) await unknown.ExpectErrorAsync(HttpStatusCode.NotFound, "not_found");
}
/// <summary>A thumbnail once its pass has drawn it (404 until then).</summary>
private static async Task<byte[]> ThumbnailAsync(HttpClient client, string url)
{
var until = DateTime.UtcNow + Wait;
while (true)
{
using var response = await client.GetAsync(url);
if (response.StatusCode == HttpStatusCode.OK) return await response.Content.ReadAsByteArrayAsync();
Assert.True(DateTime.UtcNow < until, $"{url}: {(int)response.StatusCode}");
await Task.Delay(50);
}
}
private static async Task UntilJobAsync(HttpClient client, Guid id, string state)
{
var until = DateTime.UtcNow + Wait;
while (true)
{
using var response = await client.GetAsync($"/api/v1/jobs/{id}");
var job = await response.ReadAsync(ProtocolJson.Default.JobDto);
if (job.State == state) return;
if (job.State is "failed" or "cancelled") Assert.Fail($"job {job.Kind} {job.State}: {job.Error}\n{job.LogTail}");
Assert.True(DateTime.UtcNow < until, $"job {job.Kind} still {job.State}");
await Task.Delay(50);
}
}
/// <summary>Waits until no job is queued or running (the thumbnails, proxies and analyses uploads queue).</summary>
private static async Task UntilIdleAsync(TestServer server)
{
var until = DateTime.UtcNow + Wait;
foreach (var state in new[] { JobState.Queued, JobState.Running, JobState.Queued, JobState.Running })
{
while ((await server.Database.Jobs.ListAsync(state)).Items.Count > 0)
{
Assert.True(DateTime.UtcNow < until, $"jobs still {state}");
await Task.Delay(50);
}
}
}
private static byte[] Photo(int seed)
{
const int width = 96, height = 64;
var pixels = new byte[width * height * 3];
for (var i = 0; i < pixels.Length; i++) pixels[i] = (byte)((i * 7 + seed * 31) % 251);
return JpegEncoder.Encode(pixels, width, height, 3, new JpegEncodeOptions { Quality = 90 });
}
}