fix(server): keep convert dry runs like Reorganize's, say what a new sequence lacks
The convert endpoint planned the whole selection inside the HTTP request, again for every mode, quality keystroke and confirmation, with no limit: behind a proxy an all-sequence conversion could never get past its dry run or its apply. - ReorganizePlanCache becomes PlanCache, shared by Reorganize and Convert: one plan per key made apart from the request, two at once across both kinds, eight kept, 15-minute timeout, 5-minute lifetime, 503 plan_dropped. Keys are typed records (ReorganizePlanKey, ConvertPlanKey) that also say which exceptions are refusals. - A conversion plan is keyed by archive, a convert stamp of the index (SequenceRepository.GetConvertStampAsync: the Reorganize stamp plus, per planned sequence, the photos' count, sizes, formats, pixel sizes and missing, corrupt, preview-only and undecodable flags), the sequences and the target with the quality that counts. Not the mode: the request's mode and options are set on the kept plan, and the free space is measured again at every request through the engine. A plan made while the volume was away is not kept (503 archive_offline). - New sequence holds the converted copies alone: the dry run says the skipped photos are not copied into it (Archive.Convert.NotCopied.*, Server.Convert.OnPcNew), the panel's totals, column and confirmation give the count, and so does the desktop's summary. - Panel: the preview is asked in Replace's neutral words; a dry run cut off by a proxy is asked again while the totals show how long it takes, plan_dropped is retried once; an apply the server let go plans again, a changed one shows Admin.Convert.PlanChanged, and one with no answer is not resent (its job may be queued). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit f0b1e379ce7c08b6d3459ab25ca6214c8f366f6c)
This commit is contained in:
1 parent
d0bbd77c92
commit
8174852a2c
22 files changed
+927
-342
No files matched your search
@@ -86,7 +86,7 @@ Dependencies point one way: `Core` ← `Core.Windows`, `Data`, `Protocol` ← `S
|
||||
| `Ui/Controls` | `ChronosWindow` (chrome, language direction), `NavRail`, `JobBar`, `ToastHost` (`Toast.Closed` once a toast leaves the screen), `Field` (placeholder, icon, error), `Chip`, `AutoSlider`, `StatusGlyph`, `EmptyState`, `Icon`, `Bidi` (right-to-left rules), `Motion` (animations off for screenshots) |
|
||||
| `Ui/Interop` | DWM calls of the window chrome; `TrayIcon` (`Shell_NotifyIconW`: the Uploader's tray, the client's Windows notifications) |
|
||||
| `Ui/Visuals` | `DrawingElement` (hover readout, item keyboard and activation) and its charts (`BarChart`, `SeriesChart`, `StackedBar`, `HeatmapStrip`, `TimelineBands`), `Filmstrip`, `PreviewViewport`, `CurveEditor`, `SequenceCard`, `VirtualizingWrapPanel`, `SegmentedControl`, `SplitButton` |
|
||||
| `Server/Api`, `Server/Events` | Endpoint groups (`XxxEndpoints.Map`; `ScheduleEndpoints`: `/api/v1/schedules`; `LibraryEndpoints`: the Library filters' facets, sequence thumbnail, photo delete, bulk delete/retag/jobs; `TrashEndpoints`: list, restore, empty; `ArchiveEndpoints`: settings, statistics, scan jobs, Reorganize's dry run and `move` job, with `ReorganizePlanCache`: a dry run's plan, made apart from its request, kept a few minutes while what it reads from the index stays, for the same dry run and its confirmation; Convert's dry run and `convert` job), API errors, the WebSocket `EventHub` |
|
||||
| `Server/Api`, `Server/Events` | Endpoint groups (`XxxEndpoints.Map`; `ScheduleEndpoints`: `/api/v1/schedules`; `LibraryEndpoints`: the Library filters' facets, sequence thumbnail, photo delete, bulk delete/retag/jobs; `TrashEndpoints`: list, restore, empty; `ArchiveEndpoints`: settings, statistics, scan jobs, Reorganize's dry run and `move` job, Convert's dry run and `convert` job, both through `PlanCache`: a dry run's plan, made apart from its request, a few at once, kept a few minutes while what it reads from the index stays (its `PlanKey`), for the same dry run and its confirmation), API errors, the WebSocket `EventHub` |
|
||||
| `Server/Security` | Accounts, password hashing, tokens, session cookies, login guard, access rules, audit log |
|
||||
| `Server/Storage`, `Server/Jobs` | Archive and processing modules (the archive module's `SequenceChanged` publishes `sequenceChanged` `reorganized`), uploads (`UploadMaintenance`: recovery at start, and the sweep that is the `uploads` clean-up step; `UploadService.ReceiveAgainAsync`: a verified upload whose filing was refused for now goes back to receiving, 503), render store (files a render under the sequence's lease, where the index has the folder then; `FolderMovedException`, 503 `folder_moved`, when the sequence is not in it); `JobEngineHost`, `SchedulerHost` (the `Scheduler` on the engine, started after it; archive.json read every 10 minutes whatever the scan schedule does; every schedule change and run published as a `scheduleChanged` event) and `ServerScheduleHost` (time zone, archive refreshed from archive.json, system schedules), render node service, monitor and availability |
|
||||
| `Server/Hosting`, `Server/Admin/web` | `ServerHost`, `ServerLogging` (console and file levels the logging filters read at every line, the file sink's limits, the engine's level; applied again when Settings › Logs is saved), certificates, discovery beacon, embedded static files (`EmbeddedStaticFiles`: `no-cache` and a content ETag, so an upgrade never mixes old and new modules), Windows service (start and stop failures in the Application event log); the admin panel (ES modules, one per page; `schedules.js` is the Jobs page's Scheduled tab; the Library's views in `pages/library/`: `sequence.js` a sequence and its photo viewer, `trash.js` the archive's trash, `shared.js` cards, selection, grid keyboard and the retry of thumbnails a finished job may have built; `upload.js` the upload dialog the Library opens on demand, hashing in the module worker `hash-worker.js` with the in-house incremental SHA-256 of `sha256.js`, self-tested before its first use, and sending the capture time `capture-time.js` reads: the Exif DateTimeOriginal of a JPEG or TIFF-based file, else the file's time; `reorganize.js` the Reorganize dialog of the Library's selection and of the Archive page's save toast: template switches and fields, a live dry run of `POST /archive/reorganize` one request at a time, Apply that sends the shown plan's fingerprint back; `convert.js` the Convert dialog of the Library's selection: format, JPEG quality and a mode chosen at every run (Replace or New sequence, never preselected), a debounced dry run of `POST /archive/convert` per sequence (photos converted, left as they are and why, blocked, size before and after, the space line), Apply that confirms Replace by a typed word and sends the shown plan's fingerprint back) |
|
||||
|
||||
+8
-1
@@ -92,6 +92,7 @@
|
||||
<data name="Admin.Convert.ChooseMode" xml:space="preserve"><value>Choose Replace or New sequence to convert.</value><comment>long</comment></data>
|
||||
<data name="Admin.Convert.ChooseMode.Tip" xml:space="preserve"><value>Choose Replace or New sequence first</value></data>
|
||||
<data name="Admin.Convert.Column.Converted" xml:space="preserve"><value>Converted</value></data>
|
||||
<data name="Admin.Convert.Column.NotCopied" xml:space="preserve"><value>Not copied</value></data>
|
||||
<data name="Admin.Convert.Column.Sequence" xml:space="preserve"><value>Sequence</value></data>
|
||||
<data name="Admin.Convert.Column.Size" xml:space="preserve"><value>Size</value></data>
|
||||
<data name="Admin.Convert.Column.Unchanged" xml:space="preserve"><value>Unchanged</value></data>
|
||||
@@ -101,6 +102,8 @@
|
||||
<data name="Admin.Convert.Confirm.Replace.Tip" xml:space="preserve"><value>Queue the job that converts the photos, verifies each conversion and moves the originals to the trash</value></data>
|
||||
<data name="Admin.Convert.ConfirmNew.one" xml:space="preserve"><value>{0} photo is converted into a copy in a new sequence; the original stays as it is.</value><comment>long</comment></data>
|
||||
<data name="Admin.Convert.ConfirmNew.other" xml:space="preserve"><value>{0} photos are converted into copies in a new sequence beside each one; the originals stay as they are.</value><comment>long</comment></data>
|
||||
<data name="Admin.Convert.ConfirmNewNotCopied.one" xml:space="preserve"><value>{0} photo is not copied into the new sequence, for the reason the preview gives.</value><comment>long</comment></data>
|
||||
<data name="Admin.Convert.ConfirmNewNotCopied.other" xml:space="preserve"><value>{0} photos are not copied into the new sequences, for the reasons the preview gives.</value><comment>long</comment></data>
|
||||
<data name="Admin.Convert.ConfirmNewTitle" xml:space="preserve"><value>Convert to {0} in new sequences?</value><comment>long</comment></data>
|
||||
<data name="Admin.Convert.ConfirmReplace.one" xml:space="preserve"><value>{0} photo in {1} is replaced by its conversion; the original goes to the trash, restorable until it is emptied.</value><comment>long</comment></data>
|
||||
<data name="Admin.Convert.ConfirmReplace.other" xml:space="preserve"><value>{0} photos in {1} are replaced by their conversions; the originals go to the trash, restorable until it is emptied.</value><comment>long</comment></data>
|
||||
@@ -118,14 +121,17 @@
|
||||
<data name="Admin.Convert.Mode.NewSequence.Tip" xml:space="preserve"><value>The converted copies go into a new sequence beside each one; the originals and their sequence stay exactly as they are</value></data>
|
||||
<data name="Admin.Convert.Mode.Replace" xml:space="preserve"><value>Replace</value></data>
|
||||
<data name="Admin.Convert.Mode.Replace.Tip" xml:space="preserve"><value>The converted files become the sequence's photos; the originals go to the archive's trash, restorable until it is emptied</value></data>
|
||||
<data name="Admin.Convert.NoAnswer" xml:space="preserve"><value>The server did not answer: check on the Jobs page whether the conversion was queued before converting again.</value><comment>long</comment></data>
|
||||
<data name="Admin.Convert.NoSpace.Tip" xml:space="preserve"><value>Free some space on the archive's volume first</value></data>
|
||||
<data name="Admin.Convert.Nothing.Tip" xml:space="preserve"><value>Nothing in the preview converts</value></data>
|
||||
<data name="Admin.Convert.NothingToConvert" xml:space="preserve"><value>Nothing to convert: every photo stays as it is, for the reasons shown.</value><comment>long</comment></data>
|
||||
<data name="Admin.Convert.OpenJob" xml:space="preserve"><value>Open job</value></data>
|
||||
<data name="Admin.Convert.OpenJob.Tip" xml:space="preserve"><value>Show the job and its log on the Jobs page</value></data>
|
||||
<data name="Admin.Convert.PlanItem" xml:space="preserve"><value>{0} · {1} · {2}</value><comment>invariant; a sequence's title, the photos it converts, and their size before and after</comment></data>
|
||||
<data name="Admin.Convert.PlanChanged" xml:space="preserve"><value>The archive changed during the confirmation; check the new preview and apply again.</value><comment>long</comment></data>
|
||||
<data name="Admin.Convert.PlanChanged" xml:space="preserve"><value>The archive changed since the preview; check the new one and apply again.</value><comment>long</comment></data>
|
||||
<data name="Admin.Convert.Planning" xml:space="preserve"><value>Planning…</value></data>
|
||||
<data name="Admin.Convert.Planning.Tip" xml:space="preserve"><value>The server reads every photo folder of the selection: a large archive takes minutes, and the preview appears once it is done</value></data>
|
||||
<data name="Admin.Convert.PlanningFor" xml:space="preserve"><value>Planning… {0}</value><comment>{0} how long the dry run has taken so far, e.g. "2 min"</comment></data>
|
||||
<data name="Admin.Convert.Preview" xml:space="preserve"><value>Preview</value></data>
|
||||
<data name="Admin.Convert.Preview.Tip" xml:space="preserve"><value>What each sequence converts and leaves as it is, with the size before and after (an estimate); nothing has changed yet</value></data>
|
||||
<data name="Admin.Convert.Quality" xml:space="preserve"><value>Quality</value></data>
|
||||
@@ -141,6 +147,7 @@
|
||||
<data name="Admin.Convert.TitleAll" xml:space="preserve"><value>Convert all sequences</value></data>
|
||||
<data name="Admin.Convert.TooMany" xml:space="preserve"><value>Convert takes at most {0} sequences at once, or every sequence: clear the filters and select all.</value><comment>long</comment></data>
|
||||
<data name="Admin.Convert.Totals" xml:space="preserve"><value>Sequences converted: {0} · photos: {1} · unchanged: {2} · size: {3} → {4}</value><comment>long</comment></data>
|
||||
<data name="Admin.Convert.TotalsNew" xml:space="preserve"><value>Sequences converted: {0} · photos: {1} · not copied: {2} · size: {3} → {4}</value><comment>long: the totals with New sequence chosen; {2} the photos the new sequences lack</comment></data>
|
||||
<data name="Admin.Dashboard.Activity" xml:space="preserve"><value>Recent activity</value></data>
|
||||
<data name="Admin.Dashboard.AllLogs" xml:space="preserve"><value>All logs</value></data>
|
||||
<data name="Admin.Dashboard.AllLogs.Tip" xml:space="preserve"><value>Open the audit log</value></data>
|
||||
|
||||
@@ -59,6 +59,14 @@
|
||||
<data name="Archive.Convert.HeadingAll" xml:space="preserve"><value>Convert {0}</value><comment>long: dialog heading; {0} archive name</comment></data>
|
||||
<data name="Archive.Convert.Mode" xml:space="preserve"><value>Mode</value></data>
|
||||
<data name="Archive.Convert.NewSequence" xml:space="preserve"><value>New sequence</value></data>
|
||||
<data name="Archive.Convert.NotCopied.Missing" xml:space="preserve"><value>Missing photos are not copied into the new sequence.</value><comment>long: why photos are not in the new sequence a conversion makes, in its dry run</comment></data>
|
||||
<data name="Archive.Convert.NotCopied.NoNativeDecoder" xml:space="preserve"><value>Photos of a format this computer cannot read at full resolution are not copied into the new sequence.</value><comment>long: why photos are not in the new sequence a conversion makes, in its dry run</comment></data>
|
||||
<data name="Archive.Convert.NotCopied.PreviewOnly" xml:space="preserve"><value>Photos read only from their embedded preview here are not copied into the new sequence.</value><comment>long: why photos are not in the new sequence a conversion makes, in its dry run</comment></data>
|
||||
<data name="Archive.Convert.NotCopied.SameFormat" xml:space="preserve"><value>Photos already in this format are not copied into the new sequence.</value><comment>long: why photos are not in the new sequence a conversion makes, in its dry run</comment></data>
|
||||
<data name="Archive.Convert.NotCopied.Undecodable" xml:space="preserve"><value>Unreadable photos are not copied into the new sequence.</value><comment>long: why photos are not in the new sequence a conversion makes, in its dry run</comment></data>
|
||||
<data name="Archive.Convert.NotCopied.Unhashed" xml:space="preserve"><value>Photos never verified are not copied into the new sequence; verify the sequence first.</value><comment>long: why photos are not in the new sequence a conversion makes, in its dry run</comment></data>
|
||||
<data name="Archive.Convert.NotCopiedPhotos.one" xml:space="preserve"><value>{0:N0} not copied</value><comment>photos a conversion into a new sequence leaves out of it</comment></data>
|
||||
<data name="Archive.Convert.NotCopiedPhotos.other" xml:space="preserve"><value>{0:N0} not copied</value><comment>photos a conversion into a new sequence leaves out of it</comment></data>
|
||||
<data name="Archive.Convert.NotPlaced" xml:space="preserve"><value>The converted photos could not take their place. {0}</value><comment>long: job log; {0} why (one sentence)</comment></data>
|
||||
<data name="Archive.Convert.PhotoFailed" xml:space="preserve"><value>{0} could not be converted. {1}</value><comment>long: job log; {0} the photo's file, {1} why (one sentence)</comment></data>
|
||||
<data name="Archive.Convert.Quality" xml:space="preserve"><value>Quality</value></data>
|
||||
|
||||
@@ -24,6 +24,7 @@
|
||||
<data name="Server.Conflict.TakeServer.Title" xml:space="preserve"><value>Take the server's settings?</value></data>
|
||||
<data name="Server.Connect" xml:space="preserve"><value>Connect</value></data>
|
||||
<data name="Server.Convert.OnPc" xml:space="preserve"><value>Convert these photos on a PC: the server cannot read them at full resolution.</value><comment>long: why photos are not converted in a conversion dry run on the server</comment></data>
|
||||
<data name="Server.Convert.OnPcNew" xml:space="preserve"><value>Not copied into the new sequence: the server cannot read these photos at full resolution; convert them on a PC.</value><comment>long: why photos are not in the new sequence a conversion on the server makes, in its dry run</comment></data>
|
||||
<data name="Server.Disconnect" xml:space="preserve"><value>Disconnect</value></data>
|
||||
<data name="Server.Empty" xml:space="preserve"><value>Not connected to a server</value></data>
|
||||
<data name="Server.Error.AccountLocked" xml:space="preserve"><value>Too many failed sign-ins; the account is locked for a while.</value><comment>long</comment></data>
|
||||
|
||||
@@ -92,6 +92,7 @@
|
||||
<data name="Admin.Convert.ChooseMode" xml:space="preserve"><value>Scegli Sostituisci o Nuova sequenza per convertire.</value></data>
|
||||
<data name="Admin.Convert.ChooseMode.Tip" xml:space="preserve"><value>Prima scegli Sostituisci o Nuova sequenza</value></data>
|
||||
<data name="Admin.Convert.Column.Converted" xml:space="preserve"><value>Convertite</value></data>
|
||||
<data name="Admin.Convert.Column.NotCopied" xml:space="preserve"><value>Non copiate</value></data>
|
||||
<data name="Admin.Convert.Column.Sequence" xml:space="preserve"><value>Sequenza</value></data>
|
||||
<data name="Admin.Convert.Column.Size" xml:space="preserve"><value>Dimensione</value></data>
|
||||
<data name="Admin.Convert.Column.Unchanged" xml:space="preserve"><value>Invariate</value></data>
|
||||
@@ -101,6 +102,8 @@
|
||||
<data name="Admin.Convert.Confirm.Replace.Tip" xml:space="preserve"><value>Metti in coda il job che converte le foto, verifica ogni conversione e sposta gli originali nel cestino</value></data>
|
||||
<data name="Admin.Convert.ConfirmNew.one" xml:space="preserve"><value>{0} foto viene convertita in una copia in una nuova sequenza; l'originale resta com'è.</value></data>
|
||||
<data name="Admin.Convert.ConfirmNew.other" xml:space="preserve"><value>{0} foto vengono convertite in copie, in una nuova sequenza accanto a ognuna; gli originali restano come sono.</value></data>
|
||||
<data name="Admin.Convert.ConfirmNewNotCopied.one" xml:space="preserve"><value>{0} foto non viene copiata nella nuova sequenza, per il motivo indicato nell'anteprima.</value></data>
|
||||
<data name="Admin.Convert.ConfirmNewNotCopied.other" xml:space="preserve"><value>{0} foto non vengono copiate nelle nuove sequenze, per i motivi indicati nell'anteprima.</value></data>
|
||||
<data name="Admin.Convert.ConfirmNewTitle" xml:space="preserve"><value>Convertire in {0} in nuove sequenze?</value></data>
|
||||
<data name="Admin.Convert.ConfirmReplace.one" xml:space="preserve"><value>{0} foto in {1} viene sostituita dalla sua conversione; l'originale va nel cestino, ripristinabile finché non viene svuotato.</value></data>
|
||||
<data name="Admin.Convert.ConfirmReplace.other" xml:space="preserve"><value>{0} foto in {1} vengono sostituite dalle loro conversioni; gli originali vanno nel cestino, ripristinabili finché non viene svuotato.</value></data>
|
||||
@@ -116,13 +119,16 @@
|
||||
<data name="Admin.Convert.Mode.NewSequence.Tip" xml:space="preserve"><value>Le copie convertite vanno in una nuova sequenza accanto a ognuna; gli originali e la loro sequenza restano esattamente come sono</value></data>
|
||||
<data name="Admin.Convert.Mode.Replace" xml:space="preserve"><value>Sostituisci</value></data>
|
||||
<data name="Admin.Convert.Mode.Replace.Tip" xml:space="preserve"><value>I file convertiti diventano le foto della sequenza; gli originali vanno nel cestino dell'archivio, ripristinabili finché non viene svuotato</value></data>
|
||||
<data name="Admin.Convert.NoAnswer" xml:space="preserve"><value>Il server non ha risposto: controlla nella pagina Attività se la conversione è stata accodata prima di convertire di nuovo.</value></data>
|
||||
<data name="Admin.Convert.NoSpace.Tip" xml:space="preserve"><value>Prima libera spazio sul volume dell'archivio</value></data>
|
||||
<data name="Admin.Convert.Nothing.Tip" xml:space="preserve"><value>Nell'anteprima non c'è niente da convertire</value></data>
|
||||
<data name="Admin.Convert.NothingToConvert" xml:space="preserve"><value>Niente da convertire: ogni foto resta com'è, per i motivi indicati.</value></data>
|
||||
<data name="Admin.Convert.OpenJob" xml:space="preserve"><value>Apri job</value></data>
|
||||
<data name="Admin.Convert.OpenJob.Tip" xml:space="preserve"><value>Mostra il job e il suo log nella pagina Job</value></data>
|
||||
<data name="Admin.Convert.PlanChanged" xml:space="preserve"><value>L'archivio è cambiato durante la conferma; controlla la nuova anteprima e applica di nuovo.</value></data>
|
||||
<data name="Admin.Convert.PlanChanged" xml:space="preserve"><value>L'archivio è cambiato dopo l'anteprima; controlla quella nuova e applica di nuovo.</value></data>
|
||||
<data name="Admin.Convert.Planning" xml:space="preserve"><value>Pianificazione…</value></data>
|
||||
<data name="Admin.Convert.Planning.Tip" xml:space="preserve"><value>Il server legge ogni cartella di foto della selezione: un archivio grande richiede minuti, e l'anteprima compare quando ha finito</value></data>
|
||||
<data name="Admin.Convert.PlanningFor" xml:space="preserve"><value>Pianificazione… {0}</value></data>
|
||||
<data name="Admin.Convert.Preview" xml:space="preserve"><value>Anteprima</value></data>
|
||||
<data name="Admin.Convert.Preview.Tip" xml:space="preserve"><value>Cosa converte e cosa lascia com'è ogni sequenza, con la dimensione prima e dopo (una stima); niente è ancora cambiato</value></data>
|
||||
<data name="Admin.Convert.Quality" xml:space="preserve"><value>Qualità</value></data>
|
||||
@@ -137,6 +143,7 @@
|
||||
<data name="Admin.Convert.TitleAll" xml:space="preserve"><value>Converti tutte le sequenze</value></data>
|
||||
<data name="Admin.Convert.TooMany" xml:space="preserve"><value>Converti al massimo {0} sequenze alla volta, o tutte: togli i filtri e seleziona tutto.</value></data>
|
||||
<data name="Admin.Convert.Totals" xml:space="preserve"><value>Sequenze convertite: {0} · foto: {1} · invariate: {2} · dimensione: {3} → {4}</value></data>
|
||||
<data name="Admin.Convert.TotalsNew" xml:space="preserve"><value>Sequenze convertite: {0} · foto: {1} · non copiate: {2} · dimensione: {3} → {4}</value></data>
|
||||
<data name="Admin.Dashboard.Activity" xml:space="preserve"><value>Attività recenti</value></data>
|
||||
<data name="Admin.Dashboard.AllLogs" xml:space="preserve"><value>Tutti i log</value></data>
|
||||
<data name="Admin.Dashboard.AllLogs.Tip" xml:space="preserve"><value>Apri il registro di controllo</value></data>
|
||||
|
||||
@@ -56,6 +56,14 @@
|
||||
<data name="Archive.Convert.HeadingAll" xml:space="preserve"><value>Converti {0}</value></data>
|
||||
<data name="Archive.Convert.Mode" xml:space="preserve"><value>Modalità</value></data>
|
||||
<data name="Archive.Convert.NewSequence" xml:space="preserve"><value>Nuova sequenza</value></data>
|
||||
<data name="Archive.Convert.NotCopied.Missing" xml:space="preserve"><value>Le foto mancanti non vengono copiate nella nuova sequenza.</value></data>
|
||||
<data name="Archive.Convert.NotCopied.NoNativeDecoder" xml:space="preserve"><value>Le foto di un formato che questo computer non legge a piena risoluzione non vengono copiate nella nuova sequenza.</value></data>
|
||||
<data name="Archive.Convert.NotCopied.PreviewOnly" xml:space="preserve"><value>Le foto lette qui solo dalla loro anteprima incorporata non vengono copiate nella nuova sequenza.</value></data>
|
||||
<data name="Archive.Convert.NotCopied.SameFormat" xml:space="preserve"><value>Le foto già in questo formato non vengono copiate nella nuova sequenza.</value></data>
|
||||
<data name="Archive.Convert.NotCopied.Undecodable" xml:space="preserve"><value>Le foto illeggibili non vengono copiate nella nuova sequenza.</value></data>
|
||||
<data name="Archive.Convert.NotCopied.Unhashed" xml:space="preserve"><value>Le foto mai verificate non vengono copiate nella nuova sequenza; verifica prima la sequenza.</value></data>
|
||||
<data name="Archive.Convert.NotCopiedPhotos.one" xml:space="preserve"><value>{0:N0} non copiata</value></data>
|
||||
<data name="Archive.Convert.NotCopiedPhotos.other" xml:space="preserve"><value>{0:N0} non copiate</value></data>
|
||||
<data name="Archive.Convert.NotPlaced" xml:space="preserve"><value>Le foto convertite non hanno potuto prendere il loro posto. {0}</value></data>
|
||||
<data name="Archive.Convert.PhotoFailed" xml:space="preserve"><value>Impossibile convertire {0}. {1}</value></data>
|
||||
<data name="Archive.Convert.Quality" xml:space="preserve"><value>Qualità</value></data>
|
||||
|
||||
@@ -22,6 +22,7 @@
|
||||
<data name="Server.Conflict.TakeServer.Title" xml:space="preserve"><value>Usare le impostazioni del server?</value></data>
|
||||
<data name="Server.Connect" xml:space="preserve"><value>Connetti</value></data>
|
||||
<data name="Server.Convert.OnPc" xml:space="preserve"><value>Converti queste foto su un PC: il server non le legge a piena risoluzione.</value></data>
|
||||
<data name="Server.Convert.OnPcNew" xml:space="preserve"><value>Non copiate nella nuova sequenza: il server non legge queste foto a piena risoluzione; convertile su un PC.</value></data>
|
||||
<data name="Server.Disconnect" xml:space="preserve"><value>Disconnetti</value></data>
|
||||
<data name="Server.Empty" xml:space="preserve"><value>Non connesso a un server</value></data>
|
||||
<data name="Server.Error.AccountLocked" xml:space="preserve"><value>Troppi accessi falliti; l'account è bloccato per un po'.</value></data>
|
||||
|
||||
@@ -124,6 +124,11 @@ public sealed class ConvertDialogViewModel : ArchiveDialogViewModel
|
||||
{
|
||||
if (value is { } mode && !Enum.IsDefined(mode)) return;
|
||||
if (!Set(ref _mode, value)) return;
|
||||
// The rows say what becomes of the photos skipped in the mode's words: not copied into a new sequence.
|
||||
lock (_gate)
|
||||
{
|
||||
if (_run.Plan is { } plan) _run = _run with { Rows = RowsOf(plan) };
|
||||
}
|
||||
OnPropertyChanged(nameof(IsReplace));
|
||||
OnPropertyChanged(nameof(IsNewSequence));
|
||||
OnPlanChanged();
|
||||
@@ -218,7 +223,7 @@ public sealed class ConvertDialogViewModel : ArchiveDialogViewModel
|
||||
{
|
||||
_planning?.Cancel();
|
||||
_planning = null;
|
||||
_run = new DryRun(Current(), plan, ConvertPlans.Rows(plan));
|
||||
_run = new DryRun(Current(), plan, RowsOf(plan));
|
||||
}
|
||||
BusyText = null;
|
||||
ClearError();
|
||||
@@ -238,11 +243,14 @@ public sealed class ConvertDialogViewModel : ArchiveDialogViewModel
|
||||
Quality.Refresh();
|
||||
lock (_gate)
|
||||
{
|
||||
if (_run.Plan is { } plan) _run = _run with { Rows = ConvertPlans.Rows(plan) };
|
||||
if (_run.Plan is { } plan) _run = _run with { Rows = RowsOf(plan) };
|
||||
}
|
||||
base.OnLanguageChanged();
|
||||
}
|
||||
|
||||
/// <summary>The rows of <paramref name="plan"/> in the mode chosen (as planned while none is: the skipped photos stay as they are).</summary>
|
||||
private IReadOnlyList<ConvertRow> RowsOf(ConvertPlan plan) => ConvertPlans.Rows(_mode is { } mode ? plan with { Mode = mode } : plan);
|
||||
|
||||
/// <summary>Typing makes the dry run on screen old at once (Apply waits) and plans again once the typing pauses.</summary>
|
||||
private void OnTyped()
|
||||
{
|
||||
@@ -290,7 +298,8 @@ public sealed class ConvertDialogViewModel : ArchiveDialogViewModel
|
||||
try
|
||||
{
|
||||
var plan = await Task.Run(() => _archives.PlanConvertAsync(_archive.Id, set, options!, mode, cts.Token), cts.Token);
|
||||
result = new DryRun(options, plan, ConvertPlans.Rows(plan));
|
||||
// In the mode chosen now: it may have been chosen while the plan was made.
|
||||
result = new DryRun(options, plan, RowsOf(plan));
|
||||
}
|
||||
catch (OperationCanceledException) when (cts.IsCancellationRequested)
|
||||
{
|
||||
|
||||
@@ -42,14 +42,22 @@ public static class ConvertPlans
|
||||
/// <summary>A valid quality as typed (see <see cref="ValidateQuality"/>).</summary>
|
||||
public static int ParseQuality(string text) => int.Parse(text.Trim(), NumberStyles.Integer, Loc.Culture);
|
||||
|
||||
/// <summary>The rows of a plan in its order: every sequence that converts, or whose photos stay as they are, with why.</summary>
|
||||
/// <summary>
|
||||
/// The rows of a plan in its order: every sequence that converts, or whose photos stay as they are, with why in the
|
||||
/// words of the plan's mode (<see cref="Row"/>).
|
||||
/// </summary>
|
||||
public static IReadOnlyList<ConvertRow> Rows(ConvertPlan plan)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(plan);
|
||||
return [.. plan.Sequences.Where(s => s.Frames.Count > 0 || s.Skipped.Count > 0 || s.Blocked.Count > 0).Select(Row)];
|
||||
return [.. plan.Sequences.Where(s => s.Frames.Count > 0 || s.Skipped.Count > 0 || s.Blocked.Count > 0).Select(s => Row(s, plan.Mode))];
|
||||
}
|
||||
|
||||
public static ConvertRow Row(ConvertSequencePlan step)
|
||||
/// <summary>
|
||||
/// One sequence of a plan: what converts, then what blocks it, then the photos skipped with why. With
|
||||
/// <see cref="ConvertMode.NewSequence"/> those are not copied into the new sequence, which holds the converted copies
|
||||
/// alone; with <see cref="ConvertMode.Replace"/> they stay as they are.
|
||||
/// </summary>
|
||||
public static ConvertRow Row(ConvertSequencePlan step, ConvertMode mode = ConvertMode.Replace)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(step);
|
||||
var named = !string.IsNullOrWhiteSpace(step.Title);
|
||||
@@ -59,7 +67,8 @@ public static class ConvertPlans
|
||||
: string.Empty;
|
||||
// What blocks the whole sequence comes first: the photos it would skip anyway do not matter until it can convert.
|
||||
List<string> kept = [.. step.Blocked.Distinct().Select(b => Loc.L("Archive.Convert.Blocked." + b))];
|
||||
kept.AddRange(step.Skipped.Select(s => Loc.L("Archive.Convert.Skipped." + s.Reason) + Loc.Separator + Loc.Plural("Shell.Status.Photos", s.Frames)));
|
||||
var why = mode == ConvertMode.NewSequence ? "Archive.Convert.NotCopied." : "Archive.Convert.Skipped.";
|
||||
kept.AddRange(step.Skipped.Select(s => Loc.L(why + s.Reason) + Loc.Separator + Loc.Plural("Shell.Status.Photos", s.Frames)));
|
||||
return new ConvertRow(step.SequenceId, name, !named, note, kept, step.Blocked.Count > 0);
|
||||
}
|
||||
|
||||
@@ -68,7 +77,8 @@ public static class ConvertPlans
|
||||
|
||||
/// <summary>
|
||||
/// "12 of 36 change · 3,600 photos · 41 GB → 32 GB · 20 skipped": the sequences converted of those planned, then the
|
||||
/// photos and their sizes when something converts, the photos skipped when there are any.
|
||||
/// photos and their sizes when something converts, the photos skipped when there are any ("20 not copied" with
|
||||
/// <see cref="ConvertMode.NewSequence"/>: the new sequences lack them).
|
||||
/// </summary>
|
||||
public static string Summary(ConvertPlan plan)
|
||||
{
|
||||
@@ -79,7 +89,8 @@ public static class ConvertPlans
|
||||
parts.Add(Loc.Plural("Shell.Status.Photos", plan.Frames));
|
||||
parts.Add(Sizes(plan.BytesBefore, plan.BytesAfter));
|
||||
}
|
||||
if (plan.Skipped > 0) parts.Add(Loc.Plural("Archive.Convert.SkippedPhotos", plan.Skipped));
|
||||
if (plan.Skipped > 0)
|
||||
parts.Add(Loc.Plural(plan.Mode == ConvertMode.NewSequence ? "Archive.Convert.NotCopiedPhotos" : "Archive.Convert.SkippedPhotos", plan.Skipped));
|
||||
return string.Join(Loc.Separator, parts);
|
||||
}
|
||||
|
||||
|
||||
@@ -469,6 +469,50 @@ public sealed class SequenceRepository
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// A digest of what a conversion of <paramref name="sequenceIds"/> (null: every sequence) plans from in the index of
|
||||
/// <paramref name="archiveId"/> (owner review 2, R7): what a Reorganize's does (<see cref="GetStampAsync"/>: every sequence
|
||||
/// and its folder, which decide what "every sequence" is and its order; of the sequences planned, their title, photo
|
||||
/// count, frames version and how many of their photos are hashed), and of the photos of the sequences planned, what
|
||||
/// decides whether each converts and the size it is estimated at: their format, size and pixel size, and the flags a
|
||||
/// scan, a verify or a decode records without another frames version (missing, corrupt, read only from the preview,
|
||||
/// undecodable). Another value once one of these changes. A change made on disk alone, which the index does not have
|
||||
/// (a file gone before a scan finds it), leaves it as it was.
|
||||
/// </summary>
|
||||
public async Task<string> GetConvertStampAsync(Guid archiveId, IReadOnlyCollection<Guid>? sequenceIds, CancellationToken cancellationToken = default)
|
||||
{
|
||||
var stamp = await GetStampAsync(archiveId, sequenceIds, cancellationToken).ConfigureAwait(false);
|
||||
// Per sequence planned: its photos, their bytes, formats and pixel sizes, and how many carry each flag that decides.
|
||||
var columns = "count(*), COALESCE(SUM(f.size), 0), COALESCE(SUM(f.format), 0), COALESCE(SUM(f.width), 0), COALESCE(SUM(f.height), 0), " +
|
||||
string.Join(", ", new[] { FrameFlags.Missing, FrameFlags.Corrupt, FrameFlags.PreviewOnly, FrameFlags.Undecodable }
|
||||
.Select(flag => $"COALESCE(SUM(CASE WHEN (f.flags & {(int)flag}) <> 0 THEN 1 ELSE 0 END), 0)"));
|
||||
static (Guid Id, string Photos) Read(DbRow r) =>
|
||||
(r.GetGuid(0), string.Join('\u001f', Enumerable.Range(1, 9).Select(i => r.GetInt64(i).ToString(CultureInfo.InvariantCulture))));
|
||||
var sums = new List<(Guid Id, string Photos)>();
|
||||
if (sequenceIds is null)
|
||||
{
|
||||
sums.AddRange(await _db.ListAsync(
|
||||
$"SELECT f.sequence_id, {columns} FROM frames f JOIN sequences s ON s.id = f.sequence_id WHERE s.archive_id = $1 GROUP BY f.sequence_id",
|
||||
[archiveId], Read, cancellationToken).ConfigureAwait(false));
|
||||
}
|
||||
else
|
||||
{
|
||||
// 500 ids per statement stays below SQLite's historical 999-variable limit.
|
||||
foreach (var chunk in sequenceIds.Distinct().Chunk(500))
|
||||
{
|
||||
var p = new SqlParameters();
|
||||
var ids = string.Join(", ", chunk.Select(id => p.Add(id)));
|
||||
sums.AddRange(await _db.ListAsync($"SELECT f.sequence_id, {columns} FROM frames f WHERE f.sequence_id IN ({ids}) GROUP BY f.sequence_id",
|
||||
p.ToArray(), Read, cancellationToken).ConfigureAwait(false));
|
||||
}
|
||||
}
|
||||
using var hash = IncrementalHash.CreateHash(HashAlgorithmName.SHA256);
|
||||
hash.AppendData(Encoding.UTF8.GetBytes(stamp));
|
||||
foreach (var (id, photos) in sums.OrderBy(s => s.Id))
|
||||
hash.AppendData(Encoding.UTF8.GetBytes(string.Create(CultureInfo.InvariantCulture, $"\n{id:N}\u001f{photos}")));
|
||||
return Convert.ToHexStringLower(hash.GetHashAndReset());
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// How many photos of each of <paramref name="sequenceIds"/> (null: every sequence of <paramref name="archiveId"/>) have
|
||||
/// their SHA-256 in the index; a sequence without one is not listed.
|
||||
|
||||
@@ -131,9 +131,13 @@ public sealed record ReorganizePlanDto(IReadOnlyList<ReorganizeStepDto> Steps, i
|
||||
/// <see cref="Target"/> (<c>jpeg</c>, <c>tiff16</c> or <c>linear-dng</c>, the archival choice) with <see cref="Quality"/> 1–100
|
||||
/// for a JPEG (default 95, ignored otherwise). <see cref="Mode"/>, chosen at every run: <c>replace</c> (the converted files
|
||||
/// become the frames, the originals go to the archive's trash) or <c>new-sequence</c> (copies in a new sequence, the source
|
||||
/// untouched). <see cref="DryRun"/> answers the plan (<see cref="ConvertPlanDto"/>) and changes nothing; the change itself
|
||||
/// sends that plan's <see cref="Fingerprint"/> back (409 <c>plan_changed</c> when the archive changed since) and runs as a
|
||||
/// <c>convert</c> job (202).
|
||||
/// untouched, the photos it leaves as they are not copied into it). <see cref="DryRun"/> answers the plan
|
||||
/// (<see cref="ConvertPlanDto"/>) and changes nothing; the server keeps it a few minutes, a refusal too, so the same dry run
|
||||
/// again (also after one cut off, in either mode, at any quality of a lossless target) is answered from it with the free
|
||||
/// space measured again; 503 <c>plan_dropped</c> when the server stopped making it (it took too long, or too many were
|
||||
/// asked at once). The change itself sends that plan's <see cref="Fingerprint"/> back (409 <c>plan_changed</c> when the
|
||||
/// index changed since in what the plan reads: the sequences' rows, and the formats, sizes, hashes and flags of their
|
||||
/// photos) and runs as a <c>convert</c> job (202), which plans again and leaves as it is a sequence that no longer fits.
|
||||
/// </summary>
|
||||
public sealed record ConvertRequest(IReadOnlyList<Guid>? SequenceIds, bool All, string? Target, int? Quality, string? Mode, bool DryRun,
|
||||
string? Fingerprint = null);
|
||||
@@ -143,7 +147,8 @@ public sealed record ConvertRequest(IReadOnlyList<Guid>? SequenceIds, bool All,
|
||||
/// server cannot read them at full resolution: a CR3, a HEIC, a body its RAW decoder does not support), <c>undecodable</c>,
|
||||
/// <c>missing</c>, <c>unhashed</c> (never verified), <c>eight_bit</c> (a RAW the host develops at 8 bits only, for a 16-bit
|
||||
/// target), <c>color_profile</c> (a colour profile the target can neither keep nor convert to) or <c>too_large</c>, how
|
||||
/// many, and the reason as one localised sentence.
|
||||
/// many, and the reason as one localised sentence: with <c>new-sequence</c>, that the new sequence lacks them (it holds the
|
||||
/// converted copies alone).
|
||||
/// </summary>
|
||||
public sealed record ConvertSkippedDto(string Reason, int Photos, string Message);
|
||||
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
// Convert dialog of the admin panel (owner review 2, R7; BRIEF §12.5, §14.8): the photos of the Library's selection, or of
|
||||
// every sequence, converted to JPEG, TIFF-16 or linear DNG. What happens to the originals is chosen at every run, never
|
||||
// preselected: Replace (the converted files become the sequence's photos, the originals go to the archive's trash) or New
|
||||
// sequence (converted copies in a new sequence, the source untouched). Each change asks the server for a dry run (nothing
|
||||
// is touched) and the table shows, per sequence, what converts, what stays as it is and why, and the size before and
|
||||
// after. Apply confirms that very plan by its fingerprint (Replace by typing a word), and the server runs it as a job
|
||||
// (kind "convert") that verifies every converted file before anything changes.
|
||||
// sequence (converted copies in a new sequence, the source untouched; the photos left as they are are not in it). Each
|
||||
// change asks the server for a dry run (nothing is touched) and the table shows, per sequence, what converts, what stays
|
||||
// as it is and why, and the size before and after. The server makes a dry run apart from the request and keeps it a few
|
||||
// minutes, so one that takes longer than a proxy lets a request last is asked for again until it is made. Apply confirms
|
||||
// that very plan by its fingerprint (Replace by typing a word), and the server runs it as a job (kind "convert") that
|
||||
// verifies every converted file before anything changes.
|
||||
import { t, plural, applyStrings } from "./i18n.js";
|
||||
import { escapeHtml, attr, formatBytes, formatNumber, toast, errorText, confirmDialog, markInvalid, clearInvalid } from "./ui.js";
|
||||
import { escapeHtml, attr, formatBytes, formatNumber, formatDuration, toast, errorText, confirmDialog, markInvalid, clearInvalid } from "./ui.js";
|
||||
import { listed } from "./pages/library/shared.js";
|
||||
|
||||
const ENDPOINT = "/api/v1/archive/convert";
|
||||
@@ -14,6 +16,16 @@ const ENDPOINT = "/api/v1/archive/convert";
|
||||
const PLAN_DELAY = 400;
|
||||
const PAGE = 200;
|
||||
const DEFAULT_QUALITY = 95;
|
||||
/**
|
||||
* A dry run the server is still making: a request cut off after CUT_AFTER ms or by a proxy's timeout (502, 504, 524, or
|
||||
* no answer) is asked again after RETRY_DELAY ms, for up to PLAN_PATIENCE ms (the server stops a plan after 15 minutes);
|
||||
* one the server stopped (503 plan_dropped) DROPPED_RETRIES times more. Past SLOW_PLAN seconds the totals tell how long.
|
||||
*/
|
||||
const CUT_AFTER = 20000;
|
||||
const RETRY_DELAY = 2000;
|
||||
const PLAN_PATIENCE = 16 * 60 * 1000;
|
||||
const DROPPED_RETRIES = 1;
|
||||
const SLOW_PLAN = 10;
|
||||
/** The formats, as the API names them, with their strings; linear DNG, the archival choice, is the one chosen first. */
|
||||
const TARGETS = [
|
||||
{ value: "jpeg", label: "Admin.Convert.Format.Jpeg", tip: "Admin.Convert.Format.Jpeg.Tip" },
|
||||
@@ -27,10 +39,12 @@ const MODES = [
|
||||
{ value: "new-sequence", label: "Admin.Convert.Mode.NewSequence", tip: "Admin.Convert.Mode.NewSequence.Tip" },
|
||||
];
|
||||
/**
|
||||
* The mode the dry run is asked with while none is chosen: what converts does not depend on the mode, so the table shows
|
||||
* the plan at once; Apply waits for a choice, and a choice other than this one plans again (the fingerprint names it).
|
||||
* The mode the dry run is asked with while none is chosen, whose words say only that photos stay as they are: what
|
||||
* converts does not depend on the mode, so the table shows the plan at once; Apply waits for a choice, and New sequence
|
||||
* asks again (the fingerprint names the mode, and the rows say what the new sequence lacks), answered from the plan the
|
||||
* server keeps.
|
||||
*/
|
||||
const PREVIEW_MODE = "new-sequence";
|
||||
const PREVIEW_MODE = "replace";
|
||||
|
||||
let dialogs = 0;
|
||||
|
||||
@@ -80,7 +94,7 @@ export function openConvert({ api, target, names = new Map() } = {}) {
|
||||
<div class="table convert" role="table" aria-label="${attr(t("Admin.Convert.Preview"))}">
|
||||
<div class="thead" role="row"><span role="columnheader" data-i18n="Admin.Convert.Column.Sequence"></span>
|
||||
<span role="columnheader" class="end" data-i18n="Admin.Convert.Column.Converted"></span>
|
||||
<span role="columnheader" data-i18n="Admin.Convert.Column.Unchanged"></span>
|
||||
<span role="columnheader" class="convert-left" data-i18n="Admin.Convert.Column.Unchanged"></span>
|
||||
<span role="columnheader" class="end" data-i18n="Admin.Convert.Column.Size"></span></div>
|
||||
<div role="rowgroup" class="convert-rows"></div>
|
||||
</div>
|
||||
@@ -105,6 +119,7 @@ export function openConvert({ api, target, names = new Map() } = {}) {
|
||||
const next = dialog.querySelector(".convert-next");
|
||||
const preview = dialog.querySelector(".plan-preview");
|
||||
const rows = dialog.querySelector(".convert-rows");
|
||||
const leftHeader = dialog.querySelector(".convert-left");
|
||||
const foot = dialog.querySelector(".table-foot");
|
||||
const shownText = dialog.querySelector(".convert-shown");
|
||||
const errorLine = dialog.querySelector(".error");
|
||||
@@ -212,10 +227,18 @@ export function openConvert({ api, target, names = new Map() } = {}) {
|
||||
const body = request();
|
||||
const key = JSON.stringify(body);
|
||||
planning = true;
|
||||
totals.textContent = t("Admin.Convert.Planning");
|
||||
// Waiting: how long it has taken, once that is long, with why in the tooltip.
|
||||
const started = Date.now();
|
||||
const showWait = () => {
|
||||
const seconds = Math.floor((Date.now() - started) / 1000);
|
||||
totals.textContent = seconds < SLOW_PLAN ? t("Admin.Convert.Planning") : t("Admin.Convert.PlanningFor", formatDuration(seconds));
|
||||
};
|
||||
const ticking = setInterval(showWait, 1000);
|
||||
showWait();
|
||||
totals.title = t("Admin.Convert.Planning.Tip");
|
||||
drawButtons();
|
||||
try {
|
||||
const answer = await api.post(ENDPOINT, { ...body, dryRun: true });
|
||||
const answer = await dryRun(body, key);
|
||||
if (!dialog.open) return;
|
||||
plan = { ...answer, key };
|
||||
shown = PAGE;
|
||||
@@ -228,6 +251,8 @@ export function openConvert({ api, target, names = new Map() } = {}) {
|
||||
if (key === keyNow()) showProblem(err);
|
||||
} finally {
|
||||
planning = false;
|
||||
clearInterval(ticking);
|
||||
totals.title = "";
|
||||
}
|
||||
if (again) {
|
||||
again = false;
|
||||
@@ -238,6 +263,28 @@ export function openConvert({ api, target, names = new Map() } = {}) {
|
||||
drawPlan();
|
||||
}
|
||||
|
||||
/**
|
||||
* The dry run of body. The server makes it apart from the request and keeps it a few minutes, so a request cut off on
|
||||
* the way (a proxy's timeout) is asked again, and answered once the plan is made; a plan the server stopped (let go for
|
||||
* newer ones, past its time) is asked for once more. Gives up past PLAN_PATIENCE, or once the fields changed or the
|
||||
* dialog closed: the last failure is thrown.
|
||||
*/
|
||||
async function dryRun(body, key) {
|
||||
const started = Date.now();
|
||||
let dropped = 0;
|
||||
for (;;) {
|
||||
const sent = Date.now();
|
||||
try {
|
||||
return await api.post(ENDPOINT, { ...body, dryRun: true });
|
||||
} catch (err) {
|
||||
const retry = cutOff(err, Date.now() - sent) || (err?.code === "plan_dropped" && dropped++ < DROPPED_RETRIES);
|
||||
if (!retry || Date.now() - started > PLAN_PATIENCE || !dialog.open || key !== keyNow()) throw err;
|
||||
await new Promise((done) => setTimeout(done, RETRY_DELAY));
|
||||
if (!dialog.open || key !== keyNow()) throw err;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function drawPlan() {
|
||||
const current = plan !== null && plan.key === keyNow();
|
||||
preview.classList.toggle("stale", !current);
|
||||
@@ -247,8 +294,11 @@ export function openConvert({ api, target, names = new Map() } = {}) {
|
||||
next.hidden = true;
|
||||
preview.hidden = true;
|
||||
} else {
|
||||
// New sequence's words: the photos left as they are are not copied into the new sequence.
|
||||
const copies = plan.mode === "new-sequence";
|
||||
leftHeader.textContent = t(copies ? "Admin.Convert.Column.NotCopied" : "Admin.Convert.Column.Unchanged");
|
||||
totals.textContent = plan.photos === 0 ? t("Admin.Convert.NothingToConvert")
|
||||
: t("Admin.Convert.Totals", formatNumber(plan.sequences), formatNumber(plan.photos), formatNumber(plan.skipped),
|
||||
: t(copies ? "Admin.Convert.TotalsNew" : "Admin.Convert.Totals", formatNumber(plan.sequences), formatNumber(plan.photos), formatNumber(plan.skipped),
|
||||
formatBytes(plan.bytesBefore), formatBytes(plan.bytesAfter));
|
||||
space.hidden = plan.photos === 0;
|
||||
space.textContent = plan.space.message;
|
||||
@@ -309,9 +359,9 @@ export function openConvert({ api, target, names = new Map() } = {}) {
|
||||
|
||||
/**
|
||||
* Replace asks for a word to be typed (the originals leave the sequence for the trash); New sequence for a plain
|
||||
* confirmation. The plan sent back is the one the table showed. The fields cannot change meanwhile, but a dry run
|
||||
* scheduled before may answer during it: the same plan again (same request and fingerprint) is applied all the same; a
|
||||
* different one is shown instead, and Apply waits for the user.
|
||||
* confirmation that says how many photos the new sequences lack. The plan sent back is the one the table showed. The
|
||||
* fields cannot change meanwhile, but a dry run scheduled before may answer during it: the same plan again (same
|
||||
* request and fingerprint) is applied all the same; a different one is shown instead, and Apply waits for the user.
|
||||
*/
|
||||
async function apply() {
|
||||
if (applyButton.disabled) return;
|
||||
@@ -321,6 +371,8 @@ export function openConvert({ api, target, names = new Map() } = {}) {
|
||||
const going = shownPlan.steps.filter((s) => (s.blocked ?? []).length === 0 && s.photos > 0);
|
||||
const list = listed(going.map((s) => t("Admin.Convert.PlanItem", names.get(s.sequenceId) || s.title || folderName(s.folder),
|
||||
plural("Admin.Dashboard.PhotoCount", s.photos), t("Admin.Convert.SizeChange", formatBytes(s.bytesBefore), formatBytes(s.bytesAfter)))));
|
||||
// The photos of the sequences converted that stay as they are: their new sequences hold the converted copies alone.
|
||||
const notCopied = going.reduce((sum, s) => sum + (s.skipped ?? []).reduce((photos, k) => photos + k.photos, 0), 0);
|
||||
const replace = body.mode === "replace";
|
||||
const confirmed = await confirmDialog(replace ? {
|
||||
title: t("Admin.Convert.ConfirmReplaceTitle", format),
|
||||
@@ -334,7 +386,8 @@ export function openConvert({ api, target, names = new Map() } = {}) {
|
||||
typedKind: "word",
|
||||
} : {
|
||||
title: t("Admin.Convert.ConfirmNewTitle", format),
|
||||
body: plural("Admin.Convert.ConfirmNew", shownPlan.photos),
|
||||
body: [plural("Admin.Convert.ConfirmNew", shownPlan.photos), notCopied > 0 ? plural("Admin.Convert.ConfirmNewNotCopied", notCopied) : ""]
|
||||
.filter(Boolean).join(" "),
|
||||
list,
|
||||
confirm: t("Admin.Convert.Confirm.New"),
|
||||
confirmTip: t("Admin.Convert.Confirm.New.Tip"),
|
||||
@@ -360,11 +413,15 @@ export function openConvert({ api, target, names = new Map() } = {}) {
|
||||
} catch (err) {
|
||||
busy = false;
|
||||
if (!dialog.open) return;
|
||||
if (err?.code === "plan_changed" || err?.code === "not_enough_space" || err?.code === "nothing_to_convert") {
|
||||
// The archive changed since the dry run (or its volume filled up): the new plan is shown, and Apply waits for it.
|
||||
notice = err.code === "plan_changed" ? err.message || t("Admin.Error.Unreachable") : errorText(err);
|
||||
if (["plan_changed", "plan_dropped", "not_enough_space", "nothing_to_convert"].includes(err?.code)) {
|
||||
// The archive changed since the dry run (or its volume filled up, or the server let the plan go before it was
|
||||
// confirmed): nothing was queued; the plan is made again and shown, and Apply waits for it.
|
||||
notice = err.code === "plan_changed" ? t("Admin.Convert.PlanChanged") : errorText(err);
|
||||
plan = null;
|
||||
schedule(0);
|
||||
} else if (cutOff(err, Infinity)) {
|
||||
// No answer: the job may have been queued all the same, and Apply again would queue another one.
|
||||
showError(t("Admin.Convert.NoAnswer"));
|
||||
} else {
|
||||
showProblem(err);
|
||||
}
|
||||
@@ -394,6 +451,14 @@ export function openConvert({ api, target, names = new Map() } = {}) {
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a request that failed after ms milliseconds was cut off on the way rather than answered: a proxy's timeout (504,
|
||||
* or Cloudflare's 524), or a closed connection or a 502 once it had lasted CUT_AFTER (sooner, the server is not there).
|
||||
*/
|
||||
function cutOff(err, ms) {
|
||||
return err?.status === 504 || err?.status === 524 || ((err?.status === 0 || err?.status === 502) && ms >= CUT_AFTER);
|
||||
}
|
||||
|
||||
/** A format's name in the panel's language ("jpeg" → "JPEG", "linear-dng" → "Linear DNG"). */
|
||||
function targetLabel(value) {
|
||||
const known = TARGETS.find((o) => o.value === value);
|
||||
|
||||
@@ -161,14 +161,14 @@ public static class ArchiveEndpoints
|
||||
/// completes as it starts, no one: no client is connected yet). Photos a sequence receives between the dry run and the
|
||||
/// confirmation under the names the plan leaves them keep the fingerprint as it was. A dry run's plan is made apart from
|
||||
/// the request, a few at once and each for a limited time, and kept a few minutes, a refusal too, while what it reads
|
||||
/// from the index stays as it was (<see cref="ReorganizePlanCache"/>; the sequences' rows, and how many photos of those
|
||||
/// planned are hashed): the same dry run again (also after the first was cut off), and its confirmation, take it instead
|
||||
/// of planning again. So <c>plan_changed</c> answers a change the index has, photos a verify hashed among them; one made
|
||||
/// from the index stays as it was (<see cref="PlanCache"/>; the sequences' rows, and how many photos of those planned
|
||||
/// are hashed): the same dry run again (also after the first was cut off), and its confirmation, take it instead of
|
||||
/// planning again. So <c>plan_changed</c> answers a change the index has, photos a verify hashed among them; one made
|
||||
/// on disk alone since the dry run (a folder made by hand, a desktop writing to a shared archive before a scan) is found
|
||||
/// by the job, which then fails and changes nothing.
|
||||
/// </summary>
|
||||
private static async Task<IResult> ReorganizeAsync(HttpContext context, Caller caller, ReorganizeRequest request, ChronosDatabase db, ServerArchive archive,
|
||||
[FromServices] ArchiveService? archives, JobEngineHost host, AuditLog audit, ReorganizePlanCache plans)
|
||||
[FromServices] ArchiveService? archives, JobEngineHost host, AuditLog audit, PlanCache plans)
|
||||
{
|
||||
var ct = context.RequestAborted;
|
||||
if (request.All == (request.SequenceIds is not null) || request.SequenceIds is { Count: 0 or > LibraryEndpoints.MaxBulkIds })
|
||||
@@ -181,7 +181,7 @@ public static class ArchiveEndpoints
|
||||
return ApiErrors.Result(context, StatusCodes.Status503ServiceUnavailable, "archive_offline");
|
||||
var sequences = request.All ? null : request.SequenceIds;
|
||||
// Stamped before planning: a change committed while the plan is made gives the next request another key.
|
||||
var key = ReorganizePlanCache.Key.Of(archive.Id, await db.Sequences.GetStampAsync(archive.Id, sequences, ct), sequences, templates);
|
||||
var key = ReorganizePlanKey.Of(archive.Id, await db.Sequences.GetStampAsync(archive.Id, sequences, ct), sequences, templates);
|
||||
RenamePlan plan;
|
||||
try
|
||||
{
|
||||
@@ -258,19 +258,28 @@ public static class ArchiveEndpoints
|
||||
/// Convert: the photos of <c>sequenceIds</c> or, with <c>all</c>, of every sequence of the archive, converted to
|
||||
/// <c>target</c> (<c>jpeg</c>, <c>tiff16</c>, <c>linear-dng</c>) at <c>quality</c> (JPEG, 1–100, default 95) in <c>mode</c>
|
||||
/// (<c>replace</c> or <c>new-sequence</c>; BRIEF §12.5). With <c>dryRun</c> the plan, nothing touched: per sequence the
|
||||
/// photos converted, their size and the estimate after, the photos left as they are with why (<c>convert_on_pc</c> for a
|
||||
/// format the server cannot read at full resolution), what blocks it; the totals; the space needed and free; the
|
||||
/// fingerprint. Otherwise the request carries that fingerprint and the server queues a <c>convert</c> job (202) that
|
||||
/// applies exactly that plan, leaving as it is (and reporting) a sequence that changed before it ran. 400
|
||||
/// <c>bad_request</c> (details <c>sequenceIds</c>: neither or both of them, none, or more than
|
||||
/// <see cref="LibraryEndpoints.MaxBulkIds"/>; <c>target</c>, <c>quality</c>, <c>mode</c>, <c>fingerprint</c>); 404
|
||||
/// <c>not_found</c> (<c>sequence</c>: not in the archive); 409 <c>plan_changed</c> (details: the fingerprint now),
|
||||
/// <c>nothing_to_convert</c>, <c>not_enough_space</c> (details: the bytes needed), <c>job_unavailable</c>; 503
|
||||
/// <c>archive_offline</c>. Audited (<c>archive_converted</c>: sequences · photos · target · mode); the job tells each sequence
|
||||
/// it replaces the frames of as <c>sequenceChanged</c> <c>converted</c>, each new one as <c>created</c>.
|
||||
/// photos converted, their size and the estimate after, the photos left as they are with why in the mode's words (with
|
||||
/// <c>new-sequence</c>, that the new sequence lacks them; <c>convert_on_pc</c> for a format the server cannot read at
|
||||
/// full resolution), what blocks it; the totals; the space needed and free; the fingerprint. Otherwise the request
|
||||
/// carries that fingerprint and the server queues a <c>convert</c> job (202) that applies exactly that plan, leaving as it
|
||||
/// is (and reporting) a sequence that changed before it ran. 400 <c>bad_request</c> (details <c>sequenceIds</c>: neither
|
||||
/// or both of them, none, or more than <see cref="LibraryEndpoints.MaxBulkIds"/>; <c>target</c>, <c>quality</c>,
|
||||
/// <c>mode</c>, <c>fingerprint</c>); 404 <c>not_found</c> (<c>sequence</c>: not in the archive); 409 <c>plan_changed</c>
|
||||
/// (details: the fingerprint now), <c>nothing_to_convert</c>, <c>not_enough_space</c> (details: the bytes needed),
|
||||
/// <c>job_unavailable</c>; 503 <c>archive_offline</c>, <c>plan_dropped</c> (the plan was stopped: past its time, let go
|
||||
/// for newer ones, or the server stopping). Audited (<c>archive_converted</c>: sequences · photos · target · mode); the job
|
||||
/// tells each sequence it replaces the frames of as <c>sequenceChanged</c> <c>converted</c>, each new one as
|
||||
/// <c>created</c>. A dry run's plan is made apart from the request, a few at once (Reorganize's among them) and each for
|
||||
/// a limited time, and kept a few minutes, a refusal too, while what it reads from the index stays as it was
|
||||
/// (<see cref="PlanCache"/>, <see cref="ConvertPlanKey"/>: the sequences' rows, and of the photos of those planned their
|
||||
/// formats, sizes, hashes and the flags that decide): the same dry run again (also after the first was cut off), in
|
||||
/// either mode and at any quality of a lossless target, and its confirmation, take it instead of planning again; only
|
||||
/// the free space is measured again at every request. So <c>plan_changed</c> answers a change the index has. One made on
|
||||
/// disk alone since the plan, or a job that started or ended on its sequences (which a kept plan shows as it found
|
||||
/// them, busy or not), is found by the job, which plans again and leaves as it is a sequence that no longer fits.
|
||||
/// </summary>
|
||||
private static async Task<IResult> ConvertAsync(HttpContext context, Caller caller, ConvertRequest request, ChronosDatabase db, ServerArchive archive,
|
||||
[FromServices] ArchiveService? archives, JobEngineHost host, AuditLog audit)
|
||||
[FromServices] ArchiveService? archives, JobEngineHost host, AuditLog audit, PlanCache plans)
|
||||
{
|
||||
var ct = context.RequestAborted;
|
||||
if (request.All == (request.SequenceIds is not null) || request.SequenceIds is { Count: 0 or > LibraryEndpoints.MaxBulkIds })
|
||||
@@ -282,16 +291,33 @@ public static class ArchiveEndpoints
|
||||
if (!request.DryRun && string.IsNullOrWhiteSpace(request.Fingerprint)) return ApiErrors.BadRequest(context, "bad_request", "fingerprint");
|
||||
if (ArchiveService.GetAvailability(archive.Current) != ArchiveAvailability.Online)
|
||||
return ApiErrors.Result(context, StatusCodes.Status503ServiceUnavailable, "archive_offline");
|
||||
var options = new ConvertOptions(target, quality);
|
||||
var sequences = request.All ? null : request.SequenceIds;
|
||||
var engine = LibraryEndpoints.Engine(archives, db);
|
||||
// Stamped before planning: a change committed while the plan is made gives the next request another key.
|
||||
var key = ConvertPlanKey.Of(archive.Id, await db.Sequences.GetConvertStampAsync(archive.Id, sequences, ct), sequences, options);
|
||||
ConvertPlan plan;
|
||||
try
|
||||
{
|
||||
plan = await LibraryEndpoints.Engine(archives, db).PlanConvertAsync(archive.Id, request.All ? null : request.SequenceIds,
|
||||
new ConvertOptions(target, quality), mode, ct);
|
||||
// Made whatever becomes of this request, until the cache stops it: the same request again takes it, in either mode.
|
||||
var kept = await plans.GetOrPlanAsync(key, stop => PlanConvertAsync(engine, archive.Current, sequences, options, stop), ct);
|
||||
// What converts is the same in either mode and at any quality of a lossless target: the request's own are set on
|
||||
// it, and the free space is the volume's now.
|
||||
plan = kept with { Options = options, Mode = mode, Space = kept.Space with { Free = await FreeSpaceAsync(engine, archive.Id, options, ct) } };
|
||||
}
|
||||
catch (ArgumentException)
|
||||
{
|
||||
return ApiErrors.NotFound(context, "sequence");
|
||||
}
|
||||
catch (ArchiveOfflineException)
|
||||
{
|
||||
return ApiErrors.Result(context, StatusCodes.Status503ServiceUnavailable, "archive_offline");
|
||||
}
|
||||
catch (OperationCanceledException) when (!ct.IsCancellationRequested)
|
||||
{
|
||||
// Stopped: past its time (a read that hangs), let go for newer plans, or the server stopping.
|
||||
return ApiErrors.Result(context, StatusCodes.Status503ServiceUnavailable, "plan_dropped");
|
||||
}
|
||||
var fingerprint = plan.Fingerprint();
|
||||
if (request.DryRun) return Results.Json(Plan(context, plan, fingerprint), ProtocolJson.Default.ConvertPlanDto);
|
||||
if (!string.Equals(fingerprint, request.Fingerprint, StringComparison.Ordinal)) return ApiErrors.Conflict(context, "plan_changed", fingerprint);
|
||||
@@ -299,12 +325,33 @@ public static class ArchiveEndpoints
|
||||
if (!plan.Space.Enough) return ApiErrors.Conflict(context, "not_enough_space", plan.Space.Required.ToString(CultureInfo.InvariantCulture));
|
||||
if (!host.CanRun(JobKind.Convert)) return ApiErrors.Conflict(context, "job_unavailable", JobWire.Kind(JobKind.Convert));
|
||||
var job = await host.Engine.EnqueueAsync(ArchiveJobs.Convert(archive.Current, plan), caller.Id, ct);
|
||||
plans.Forget(key);
|
||||
await audit.WriteAsync(caller, context, "archive_converted", "job", job.Id.ToString(),
|
||||
$"{plan.Converted.Count()} · {plan.Frames} · {Wire(target)} · {Wire(mode)}");
|
||||
return Results.Json(Dto.Job(job), ProtocolJson.Default.JobDto, statusCode: StatusCodes.Status202Accepted);
|
||||
}
|
||||
|
||||
/// <summary>The conversion dry run as the API answers it, each reason in the request's language.</summary>
|
||||
/// <summary>
|
||||
/// The conversion plan <see cref="PlanCache"/> keeps for a <see cref="ConvertPlanKey"/>, made in one mode for both (the
|
||||
/// request's is set on it). One made while the archive's volume was away read nothing of it: it is not kept, and the
|
||||
/// request is answered <c>archive_offline</c>.
|
||||
/// </summary>
|
||||
private static async Task<ConvertPlan> PlanConvertAsync(ArchiveService engine, Archive archive, IReadOnlyList<Guid>? sequences, ConvertOptions options,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
var plan = await engine.PlanConvertAsync(archive.Id, sequences, options, ConvertMode.Replace, cancellationToken);
|
||||
if (plan.Sequences.Any(s => s.Blocked.Contains(ConvertBlock.Offline))) throw new ArchiveOfflineException(archive.RootPath, ArchiveAvailability.Offline);
|
||||
return plan;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The free space of the archive's volume now, as the engine measures it for a conversion's plan and its job
|
||||
/// (<see cref="ArchiveServiceOptions.FreeSpace"/>): the plan of no sequence reads no sidecar and no photo.
|
||||
/// </summary>
|
||||
private static async Task<long?> FreeSpaceAsync(ArchiveService engine, Guid archiveId, ConvertOptions options, CancellationToken cancellationToken) =>
|
||||
(await engine.PlanConvertAsync(archiveId, [], options, ConvertMode.Replace, cancellationToken)).Space.Free;
|
||||
|
||||
/// <summary>The conversion dry run as the API answers it, each reason in the request's language and in its mode's words.</summary>
|
||||
private static ConvertPlanDto Plan(HttpContext context, ConvertPlan plan, string fingerprint)
|
||||
{
|
||||
var culture = RequestCulture.Of(context);
|
||||
@@ -314,7 +361,8 @@ public static class ArchiveEndpoints
|
||||
: Loc.Text(space.Enough ? "Archive.Convert.Space" : "Archive.Convert.SpaceShort", [Size(space.Required, culture), Size(free, culture)], culture);
|
||||
return new ConvertPlanDto(Wire(plan.Options.Target), plan.Options.Quality, Wire(plan.Mode),
|
||||
[.. plan.Sequences.Select(s => new ConvertSequenceDto(s.SequenceId, s.Title, s.Folder, s.FramesVersion, s.Frames.Count, s.BytesBefore, s.BytesAfter,
|
||||
[.. s.Skipped.GroupBy(k => Wire(k.Reason)).Select(g => new ConvertSkippedDto(g.Key, g.Sum(k => k.Frames), SkipMessage(g.First().Reason, culture)))],
|
||||
[.. s.Skipped.GroupBy(k => Wire(k.Reason))
|
||||
.Select(g => new ConvertSkippedDto(g.Key, g.Sum(k => k.Frames), SkipMessage(g.First().Reason, plan.Mode, culture)))],
|
||||
[.. s.Blocked.Select(b => new ConvertBlockedDto(Wire(b), Loc.Get("Archive.Convert.Blocked." + b, culture) ?? Wire(b)))]))],
|
||||
plan.Converted.Count(),
|
||||
plan.Frames,
|
||||
@@ -370,10 +418,19 @@ public static class ArchiveEndpoints
|
||||
private static ConvertMode? ModeOf(string? text) =>
|
||||
Enum.GetValues<ConvertMode>().Select(m => (ConvertMode?)m).FirstOrDefault(m => string.Equals(Wire(m!.Value), text?.Trim(), StringComparison.OrdinalIgnoreCase));
|
||||
|
||||
/// <summary>Why photos are not converted, in <paramref name="culture"/>: on the server, a format it cannot read says to convert on a PC.</summary>
|
||||
private static string SkipMessage(ConvertSkip reason, CultureInfo culture) => reason is ConvertSkip.NoNativeDecoder or ConvertSkip.PreviewOnly
|
||||
? Loc.Get("Server.Convert.OnPc", culture) ?? Wire(reason)
|
||||
: Loc.Get("Archive.Convert.Skipped." + reason, culture) ?? Wire(reason);
|
||||
/// <summary>
|
||||
/// Why photos are not converted, in <paramref name="culture"/>: on the server, a format it cannot read says to convert on
|
||||
/// a PC. With <see cref="ConvertMode.NewSequence"/> it says that the new sequence lacks them (it holds the converted
|
||||
/// copies alone); otherwise that they stay as they are.
|
||||
/// </summary>
|
||||
private static string SkipMessage(ConvertSkip reason, ConvertMode mode, CultureInfo culture)
|
||||
{
|
||||
var onPc = reason is ConvertSkip.NoNativeDecoder or ConvertSkip.PreviewOnly;
|
||||
var key = mode == ConvertMode.NewSequence
|
||||
? onPc ? "Server.Convert.OnPcNew" : "Archive.Convert.NotCopied." + reason
|
||||
: onPc ? "Server.Convert.OnPc" : "Archive.Convert.Skipped." + reason;
|
||||
return Loc.Get(key, culture) ?? Wire(reason);
|
||||
}
|
||||
|
||||
/// <summary>A size as the dry run's space line shows it: GB from one gigabyte, MB below.</summary>
|
||||
private static string Size(long bytes, CultureInfo culture) => bytes >= 1_000_000_000
|
||||
|
||||
@@ -0,0 +1,267 @@
|
||||
using Chronos.Core.Archiving;
|
||||
using Chronos.Core.Conversion;
|
||||
using Chronos.Data.Repositories;
|
||||
|
||||
namespace Chronos.Server.Api;
|
||||
|
||||
/// <summary>
|
||||
/// The dry runs of Reorganize and Convert kept a short while (owner review 2, R7): the same dry run asked again, and the
|
||||
/// confirmation that follows it, take the plan made then instead of reading every sidecar and photo folder of the selection
|
||||
/// again (with <c>all</c>, of the whole archive), while what the plan reads from the index is as it was then (the stamp its
|
||||
/// key holds: <see cref="SequenceRepository.GetStampAsync"/>, <see cref="SequenceRepository.GetConvertStampAsync"/>) and for
|
||||
/// at most <see cref="Lifetime"/> once made. A refusal (a folder inside a sequence's, a sequence not in the archive) follows
|
||||
/// from the same reads, so it is kept as a plan is (<see cref="PlanKey.Refuses"/>); a failure is not. A plan is made apart
|
||||
/// from the request that asked for it, once for every request of the same key: a request cut off meanwhile (a proxy's
|
||||
/// timeout) leaves it to be made, and the same request again waits for it or takes it. At most <see cref="Concurrency"/>
|
||||
/// plans are made at once, of either kind; one still being made past <see cref="PlanTimeout"/>, let go for newer ones
|
||||
/// (<see cref="Capacity"/>), or when the server stops, is stopped, and whoever waits for it is told it was dropped
|
||||
/// (<see cref="OperationCanceledException"/>). A change made on disk alone, which the index does not have, is not seen by a
|
||||
/// kept plan, and neither is one of the jobs running: the job that applies a plan plans again and leaves as it is what no
|
||||
/// longer matches it, so nothing changes that the dry run did not show.
|
||||
/// </summary>
|
||||
public sealed class PlanCache : IDisposable
|
||||
{
|
||||
/// <summary>How long a plan is kept once made; then its timer lets it go.</summary>
|
||||
public static readonly TimeSpan Lifetime = TimeSpan.FromMinutes(5);
|
||||
|
||||
/// <summary>
|
||||
/// Plans kept or being made at most, of either kind (each holds a step for every sequence it plans). One more lets go
|
||||
/// the kept plan asked for first; with none kept, the plan being made asked for first, which stops.
|
||||
/// </summary>
|
||||
public const int Capacity = 8;
|
||||
|
||||
/// <summary>Plans made at once at most, of either kind; the others wait for their turn.</summary>
|
||||
public const int Concurrency = 2;
|
||||
|
||||
/// <summary>
|
||||
/// How long a plan may take, from when it was asked (its wait for a turn included): then it stops. A read that hangs
|
||||
/// still holds its turn until it returns.
|
||||
/// </summary>
|
||||
public static readonly TimeSpan PlanTimeout = TimeSpan.FromMinutes(15);
|
||||
|
||||
private readonly TimeProvider _time;
|
||||
private readonly Lock _gate = new();
|
||||
private readonly Dictionary<PlanKey, Entry> _plans = [];
|
||||
private readonly SemaphoreSlim _turns = new(Concurrency);
|
||||
private readonly CancellationTokenRegistration _stopping;
|
||||
private long _order;
|
||||
|
||||
/// <param name="lifetime">The host, whose stopping stops the plans being made.</param>
|
||||
public PlanCache(TimeProvider time, IHostApplicationLifetime? lifetime = null)
|
||||
{
|
||||
_time = time;
|
||||
if (lifetime is not null) _stopping = lifetime.ApplicationStopping.Register(Clear);
|
||||
}
|
||||
|
||||
/// <summary>Plans kept or being made now.</summary>
|
||||
internal int Count
|
||||
{
|
||||
get
|
||||
{
|
||||
lock (_gate) return _plans.Count;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The plan kept for <paramref name="key"/>, the one being made for it, or the one <paramref name="plan"/> starts making
|
||||
/// now, apart from the caller, then kept; <paramref name="plan"/> stops when its token is cancelled. A refusal
|
||||
/// (<see cref="PlanKey.Refuses"/>) is kept as a plan is; a failure is passed to every caller waiting for it and not kept,
|
||||
/// and so is a plan stopped (<see cref="OperationCanceledException"/>). <paramref name="cancellationToken"/> stops the
|
||||
/// wait only: the plan is still made, and kept.
|
||||
/// </summary>
|
||||
public async Task<TPlan> GetOrPlanAsync<TPlan>(PlanKey<TPlan> key, Func<CancellationToken, Task<TPlan>> plan, CancellationToken cancellationToken = default)
|
||||
where TPlan : class
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(key);
|
||||
ArgumentNullException.ThrowIfNull(plan);
|
||||
Entry? entry;
|
||||
bool start;
|
||||
lock (_gate)
|
||||
{
|
||||
// Never past its lifetime, whatever its timer.
|
||||
if (_plans.TryGetValue(key, out entry) && entry.Made is { } made && _time.GetUtcNow() - made >= Lifetime) Remove(key);
|
||||
start = !_plans.TryGetValue(key, out entry);
|
||||
if (start)
|
||||
{
|
||||
if (_plans.Count >= Capacity) Remove(_plans.MinBy(p => (p.Value.Made is null, p.Value.Order)).Key);
|
||||
var asked = new Entry(++_order);
|
||||
asked.Deadline = _time.CreateTimer(_ => TimeOut(key, asked), null, PlanTimeout, Timeout.InfiniteTimeSpan);
|
||||
_plans[key] = entry = asked;
|
||||
}
|
||||
}
|
||||
if (start) _ = MakeAsync(key, entry!, async stop => await plan(stop).ConfigureAwait(false));
|
||||
// The key's kind names the plan's: what was made for it is a TPlan.
|
||||
return (TPlan)await entry!.Plan.Task.WaitAsync(cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
/// <summary>Forgets the plan kept for <paramref name="key"/>: its job is queued, and what the job changes gives the index another stamp.</summary>
|
||||
public void Forget(PlanKey key)
|
||||
{
|
||||
lock (_gate) Remove(key);
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
_stopping.Dispose();
|
||||
Clear();
|
||||
}
|
||||
|
||||
/// <summary>Lets every plan go; those being made stop.</summary>
|
||||
private void Clear()
|
||||
{
|
||||
lock (_gate)
|
||||
{
|
||||
foreach (var key in _plans.Keys.ToList()) Remove(key);
|
||||
}
|
||||
}
|
||||
|
||||
private async Task MakeAsync(PlanKey key, Entry entry, Func<CancellationToken, Task<object>> plan)
|
||||
{
|
||||
var stop = entry.Stop.Token;
|
||||
object? made = null;
|
||||
Exception? refused = null;
|
||||
try
|
||||
{
|
||||
await _turns.WaitAsync(stop).ConfigureAwait(false);
|
||||
var making = Task.Run(() => plan(stop), CancellationToken.None);
|
||||
// The turn ends with the work, also when no one waits for it any more (a read that hangs).
|
||||
_ = making.ContinueWith(t =>
|
||||
{
|
||||
_ = t.Exception;
|
||||
_turns.Release();
|
||||
}, CancellationToken.None, TaskContinuationOptions.ExecuteSynchronously, TaskScheduler.Default);
|
||||
made = await making.WaitAsync(stop).ConfigureAwait(false);
|
||||
}
|
||||
catch (Exception ex) when (ex is not OperationCanceledException && key.Refuses(ex))
|
||||
{
|
||||
// A folder inside a sequence's, a sequence not in the archive: what the key holds decides it, as it does a plan.
|
||||
refused = ex;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
// A failure, or the plan stopped (past its time, let go, the server stopping): the next request plans again.
|
||||
lock (_gate)
|
||||
{
|
||||
if (_plans.GetValueOrDefault(key) == entry) Remove(key);
|
||||
}
|
||||
if (ex is OperationCanceledException canceled) entry.Plan.SetCanceled(canceled.CancellationToken);
|
||||
else entry.Plan.SetException(ex);
|
||||
// Observed here: a request cut off before it ended no longer waits for it.
|
||||
_ = entry.Plan.Task.Exception;
|
||||
return;
|
||||
}
|
||||
lock (_gate)
|
||||
{
|
||||
if (_plans.GetValueOrDefault(key) == entry)
|
||||
{
|
||||
entry.Made = _time.GetUtcNow();
|
||||
entry.Deadline?.Dispose();
|
||||
entry.Expiry = _time.CreateTimer(_ => Expire(key, entry), null, Lifetime, Timeout.InfiniteTimeSpan);
|
||||
}
|
||||
}
|
||||
if (refused is null)
|
||||
{
|
||||
entry.Plan.SetResult(made!);
|
||||
return;
|
||||
}
|
||||
entry.Plan.SetException(refused);
|
||||
_ = entry.Plan.Task.Exception;
|
||||
}
|
||||
|
||||
/// <summary>A plan still being made at <see cref="PlanTimeout"/> stops.</summary>
|
||||
private void TimeOut(PlanKey key, Entry entry)
|
||||
{
|
||||
lock (_gate)
|
||||
{
|
||||
// Made as its deadline fired: kept for its lifetime.
|
||||
if (_plans.GetValueOrDefault(key) == entry && entry.Made is null) Remove(key);
|
||||
}
|
||||
}
|
||||
|
||||
private void Expire(PlanKey key, Entry entry)
|
||||
{
|
||||
lock (_gate)
|
||||
{
|
||||
if (_plans.GetValueOrDefault(key) == entry) Remove(key);
|
||||
}
|
||||
}
|
||||
|
||||
private void Remove(PlanKey key)
|
||||
{
|
||||
if (!_plans.Remove(key, out var entry)) return;
|
||||
entry.Deadline?.Dispose();
|
||||
entry.Expiry?.Dispose();
|
||||
// Stopped outside this lock: the wait for it ends on the thread pool.
|
||||
if (entry.Made is null) _ = entry.Stop.CancelAsync();
|
||||
}
|
||||
|
||||
/// <summary>A plan being made (<see cref="Made"/> null) or kept, and the order it was asked in.</summary>
|
||||
private sealed class Entry(long order)
|
||||
{
|
||||
public long Order { get; } = order;
|
||||
|
||||
public TaskCompletionSource<object> Plan { get; } = new(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
|
||||
/// <summary>Stops the plan while it is made; a source with no timer and no parent, so nothing to release.</summary>
|
||||
public CancellationTokenSource Stop { get; } = new();
|
||||
|
||||
public DateTimeOffset? Made { get; set; }
|
||||
|
||||
/// <summary>Stops the plan at <see cref="PlanTimeout"/> while it is made.</summary>
|
||||
public ITimer? Deadline { get; set; }
|
||||
|
||||
/// <summary>Lets the plan go at <see cref="Lifetime"/> once made.</summary>
|
||||
public ITimer? Expiry { get; set; }
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>What a kept plan is kept by (<see cref="PlanCache"/>): everything it is made from, by value, its kind included.</summary>
|
||||
public abstract record PlanKey
|
||||
{
|
||||
/// <summary>
|
||||
/// Whether <paramref name="exception"/>, thrown while the plan is made, refuses the request for what the key holds (kept
|
||||
/// as a plan is: the same reads refuse it again) rather than failing it (not kept).
|
||||
/// </summary>
|
||||
public abstract bool Refuses(Exception exception);
|
||||
|
||||
/// <summary>The sequences of a request as a key holds them: <c>*</c> for every sequence, else the ids in the order given.</summary>
|
||||
protected static string Selection(IReadOnlyList<Guid>? sequenceIds) =>
|
||||
sequenceIds is null ? "*" : string.Join(',', sequenceIds.Select(id => id.ToString("N")));
|
||||
}
|
||||
|
||||
/// <summary>The key of a plan of <typeparamref name="TPlan"/>: what <see cref="PlanCache.GetOrPlanAsync"/> answers for it.</summary>
|
||||
public abstract record PlanKey<TPlan> : PlanKey where TPlan : class;
|
||||
|
||||
/// <summary>
|
||||
/// What a Reorganize plan is kept by: the archive, the stamp of what it reads from the index
|
||||
/// (<see cref="SequenceRepository.GetStampAsync"/>), the sequences (all of them, or the ids in the order given, which decides
|
||||
/// the folder that gets " (2)") and the templates. A folder inside a sequence's (<see cref="NestedRenameException"/>) or a
|
||||
/// sequence not in the archive (<see cref="ArgumentException"/>) refuses it.
|
||||
/// </summary>
|
||||
public sealed record ReorganizePlanKey(Guid ArchiveId, string Stamp, string Sequences, RenameTemplates Templates) : PlanKey<RenamePlan>
|
||||
{
|
||||
/// <param name="sequenceIds">The sequences of the request; null for every sequence of the archive.</param>
|
||||
public static ReorganizePlanKey Of(Guid archiveId, string stamp, IReadOnlyList<Guid>? sequenceIds, RenameTemplates templates) =>
|
||||
new(archiveId, stamp, Selection(sequenceIds), templates);
|
||||
|
||||
public override bool Refuses(Exception exception) => exception is NestedRenameException || exception.GetType() == typeof(ArgumentException);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// What a conversion plan is kept by: the archive, the stamp of what it reads from the index
|
||||
/// (<see cref="SequenceRepository.GetConvertStampAsync"/>), the sequences (all of them, or the ids in the order given) and
|
||||
/// the target with the quality that counts (<see cref="ConvertOptions.Key"/>: a lossless target's ignores it). Not the
|
||||
/// mode: what converts is the same in either, so the plan made with one serves the other (the endpoint sets the request's).
|
||||
/// A sequence not in the archive (<see cref="ArgumentException"/>) refuses it.
|
||||
/// </summary>
|
||||
public sealed record ConvertPlanKey(Guid ArchiveId, string Stamp, string Sequences, string Options) : PlanKey<ConvertPlan>
|
||||
{
|
||||
/// <param name="sequenceIds">The sequences of the request; null for every sequence of the archive.</param>
|
||||
public static ConvertPlanKey Of(Guid archiveId, string stamp, IReadOnlyList<Guid>? sequenceIds, ConvertOptions options)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(options);
|
||||
return new(archiveId, stamp, Selection(sequenceIds), options.Key);
|
||||
}
|
||||
|
||||
public override bool Refuses(Exception exception) => exception.GetType() == typeof(ArgumentException);
|
||||
}
|
||||
@@ -1,223 +0,0 @@
|
||||
using Chronos.Core.Archiving;
|
||||
using Chronos.Data.Repositories;
|
||||
|
||||
namespace Chronos.Server.Api;
|
||||
|
||||
/// <summary>
|
||||
/// Reorganize's dry runs kept a short while (owner review 2, R7): the same dry run asked again, and the confirmation that
|
||||
/// follows it, take the plan made then instead of reading every sidecar and photo folder of the selection again (with
|
||||
/// <c>all</c>, of the whole archive), while what the plan reads from the index is as it was then
|
||||
/// (<see cref="SequenceRepository.GetStampAsync"/>) and for at most <see cref="Lifetime"/> once made. A refusal (a folder
|
||||
/// inside a sequence's, a sequence not in the archive) follows from the same reads, so it is kept as a plan is; a failure
|
||||
/// is not. A plan is made apart from the request that asked for it, once for every request of the same key: a request cut
|
||||
/// off meanwhile (a proxy's timeout) leaves it to be made, and the same request again waits for it or takes it. At most
|
||||
/// <see cref="Concurrency"/> plans are made at once; one still being made past <see cref="PlanTimeout"/>, let go for newer
|
||||
/// ones (<see cref="Capacity"/>), or when the server stops, is stopped, and whoever waits for it is told it was dropped
|
||||
/// (<see cref="OperationCanceledException"/>). A change made on disk alone, which the index does not have, is not seen by a
|
||||
/// kept plan: the <c>move</c> job plans again and refuses a plan that no longer matches the archive, so nothing changes
|
||||
/// that the dry run did not show.
|
||||
/// </summary>
|
||||
public sealed class ReorganizePlanCache : IDisposable
|
||||
{
|
||||
/// <summary>How long a plan is kept once made; then its timer lets it go.</summary>
|
||||
public static readonly TimeSpan Lifetime = TimeSpan.FromMinutes(5);
|
||||
|
||||
/// <summary>
|
||||
/// Plans kept or being made at most (each holds a step for every sequence it changes or carries, or whose photos keep
|
||||
/// their names). One more lets go the kept plan asked for first; with none kept, the plan being made asked for first,
|
||||
/// which stops.
|
||||
/// </summary>
|
||||
public const int Capacity = 8;
|
||||
|
||||
/// <summary>Plans made at once at most; the others wait for their turn.</summary>
|
||||
public const int Concurrency = 2;
|
||||
|
||||
/// <summary>
|
||||
/// How long a plan may take, from when it was asked (its wait for a turn included): then it stops. A read that hangs
|
||||
/// still holds its turn until it returns.
|
||||
/// </summary>
|
||||
public static readonly TimeSpan PlanTimeout = TimeSpan.FromMinutes(15);
|
||||
|
||||
private readonly TimeProvider _time;
|
||||
private readonly Lock _gate = new();
|
||||
private readonly Dictionary<Key, Entry> _plans = [];
|
||||
private readonly SemaphoreSlim _turns = new(Concurrency);
|
||||
private readonly CancellationTokenRegistration _stopping;
|
||||
private long _order;
|
||||
|
||||
/// <param name="lifetime">The host, whose stopping stops the plans being made.</param>
|
||||
public ReorganizePlanCache(TimeProvider time, IHostApplicationLifetime? lifetime = null)
|
||||
{
|
||||
_time = time;
|
||||
if (lifetime is not null) _stopping = lifetime.ApplicationStopping.Register(Clear);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// What a plan is kept by: the archive, the stamp of what it reads from the index, the sequences (all of them, or the ids
|
||||
/// in the order given, which decides the folder that gets " (2)") and the templates.
|
||||
/// </summary>
|
||||
public readonly record struct Key(Guid ArchiveId, string Stamp, string Sequences, RenameTemplates Templates)
|
||||
{
|
||||
/// <param name="sequenceIds">The sequences of the request; null for every sequence of the archive.</param>
|
||||
public static Key Of(Guid archiveId, string stamp, IReadOnlyList<Guid>? sequenceIds, RenameTemplates templates) =>
|
||||
new(archiveId, stamp, sequenceIds is null ? "*" : string.Join(',', sequenceIds.Select(id => id.ToString("N"))), templates);
|
||||
}
|
||||
|
||||
/// <summary>Plans kept or being made now.</summary>
|
||||
internal int Count
|
||||
{
|
||||
get
|
||||
{
|
||||
lock (_gate) return _plans.Count;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The plan kept for <paramref name="key"/>, the one being made for it, or the one <paramref name="plan"/> starts making
|
||||
/// now, apart from the caller, then kept; <paramref name="plan"/> stops when its token is cancelled. A refusal
|
||||
/// (<see cref="NestedRenameException"/>, <see cref="ArgumentException"/>) is kept as a plan is; a failure is passed to
|
||||
/// every caller waiting for it and not kept, and so is a plan stopped (<see cref="OperationCanceledException"/>).
|
||||
/// <paramref name="cancellationToken"/> stops the wait only: the plan is still made, and kept.
|
||||
/// </summary>
|
||||
public Task<RenamePlan> GetOrPlanAsync(Key key, Func<CancellationToken, Task<RenamePlan>> plan, CancellationToken cancellationToken = default)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(plan);
|
||||
Entry? entry;
|
||||
bool start;
|
||||
lock (_gate)
|
||||
{
|
||||
// Never past its lifetime, whatever its timer.
|
||||
if (_plans.TryGetValue(key, out entry) && entry.Made is { } made && _time.GetUtcNow() - made >= Lifetime) Remove(key);
|
||||
start = !_plans.TryGetValue(key, out entry);
|
||||
if (start)
|
||||
{
|
||||
if (_plans.Count >= Capacity) Remove(_plans.MinBy(p => (p.Value.Made is null, p.Value.Order)).Key);
|
||||
var asked = new Entry(++_order);
|
||||
asked.Deadline = _time.CreateTimer(_ => TimeOut(key, asked), null, PlanTimeout, Timeout.InfiniteTimeSpan);
|
||||
_plans[key] = entry = asked;
|
||||
}
|
||||
}
|
||||
if (start) _ = MakeAsync(key, entry!, plan);
|
||||
return entry!.Plan.Task.WaitAsync(cancellationToken);
|
||||
}
|
||||
|
||||
/// <summary>Forgets the plan kept for <paramref name="key"/>: its job is queued, and what the job changes gives the index another stamp.</summary>
|
||||
public void Forget(Key key)
|
||||
{
|
||||
lock (_gate) Remove(key);
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
_stopping.Dispose();
|
||||
Clear();
|
||||
}
|
||||
|
||||
/// <summary>Lets every plan go; those being made stop.</summary>
|
||||
private void Clear()
|
||||
{
|
||||
lock (_gate)
|
||||
{
|
||||
foreach (var key in _plans.Keys.ToList()) Remove(key);
|
||||
}
|
||||
}
|
||||
|
||||
private async Task MakeAsync(Key key, Entry entry, Func<CancellationToken, Task<RenamePlan>> plan)
|
||||
{
|
||||
var stop = entry.Stop.Token;
|
||||
RenamePlan? made = null;
|
||||
Exception? refused = null;
|
||||
try
|
||||
{
|
||||
await _turns.WaitAsync(stop).ConfigureAwait(false);
|
||||
var making = Task.Run(() => plan(stop), CancellationToken.None);
|
||||
// The turn ends with the work, also when no one waits for it any more (a read that hangs).
|
||||
_ = making.ContinueWith(t =>
|
||||
{
|
||||
_ = t.Exception;
|
||||
_turns.Release();
|
||||
}, CancellationToken.None, TaskContinuationOptions.ExecuteSynchronously, TaskScheduler.Default);
|
||||
made = await making.WaitAsync(stop).ConfigureAwait(false);
|
||||
}
|
||||
catch (Exception ex) when (ex is NestedRenameException || ex.GetType() == typeof(ArgumentException))
|
||||
{
|
||||
// A folder inside a sequence's, a sequence not in the archive: what the key holds decides it, as it does a plan.
|
||||
refused = ex;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
// A failure, or the plan stopped (past its time, let go, the server stopping): the next request plans again.
|
||||
lock (_gate)
|
||||
{
|
||||
if (_plans.GetValueOrDefault(key) == entry) Remove(key);
|
||||
}
|
||||
if (ex is OperationCanceledException canceled) entry.Plan.SetCanceled(canceled.CancellationToken);
|
||||
else entry.Plan.SetException(ex);
|
||||
// Observed here: a request cut off before it ended no longer waits for it.
|
||||
_ = entry.Plan.Task.Exception;
|
||||
return;
|
||||
}
|
||||
lock (_gate)
|
||||
{
|
||||
if (_plans.GetValueOrDefault(key) == entry)
|
||||
{
|
||||
entry.Made = _time.GetUtcNow();
|
||||
entry.Deadline?.Dispose();
|
||||
entry.Expiry = _time.CreateTimer(_ => Expire(key, entry), null, Lifetime, Timeout.InfiniteTimeSpan);
|
||||
}
|
||||
}
|
||||
if (refused is null)
|
||||
{
|
||||
entry.Plan.SetResult(made!);
|
||||
return;
|
||||
}
|
||||
entry.Plan.SetException(refused);
|
||||
_ = entry.Plan.Task.Exception;
|
||||
}
|
||||
|
||||
/// <summary>A plan still being made at <see cref="PlanTimeout"/> stops.</summary>
|
||||
private void TimeOut(Key key, Entry entry)
|
||||
{
|
||||
lock (_gate)
|
||||
{
|
||||
// Made as its deadline fired: kept for its lifetime.
|
||||
if (_plans.GetValueOrDefault(key) == entry && entry.Made is null) Remove(key);
|
||||
}
|
||||
}
|
||||
|
||||
private void Expire(Key key, Entry entry)
|
||||
{
|
||||
lock (_gate)
|
||||
{
|
||||
if (_plans.GetValueOrDefault(key) == entry) Remove(key);
|
||||
}
|
||||
}
|
||||
|
||||
private void Remove(Key key)
|
||||
{
|
||||
if (!_plans.Remove(key, out var entry)) return;
|
||||
entry.Deadline?.Dispose();
|
||||
entry.Expiry?.Dispose();
|
||||
// Stopped outside this lock: the wait for it ends on the thread pool.
|
||||
if (entry.Made is null) _ = entry.Stop.CancelAsync();
|
||||
}
|
||||
|
||||
/// <summary>A plan being made (<see cref="Made"/> null) or kept, and the order it was asked in.</summary>
|
||||
private sealed class Entry(long order)
|
||||
{
|
||||
public long Order { get; } = order;
|
||||
|
||||
public TaskCompletionSource<RenamePlan> Plan { get; } = new(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
|
||||
/// <summary>Stops the plan while it is made; a source with no timer and no parent, so nothing to release.</summary>
|
||||
public CancellationTokenSource Stop { get; } = new();
|
||||
|
||||
public DateTimeOffset? Made { get; set; }
|
||||
|
||||
/// <summary>Stops the plan at <see cref="PlanTimeout"/> while it is made.</summary>
|
||||
public ITimer? Deadline { get; set; }
|
||||
|
||||
/// <summary>Lets the plan go at <see cref="Lifetime"/> once made.</summary>
|
||||
public ITimer? Expiry { get; set; }
|
||||
}
|
||||
}
|
||||
@@ -142,7 +142,7 @@ public static class ServerHost
|
||||
services.AddSingleton<RenderStore>();
|
||||
services.AddSingleton<RenderUploads>();
|
||||
services.AddSingleton<SequenceJobs>();
|
||||
services.AddSingleton<ReorganizePlanCache>();
|
||||
services.AddSingleton<PlanCache>();
|
||||
services.AddSingleton<RenderNodeService>();
|
||||
services.AddSingleton<JobEngineHost>();
|
||||
services.AddHostedService(sp => sp.GetRequiredService<JobEngineHost>());
|
||||
|
||||
@@ -70,6 +70,18 @@ public sealed class ConvertDialogTests(TestContext context)
|
||||
Loc.Plural("Archive.Convert.SkippedPhotos", 3)), ConvertPlans.Summary(plan));
|
||||
Assert.Equal(Loc.Plural("Archive.Rename.Summary", 0, 1), ConvertPlans.Summary(plan with { Sequences = [comet] }), "nothing converts, nothing more is said");
|
||||
|
||||
// Into a new sequence, which holds the converted copies alone: the photos skipped are not copied into it.
|
||||
var copies = plan with { Mode = ConvertMode.NewSequence };
|
||||
Assert.SequenceEqual(
|
||||
[
|
||||
Loc.L("Archive.Convert.NotCopied.SameFormat") + Loc.Separator + Loc.Plural("Shell.Status.Photos", 2),
|
||||
Loc.L("Archive.Convert.NotCopied.Unhashed") + Loc.Separator + Loc.Plural("Shell.Status.Photos", 1),
|
||||
],
|
||||
ConvertPlans.Rows(copies)[0].Kept);
|
||||
Assert.SequenceEqual(busy.Kept, ConvertPlans.Rows(copies)[1].Kept, "what blocks a sequence reads the same");
|
||||
Assert.Equal(string.Join(Loc.Separator, Loc.Plural("Archive.Rename.Summary", 2, 4), Loc.Plural("Shell.Status.Photos", 4), ConvertPlans.Sizes(3_500_000, 6_700_000),
|
||||
Loc.Plural("Archive.Convert.NotCopiedPhotos", 3)), ConvertPlans.Summary(copies));
|
||||
|
||||
// The space line: enough, short, or not measurable.
|
||||
var required = Formatting.Bytes(plan.Space.Required);
|
||||
Assert.Equal(new ConvertSpaceLine(Loc.L("Archive.Convert.Space", required, Formatting.Bytes(40_000_000_000)), false), ConvertPlans.Space(plan.Space));
|
||||
@@ -145,11 +157,17 @@ public sealed class ConvertDialogTests(TestContext context)
|
||||
Assert.Equal(Loc.L("Tooltips.Archive.ConvertApplyNew"), dialog.ApplyToolTip);
|
||||
Assert.True(dialog.ApplyCommand.CanExecute(null));
|
||||
|
||||
// JPEG to JPEG converts nothing; a quality outside 1-100 blocks a JPEG only.
|
||||
// JPEG to JPEG converts nothing: into a new sequence those photos are not copied, which the rows and the summary
|
||||
// say, and Replace's words are that they stay as they are. A quality outside 1-100 blocks a JPEG only.
|
||||
dialog.IsJpeg = true;
|
||||
Assert.Equal(Loc.L("Archive.Blocked.Planning"), dialog.ApplyToolTip, "the dry run on screen is not the one of the format chosen");
|
||||
await dialog.PlanAsync();
|
||||
Assert.SequenceEqual([Loc.L("Archive.Convert.NotCopied.SameFormat") + Loc.Separator + Loc.Plural("Shell.Status.Photos", 2)], dialog.Rows[0].Kept);
|
||||
Assert.Equal(Loc.Plural("Archive.Convert.NotCopiedPhotos", 4), dialog.SummaryText.Split(Loc.Separator)[^1]);
|
||||
dialog.IsReplace = true;
|
||||
Assert.SequenceEqual([Loc.L("Archive.Convert.Skipped.SameFormat") + Loc.Separator + Loc.Plural("Shell.Status.Photos", 2)], dialog.Rows[0].Kept);
|
||||
Assert.Equal(Loc.Plural("Archive.Convert.SkippedPhotos", 4), dialog.SummaryText.Split(Loc.Separator)[^1]);
|
||||
dialog.IsNewSequence = true;
|
||||
Assert.Equal(Loc.L("Archive.Blocked.NothingToConvert"), dialog.ApplyToolTip);
|
||||
// A click on Apply takes the focus from the quality first, which commits it: the dry run on screen is of that
|
||||
// quality already, so no other one starts under the click.
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
using Chronos.Core.Imaging;
|
||||
using Chronos.Core.Model;
|
||||
using Chronos.Data.Repositories;
|
||||
using Chronos.Data.Tests.Infrastructure;
|
||||
@@ -6,8 +7,9 @@ using Chronos.Testing;
|
||||
namespace Chronos.Data.Tests.Repositories;
|
||||
|
||||
/// <summary>
|
||||
/// The stamp of what a Reorganize plans from in an archive's index (<see cref="SequenceRepository.GetStampAsync"/>) on both
|
||||
/// dialects: the server makes a kept Reorganize plan again once it is another one, and only then (owner review 2, R7).
|
||||
/// The stamps of what a Reorganize and a conversion plan from in an archive's index (<see cref="SequenceRepository.GetStampAsync"/>,
|
||||
/// <see cref="SequenceRepository.GetConvertStampAsync"/>) on both dialects: the server makes a kept plan again once its
|
||||
/// stamp is another one, and only then (owner review 2, R7).
|
||||
/// </summary>
|
||||
public abstract class SequenceStampTests(TestContext context) : RepositoryTest(context)
|
||||
{
|
||||
@@ -77,6 +79,53 @@ public abstract class SequenceStampTests(TestContext context) : RepositoryTest(c
|
||||
Assert.False(verified == all || verified == uploaded, "a photo hashed");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task TheConvertStampFollowsThePhotosAConversionPlansFrom()
|
||||
{
|
||||
var db = await OpenChronosAsync();
|
||||
var archive = TestData.Archive();
|
||||
await db.Archives.UpsertAsync(archive, Ct);
|
||||
var roof = TestData.Sequence(archive.Id, new DateTime(2026, 9, 28, 18, 0, 0), "Roof");
|
||||
var ridge = TestData.Sequence(archive.Id, new DateTime(2026, 9, 29, 7, 0, 0), "Ridge");
|
||||
foreach (var sequence in new[] { roof, ridge }) await db.Sequences.UpsertAsync(sequence, Ct);
|
||||
var photo = await PhotoAsync(db, roof, 0);
|
||||
var elsewhere = await PhotoAsync(db, ridge, 0);
|
||||
IReadOnlyList<Guid> planned = [roof.Id];
|
||||
var stamp = await db.Sequences.GetConvertStampAsync(archive.Id, planned, Ct);
|
||||
Assert.Equal(stamp, await db.Sequences.GetConvertStampAsync(archive.Id, planned, Ct));
|
||||
Assert.Equal(stamp, await db.Sequences.GetConvertStampAsync(archive.Id, [roof.Id, roof.Id], Ct));
|
||||
Assert.NotEqual(stamp, await db.Sequences.GetConvertStampAsync(archive.Id, null, Ct));
|
||||
|
||||
// What a conversion does not read: the photos of a sequence not planned, light statistics, tags, a verify time.
|
||||
Assert.Equal(1, await db.Frames.UpdateFlagsAsync([elsewhere.Id], FrameFlags.Missing, FrameFlags.None, Ct));
|
||||
Assert.Equal(1, await db.Frames.UpdateLightStatsAsync([new FrameLightStats(photo.Id, -2.5, 0.01, 0.02)], Ct));
|
||||
Assert.True(await db.Sequences.SetTagsAsync(roof.Id, ["sky"], Ct));
|
||||
Assert.True(await db.Sequences.RecordVerifiedAsync(roof.Id, SequenceStatus.Verified, SequenceStatus.None, TestData.Now, Ct));
|
||||
Assert.Equal(stamp, await db.Sequences.GetConvertStampAsync(archive.Id, planned, Ct));
|
||||
|
||||
// What it reads, without another frames version: each flag that keeps a photo as it is (a scan's, a verify's, a
|
||||
// decode's), the format a decode records, the hash a verify records. Cleared again, the stamp is the one before.
|
||||
var seen = new HashSet<string>(StringComparer.Ordinal) { stamp };
|
||||
foreach (var flag in new[] { FrameFlags.Missing, FrameFlags.Corrupt, FrameFlags.PreviewOnly, FrameFlags.Undecodable })
|
||||
{
|
||||
Assert.Equal(1, await db.Frames.UpdateFlagsAsync([photo.Id], flag, FrameFlags.None, Ct));
|
||||
Assert.True(seen.Add(await db.Sequences.GetConvertStampAsync(archive.Id, planned, Ct)), flag.ToString());
|
||||
Assert.Equal(1, await db.Frames.UpdateFlagsAsync([photo.Id], FrameFlags.None, flag, Ct));
|
||||
Assert.Equal(stamp, await db.Sequences.GetConvertStampAsync(archive.Id, planned, Ct));
|
||||
}
|
||||
Assert.Equal(1, await db.Frames.UpdateTiersAsync([new FrameTierFields(photo.Id, FrameFlags.None, 0, 0, ImageFileFormat.Dng)], Ct));
|
||||
Assert.True(seen.Add(await db.Sequences.GetConvertStampAsync(archive.Id, planned, Ct)), "format");
|
||||
var unhashed = await PhotoAsync(db, roof, 1, hashed: false);
|
||||
Assert.True(seen.Add(await db.Sequences.GetConvertStampAsync(archive.Id, planned, Ct)), "a photo added");
|
||||
await VerifyAsync(db, unhashed);
|
||||
Assert.True(seen.Add(await db.Sequences.GetConvertStampAsync(archive.Id, planned, Ct)), "hashed by a verify");
|
||||
|
||||
// With every sequence planned, a flag on any photo changes it.
|
||||
var all = await db.Sequences.GetConvertStampAsync(archive.Id, null, Ct);
|
||||
Assert.Equal(1, await db.Frames.UpdateFlagsAsync([elsewhere.Id], FrameFlags.None, FrameFlags.Missing, Ct));
|
||||
Assert.NotEqual(all, await db.Sequences.GetConvertStampAsync(archive.Id, null, Ct));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// A photo indexed into a sequence, as an upload does, or without its hash, as a scan does: the frame, then the row's
|
||||
/// counts and frames version.
|
||||
|
||||
@@ -1,10 +1,13 @@
|
||||
using System.Globalization;
|
||||
using System.Net;
|
||||
using System.Net.Http.Json;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Chronos.Core.Codecs.Jpeg;
|
||||
using Chronos.Core.Imaging;
|
||||
using Chronos.Core.Localization;
|
||||
using Chronos.Core.Model;
|
||||
using Chronos.Protocol;
|
||||
using Chronos.Server.Api;
|
||||
using Chronos.Server.Events;
|
||||
using Chronos.Server.Storage;
|
||||
using Chronos.Testing;
|
||||
@@ -12,8 +15,9 @@ using Chronos.Testing;
|
||||
namespace Chronos.Server.Tests.Api;
|
||||
|
||||
/// <summary>
|
||||
/// Mass conversion on the server (owner review 2, R7): <c>POST /api/v1/archive/convert</c>, its dry run, the <c>convert</c> job
|
||||
/// that applies the confirmed plan, what the request may not do, and the formats the server leaves to a PC.
|
||||
/// Mass conversion on the server (owner review 2, R7): <c>POST /api/v1/archive/convert</c>, its dry run and the plan kept from
|
||||
/// it in either mode (<see cref="PlanCache"/>) with the free space measured at every request, the <c>convert</c> job that
|
||||
/// applies the confirmed plan, what the request may not do, and the formats the server leaves to a PC.
|
||||
/// </summary>
|
||||
[Category("server")]
|
||||
public sealed class ConvertTests(TestContext context)
|
||||
@@ -174,12 +178,126 @@ public sealed class ConvertTests(TestContext context)
|
||||
await UntilIdleAsync(server);
|
||||
|
||||
admin.DefaultRequestHeaders.AcceptLanguage.ParseAdd("it");
|
||||
using var dry = await admin.PostAsJsonAsync(Url, new ConvertRequest([id], false, "linear-dng", null, "new-sequence", DryRun: true), ProtocolJson.Default.ConvertRequest);
|
||||
var plan = await dry.ReadAsync(ProtocolJson.Default.ConvertPlanDto);
|
||||
var italian = CultureInfo.GetCultureInfo("it");
|
||||
var plan = await DryRunAsync(admin, new ConvertRequest([id], false, "linear-dng", null, "replace", DryRun: true));
|
||||
Assert.Equal((1, 2), (plan.Photos, plan.Skipped));
|
||||
var skipped = plan.Steps.Single().Skipped.Single();
|
||||
Assert.Equal(("convert_on_pc", 2), (skipped.Reason, skipped.Photos));
|
||||
Assert.Equal(Chronos.Core.Localization.Loc.Get("Server.Convert.OnPc", System.Globalization.CultureInfo.GetCultureInfo("it")), skipped.Message);
|
||||
Assert.Equal(Loc.Get("Server.Convert.OnPc", italian), skipped.Message);
|
||||
// Into a new sequence, which holds the converted copies alone: the dry run says the copy lacks them.
|
||||
var copies = await DryRunAsync(admin, new ConvertRequest([id], false, "linear-dng", null, "new-sequence", DryRun: true));
|
||||
Assert.Equal((1, 2), (copies.Photos, copies.Skipped));
|
||||
var left = copies.Steps.Single().Skipped.Single();
|
||||
Assert.Equal(("convert_on_pc", 2, Loc.Get("Server.Convert.OnPcNew", italian)), (left.Reason, left.Photos, left.Message));
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task ADryRunIsKeptInEitherModeWhileTheIndexStaysAsItWas()
|
||||
{
|
||||
await using var server = await TestServer.StartAsync(context, services: ArchiveModule.Register);
|
||||
var (editor, _, _) = await server.UserAsync("alberto", "editor");
|
||||
var admin = await server.AdminAsync();
|
||||
admin.DefaultRequestHeaders.AcceptLanguage.ParseAdd("it");
|
||||
var italian = CultureInfo.GetCultureInfo("it");
|
||||
var plans = server.Services.GetRequiredService<PlanCache>();
|
||||
Guid id = default;
|
||||
for (var i = 0; i < 3; i++) id = (await Uploads.UploadAsync(editor, Photo(i), $"IMG_{i + 1:0000}.JPG", sessionKey: "roof")).SequenceId;
|
||||
await UntilIdleAsync(server);
|
||||
var request = new ConvertRequest([id], false, "tiff16", null, "replace", DryRun: true);
|
||||
var shown = await DryRunAsync(admin, request);
|
||||
Assert.Equal(("replace", 3, 0), (shown.Mode, shown.Photos, shown.Skipped));
|
||||
Assert.Equal(1, plans.Count);
|
||||
|
||||
// A photo gone on disk alone is not in the index: the same dry run again, in the other mode or at another quality of
|
||||
// a lossless target, is the plan made before, in the request's mode and quality.
|
||||
var first = (await server.Database.Frames.ListBySequenceAsync(id))[0];
|
||||
var file = Path.Combine(server.Options.ArchivePath, first.RelativePath);
|
||||
var bytes = await File.ReadAllBytesAsync(file);
|
||||
File.Delete(file);
|
||||
var copies = await DryRunAsync(admin, request with { Mode = "new-sequence" });
|
||||
Assert.Equal(("new-sequence", 3, 0), (copies.Mode, copies.Photos, copies.Skipped));
|
||||
Assert.NotEqual(shown.Fingerprint, copies.Fingerprint);
|
||||
var quality = await DryRunAsync(admin, request with { Quality = 50 });
|
||||
Assert.Equal((50, 3, shown.Fingerprint), (quality.Quality, quality.Photos, quality.Fingerprint));
|
||||
Assert.Equal(1, plans.Count);
|
||||
|
||||
// A change of what it reads plans again at once: the photo is found missing, and each mode says what becomes of it.
|
||||
await Uploads.UploadAsync(editor, Photo(3), "IMG_0004.JPG", sessionKey: "roof");
|
||||
await UntilIdleAsync(server);
|
||||
var replan = await DryRunAsync(admin, request);
|
||||
Assert.Equal((3, 1), (replan.Photos, replan.Skipped));
|
||||
var stays = replan.Steps.Single().Skipped.Single();
|
||||
Assert.Equal(("missing", 1, Loc.Get("Archive.Convert.Skipped.Missing", italian)), (stays.Reason, stays.Photos, stays.Message));
|
||||
var left = (await DryRunAsync(admin, request with { Mode = "new-sequence" })).Steps.Single().Skipped.Single();
|
||||
Assert.Equal(("missing", 1, Loc.Get("Archive.Convert.NotCopied.Missing", italian)), (left.Reason, left.Photos, left.Message));
|
||||
|
||||
// The photo back on disk alone: still the plan kept. Another upload plans again, and the confirmation takes that
|
||||
// plan, then lets it go once its job is queued.
|
||||
await File.WriteAllBytesAsync(file, bytes);
|
||||
Assert.Equal(replan.Fingerprint, (await DryRunAsync(admin, request)).Fingerprint);
|
||||
await Uploads.UploadAsync(editor, Photo(4), "IMG_0005.JPG", sessionKey: "roof");
|
||||
await UntilIdleAsync(server);
|
||||
var whole = await DryRunAsync(admin, request);
|
||||
Assert.Equal((5, 0), (whole.Photos, whole.Skipped));
|
||||
var kept = plans.Count;
|
||||
JobDto job;
|
||||
using (var applied = await admin.PostAsJsonAsync(Url, request with { DryRun = false, Fingerprint = whole.Fingerprint }, ProtocolJson.Default.ConvertRequest))
|
||||
{
|
||||
await applied.ExpectAsync(HttpStatusCode.Accepted);
|
||||
job = await applied.ReadAsync(ProtocolJson.Default.JobDto);
|
||||
}
|
||||
Assert.Equal(kept - 1, plans.Count);
|
||||
await UntilJobAsync(admin, job.Id, "done");
|
||||
Assert.True((await server.Database.Frames.ListBySequenceAsync(id)).All(f => f.Format == ImageFileFormat.Tiff), "converted");
|
||||
|
||||
// Converted: the next dry run is another plan, which says why nothing converts, in New sequence's words too.
|
||||
var done = await DryRunAsync(admin, request with { Mode = "new-sequence" });
|
||||
Assert.Equal((0, 5), (done.Photos, done.Skipped));
|
||||
Assert.Equal(Loc.Get("Archive.Convert.NotCopied.SameFormat", italian), done.Steps.Single().Skipped.Single().Message);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task TheFreeSpaceIsMeasuredAgainAtEveryRequest()
|
||||
{
|
||||
await using var server = await TestServer.StartAsync(context, services: s =>
|
||||
{
|
||||
ArchiveModule.Register(s);
|
||||
s.AddSingleton<FakeVolumeSpace>();
|
||||
s.AddSingleton<VolumeSpace>(sp => sp.GetRequiredService<FakeVolumeSpace>());
|
||||
});
|
||||
var volumes = server.Services.GetRequiredService<FakeVolumeSpace>();
|
||||
var (editor, _, _) = await server.UserAsync("alberto", "editor");
|
||||
var admin = await server.AdminAsync();
|
||||
Guid id = default;
|
||||
for (var i = 0; i < 2; i++) id = (await Uploads.UploadAsync(editor, Photo(i), $"IMG_{i + 1:0000}.JPG", sessionKey: "roof")).SequenceId;
|
||||
await UntilIdleAsync(server);
|
||||
var request = new ConvertRequest([id], false, "linear-dng", null, "replace", DryRun: true);
|
||||
const long Full = 1_000_000, Free = 100_000_000_000;
|
||||
|
||||
volumes.Set(server.Options.ArchivePath, new VolumeFigures(Full, 4 * Free));
|
||||
var tight = await DryRunAsync(admin, request);
|
||||
Assert.Equal((false, (long?)Full), (tight.Space.Enough, tight.Space.Free));
|
||||
using (var refused = await admin.PostAsJsonAsync(Url, request with { DryRun = false, Fingerprint = tight.Fingerprint }, ProtocolJson.Default.ConvertRequest))
|
||||
Assert.Equal(tight.Space.Required.ToString(CultureInfo.InvariantCulture), (await refused.ExpectErrorAsync(HttpStatusCode.Conflict, "not_enough_space")).Details);
|
||||
|
||||
// Space freed since: the plan kept, with the free space now; and the confirmation measures it again too.
|
||||
volumes.Set(server.Options.ArchivePath, new VolumeFigures(Free, 4 * Free));
|
||||
var freed = await DryRunAsync(admin, request);
|
||||
Assert.Equal((true, (long?)Free, tight.Fingerprint, tight.Space.Required), (freed.Space.Enough, freed.Space.Free, freed.Fingerprint, freed.Space.Required));
|
||||
Assert.Equal(1, server.Services.GetRequiredService<PlanCache>().Count);
|
||||
volumes.Set(server.Options.ArchivePath, new VolumeFigures(Full, 4 * Free));
|
||||
using (var filled = await admin.PostAsJsonAsync(Url, request with { DryRun = false, Fingerprint = freed.Fingerprint }, ProtocolJson.Default.ConvertRequest))
|
||||
await filled.ExpectErrorAsync(HttpStatusCode.Conflict, "not_enough_space");
|
||||
volumes.Set(server.Options.ArchivePath, new VolumeFigures(Free, 4 * Free));
|
||||
using (var applied = await admin.PostAsJsonAsync(Url, request with { DryRun = false, Fingerprint = freed.Fingerprint }, ProtocolJson.Default.ConvertRequest))
|
||||
await UntilJobAsync(admin, (await applied.ReadAsync(ProtocolJson.Default.JobDto)).Id, "done");
|
||||
Assert.True((await server.Database.Frames.ListBySequenceAsync(id)).All(f => f.Format == ImageFileFormat.Dng), "converted");
|
||||
}
|
||||
|
||||
private static async Task<ConvertPlanDto> DryRunAsync(HttpClient client, ConvertRequest request)
|
||||
{
|
||||
using var response = await client.PostAsJsonAsync(Url, request with { DryRun = true }, ProtocolJson.Default.ConvertRequest);
|
||||
return await response.ReadAsync(ProtocolJson.Default.ConvertPlanDto);
|
||||
}
|
||||
|
||||
/// <summary>Every reason a photo stays as it is has its wire name and its sentence in the catalog (en and it).</summary>
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
using System.Collections.Concurrent;
|
||||
using Chronos.Core.Archiving;
|
||||
using Chronos.Core.Conversion;
|
||||
using Chronos.Server.Api;
|
||||
using Chronos.Testing;
|
||||
|
||||
namespace Chronos.Server.Tests.Api;
|
||||
|
||||
/// <summary>
|
||||
/// The dry runs the server keeps (<see cref="PlanCache"/>, owner review 2, R7) as Convert's endpoint uses them: what a
|
||||
/// conversion plan is kept by (<see cref="ConvertPlanKey"/>), what refuses it, and the turns and room it shares with
|
||||
/// Reorganize's plans. The rest of what the cache does is in <see cref="ReorganizeTests"/>.
|
||||
/// </summary>
|
||||
[Category("server")]
|
||||
public sealed class PlanCacheTests
|
||||
{
|
||||
private static readonly TimeSpan Wait = TimeSpan.FromSeconds(TestTiming.Budget(30));
|
||||
|
||||
[Test]
|
||||
public void AConversionPlanIsKeptByWhatConvertsNotByTheMode()
|
||||
{
|
||||
var archive = Guid.NewGuid();
|
||||
var (x, y) = (Guid.NewGuid(), Guid.NewGuid());
|
||||
var tiff = ConvertPlanKey.Of(archive, "stamp", [x, y], new ConvertOptions(ConvertTarget.Tiff16, 80));
|
||||
// The mode is not part of it, and neither is the quality of a lossless target; a JPEG's is.
|
||||
Assert.Equal(tiff, ConvertPlanKey.Of(archive, "stamp", [x, y], new ConvertOptions(ConvertTarget.Tiff16)));
|
||||
Assert.Equal(ConvertPlanKey.Of(archive, "stamp", [x], new ConvertOptions(ConvertTarget.Jpeg, 80)),
|
||||
ConvertPlanKey.Of(archive, "stamp", [x], new ConvertOptions(ConvertTarget.Jpeg, 80)));
|
||||
Assert.NotEqual(ConvertPlanKey.Of(archive, "stamp", [x], new ConvertOptions(ConvertTarget.Jpeg, 80)),
|
||||
ConvertPlanKey.Of(archive, "stamp", [x], new ConvertOptions(ConvertTarget.Jpeg, 95)));
|
||||
// Another target, order of the sequences, every sequence, stamp or archive is another plan.
|
||||
Assert.NotEqual(tiff, ConvertPlanKey.Of(archive, "stamp", [x, y], new ConvertOptions(ConvertTarget.LinearDng)));
|
||||
Assert.NotEqual(tiff, ConvertPlanKey.Of(archive, "stamp", [y, x], new ConvertOptions(ConvertTarget.Tiff16)));
|
||||
Assert.NotEqual(tiff, ConvertPlanKey.Of(archive, "stamp", null, new ConvertOptions(ConvertTarget.Tiff16)));
|
||||
Assert.NotEqual(tiff, ConvertPlanKey.Of(archive, "other", [x, y], new ConvertOptions(ConvertTarget.Tiff16)));
|
||||
Assert.NotEqual(tiff, ConvertPlanKey.Of(Guid.NewGuid(), "stamp", [x, y], new ConvertOptions(ConvertTarget.Tiff16)));
|
||||
// A Reorganize of the same sequences is never the same plan.
|
||||
Assert.NotEqual<PlanKey>(ConvertPlanKey.Of(archive, "stamp", null, new ConvertOptions(ConvertTarget.Tiff16)),
|
||||
ReorganizePlanKey.Of(archive, "stamp", null, new RenameTemplates(Folder: "moved/{date}")));
|
||||
|
||||
// A sequence not in the archive refuses it; a failure (a sidecar unreadable, a bug) does not.
|
||||
Assert.True(tiff.Refuses(new ArgumentException("Not in the archive.")));
|
||||
foreach (var failure in new Exception[] { new ArgumentNullException("plan"), new IOException("The disk went away."), new NestedRenameException("a/b", "a") })
|
||||
Assert.False(tiff.Refuses(failure), failure.GetType().Name);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task ConversionAndReorganizePlansShareTheTurnsAndTheRoom()
|
||||
{
|
||||
var clock = new SteppedTimeProvider();
|
||||
using var plans = new PlanCache(clock);
|
||||
var archive = Guid.NewGuid();
|
||||
var templates = new RenameTemplates(Folder: "moved/{date}");
|
||||
var release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
var started = new ConcurrentDictionary<string, bool>();
|
||||
Func<CancellationToken, Task<RenamePlan>> Rename(int i) => async stop =>
|
||||
{
|
||||
started["reorganize" + i] = true;
|
||||
await release.Task.WaitAsync(stop);
|
||||
return new RenamePlan(Guid.NewGuid(), archive, []);
|
||||
};
|
||||
var made = 0;
|
||||
Task<ConvertPlan> Convert(CancellationToken stop)
|
||||
{
|
||||
started["convert"] = true;
|
||||
Interlocked.Increment(ref made);
|
||||
return Task.FromResult(new ConvertPlan(archive, new ConvertOptions(ConvertTarget.LinearDng), ConvertMode.Replace, [], new ConvertSpace(0, null)));
|
||||
}
|
||||
|
||||
// Every turn taken by a Reorganize: a conversion waits for one.
|
||||
var renames = Enumerable.Range(0, PlanCache.Concurrency)
|
||||
.Select(i => plans.GetOrPlanAsync(ReorganizePlanKey.Of(archive, "stamp", [new Guid(i, 0, 0, new byte[8])], templates), Rename(i))).ToList();
|
||||
await EventuallyAsync(() => started.Count == PlanCache.Concurrency);
|
||||
var key = ConvertPlanKey.Of(archive, "stamp", null, new ConvertOptions(ConvertTarget.LinearDng));
|
||||
var converting = plans.GetOrPlanAsync(key, Convert);
|
||||
await Task.Delay(200);
|
||||
Assert.False(started.ContainsKey("convert"), "no turn while Reorganize holds them");
|
||||
release.SetResult();
|
||||
var plan = await converting;
|
||||
foreach (var rename in renames) await rename;
|
||||
Assert.True(ReferenceEquals(plan, await plans.GetOrPlanAsync(key, Convert)), "kept");
|
||||
Assert.Equal(1, made);
|
||||
Assert.Equal(PlanCache.Concurrency + 1, plans.Count);
|
||||
|
||||
// A refusal is kept as a plan is, a failure is not; the room is shared: the oldest of either kind goes first.
|
||||
var unknown = ConvertPlanKey.Of(archive, "stamp", [Guid.NewGuid()], new ConvertOptions(ConvertTarget.LinearDng));
|
||||
await Assert.ThrowsAsync<ArgumentException>(() => plans.GetOrPlanAsync(unknown, _ => throw new ArgumentException("Not in the archive.")));
|
||||
await Assert.ThrowsAsync<ArgumentException>(() => plans.GetOrPlanAsync(unknown, Convert));
|
||||
var offline = ConvertPlanKey.Of(archive, "stamp", [Guid.NewGuid()], new ConvertOptions(ConvertTarget.Tiff16));
|
||||
await Assert.ThrowsAsync<ArchiveOfflineException>(() => plans.GetOrPlanAsync(offline, _ => throw new ArchiveOfflineException("/archive", ArchiveAvailability.Offline)));
|
||||
await plans.GetOrPlanAsync(offline, Convert);
|
||||
Assert.Equal(2, made);
|
||||
for (var i = 0; i < PlanCache.Capacity; i++)
|
||||
{
|
||||
clock.Advance(TimeSpan.FromSeconds(1));
|
||||
await plans.GetOrPlanAsync(ConvertPlanKey.Of(archive, "stamp", [new Guid(i, 0, 0, new byte[8])], new ConvertOptions(ConvertTarget.Jpeg)), Convert);
|
||||
}
|
||||
Assert.Equal(PlanCache.Capacity, plans.Count);
|
||||
await plans.GetOrPlanAsync(key, Convert);
|
||||
Assert.Equal(3 + PlanCache.Capacity, made);
|
||||
}
|
||||
|
||||
/// <summary>Waits until <paramref name="condition"/> holds.</summary>
|
||||
private static async Task EventuallyAsync(Func<bool> condition, [System.Runtime.CompilerServices.CallerArgumentExpression(nameof(condition))] string? expression = null)
|
||||
{
|
||||
var until = DateTime.UtcNow + Wait;
|
||||
while (!condition())
|
||||
{
|
||||
Assert.True(DateTime.UtcNow < until, expression);
|
||||
await Task.Delay(10);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -16,7 +16,7 @@ namespace Chronos.Server.Tests.Api;
|
||||
|
||||
/// <summary>
|
||||
/// Reorganize on the server (owner review 2, R7): <c>POST /api/v1/archive/reorganize</c>, its dry run and the plan kept from it
|
||||
/// for the confirmation (<see cref="ReorganizePlanCache"/>), the <c>move</c> job that applies the confirmed plan (paused and
|
||||
/// for the confirmation (<see cref="PlanCache"/>), the <c>move</c> job that applies the confirmed plan (paused and
|
||||
/// resumed too), and what the request may not do, against the real archive engine.
|
||||
/// </summary>
|
||||
[Category("server")]
|
||||
@@ -562,7 +562,7 @@ public sealed class ReorganizeTests(TestContext context)
|
||||
Directory.CreateDirectory(target);
|
||||
Assert.Equal(free.Fingerprint, (await DryRunAsync(admin, request)).Fingerprint);
|
||||
await UntilIdleAsync(server);
|
||||
clock.Advance(ReorganizePlanCache.Lifetime);
|
||||
clock.Advance(PlanCache.Lifetime);
|
||||
var later = await DryRunAsync(admin, request);
|
||||
Assert.Equal("moved/2026-09-28 (2)", later.Steps.Single().ToFolder);
|
||||
using (var changed = await admin.PostAsJsonAsync(Url, request with { DryRun = false, Fingerprint = free.Fingerprint }, ProtocolJson.Default.ReorganizeRequest))
|
||||
@@ -604,7 +604,7 @@ public sealed class ReorganizeTests(TestContext context)
|
||||
public async Task KeptPlansAreTheNewestFewWithinTheirLifetime()
|
||||
{
|
||||
var clock = new ManualTimeProvider();
|
||||
using var plans = new ReorganizePlanCache(clock);
|
||||
using var plans = new PlanCache(clock);
|
||||
var archive = Guid.NewGuid();
|
||||
var templates = new RenameTemplates(Folder: "moved/{date}");
|
||||
var made = 0;
|
||||
@@ -613,38 +613,38 @@ public sealed class ReorganizeTests(TestContext context)
|
||||
made++;
|
||||
return Task.FromResult(new RenamePlan(Guid.NewGuid(), archive, []));
|
||||
}
|
||||
ReorganizePlanCache.Key Key(int i) => ReorganizePlanCache.Key.Of(archive, "stamp", [new Guid(i, 0, 0, new byte[8])], templates);
|
||||
ReorganizePlanKey Key(int i) => ReorganizePlanKey.Of(archive, "stamp", [new Guid(i, 0, 0, new byte[8])], templates);
|
||||
|
||||
var kept = await plans.GetOrPlanAsync(Key(0), Plan);
|
||||
Assert.True(ReferenceEquals(kept, await plans.GetOrPlanAsync(Key(0), Plan)), "kept");
|
||||
Assert.Equal(1, made);
|
||||
// Another order of the sequences, every sequence, another stamp or other templates is another plan.
|
||||
var (x, y) = (Guid.NewGuid(), Guid.NewGuid());
|
||||
Assert.NotEqual(ReorganizePlanCache.Key.Of(archive, "stamp", [x, y], templates), ReorganizePlanCache.Key.Of(archive, "stamp", [y, x], templates));
|
||||
Assert.Equal(ReorganizePlanCache.Key.Of(archive, "stamp", [x, y], templates), ReorganizePlanCache.Key.Of(archive, "stamp", [x, y], templates with { }));
|
||||
Assert.NotEqual(ReorganizePlanCache.Key.Of(archive, "stamp", null, templates), ReorganizePlanCache.Key.Of(archive, "stamp", [x], templates));
|
||||
Assert.NotEqual(Key(0), ReorganizePlanCache.Key.Of(archive, "other", [new Guid(0, 0, 0, new byte[8])], templates));
|
||||
Assert.NotEqual(Key(0), ReorganizePlanCache.Key.Of(archive, "stamp", [new Guid(0, 0, 0, new byte[8])], templates with { File = "{n}" }));
|
||||
Assert.NotEqual(ReorganizePlanKey.Of(archive, "stamp", [x, y], templates), ReorganizePlanKey.Of(archive, "stamp", [y, x], templates));
|
||||
Assert.Equal(ReorganizePlanKey.Of(archive, "stamp", [x, y], templates), ReorganizePlanKey.Of(archive, "stamp", [x, y], templates with { }));
|
||||
Assert.NotEqual(ReorganizePlanKey.Of(archive, "stamp", null, templates), ReorganizePlanKey.Of(archive, "stamp", [x], templates));
|
||||
Assert.NotEqual(Key(0), ReorganizePlanKey.Of(archive, "other", [new Guid(0, 0, 0, new byte[8])], templates));
|
||||
Assert.NotEqual(Key(0), ReorganizePlanKey.Of(archive, "stamp", [new Guid(0, 0, 0, new byte[8])], templates with { File = "{n}" }));
|
||||
|
||||
// At most Capacity plans: the oldest goes first.
|
||||
for (var i = 1; i <= ReorganizePlanCache.Capacity; i++)
|
||||
for (var i = 1; i <= PlanCache.Capacity; i++)
|
||||
{
|
||||
clock.Advance(TimeSpan.FromSeconds(1));
|
||||
await plans.GetOrPlanAsync(Key(i), Plan);
|
||||
}
|
||||
Assert.Equal(1 + ReorganizePlanCache.Capacity, made);
|
||||
await plans.GetOrPlanAsync(Key(ReorganizePlanCache.Capacity), Plan);
|
||||
Assert.Equal(1 + ReorganizePlanCache.Capacity, made);
|
||||
Assert.Equal(1 + PlanCache.Capacity, made);
|
||||
await plans.GetOrPlanAsync(Key(PlanCache.Capacity), Plan);
|
||||
Assert.Equal(1 + PlanCache.Capacity, made);
|
||||
await plans.GetOrPlanAsync(Key(0), Plan);
|
||||
Assert.Equal(2 + ReorganizePlanCache.Capacity, made);
|
||||
Assert.Equal(2 + PlanCache.Capacity, made);
|
||||
|
||||
// A plan is kept for its lifetime, from when it was made, even before its timer lets it go.
|
||||
clock.Advance(ReorganizePlanCache.Lifetime - TimeSpan.FromSeconds(1));
|
||||
clock.Advance(PlanCache.Lifetime - TimeSpan.FromSeconds(1));
|
||||
await plans.GetOrPlanAsync(Key(0), Plan);
|
||||
Assert.Equal(2 + ReorganizePlanCache.Capacity, made);
|
||||
Assert.Equal(2 + PlanCache.Capacity, made);
|
||||
clock.Advance(TimeSpan.FromSeconds(1));
|
||||
await plans.GetOrPlanAsync(Key(0), Plan);
|
||||
Assert.Equal(3 + ReorganizePlanCache.Capacity, made);
|
||||
Assert.Equal(3 + PlanCache.Capacity, made);
|
||||
|
||||
// A refusal follows from the same reads, and is kept as a plan is: a folder inside a sequence's, a sequence not in the
|
||||
// archive. A failure is not kept, and neither is a plan or refusal forgotten.
|
||||
@@ -652,22 +652,22 @@ public sealed class ReorganizeTests(TestContext context)
|
||||
await Assert.ThrowsAsync<NestedRenameException>(() => plans.GetOrPlanAsync(Key(99), Plan));
|
||||
await Assert.ThrowsAsync<ArgumentException>(() => plans.GetOrPlanAsync(Key(98), _ => throw new ArgumentException("Not in the archive.")));
|
||||
await Assert.ThrowsAsync<ArgumentException>(() => plans.GetOrPlanAsync(Key(98), Plan));
|
||||
Assert.Equal(3 + ReorganizePlanCache.Capacity, made);
|
||||
Assert.Equal(3 + PlanCache.Capacity, made);
|
||||
await Assert.ThrowsAsync<IOException>(() => plans.GetOrPlanAsync(Key(97), _ => throw new IOException("The sidecar cannot be read.")));
|
||||
await plans.GetOrPlanAsync(Key(97), Plan);
|
||||
Assert.Equal(4 + ReorganizePlanCache.Capacity, made);
|
||||
Assert.Equal(4 + PlanCache.Capacity, made);
|
||||
plans.Forget(Key(99));
|
||||
await plans.GetOrPlanAsync(Key(99), Plan);
|
||||
Assert.Equal(5 + ReorganizePlanCache.Capacity, made);
|
||||
Assert.Equal(5 + PlanCache.Capacity, made);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task APlanIsMadeOnceForEveryRequestAndOutlivesOneCutOff()
|
||||
{
|
||||
var clock = new SteppedTimeProvider();
|
||||
using var plans = new ReorganizePlanCache(clock);
|
||||
using var plans = new PlanCache(clock);
|
||||
var templates = new RenameTemplates(Folder: "moved/{date}");
|
||||
var key = ReorganizePlanCache.Key.Of(Guid.NewGuid(), "stamp", null, templates);
|
||||
var key = ReorganizePlanKey.Of(Guid.NewGuid(), "stamp", null, templates);
|
||||
var release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
var made = 0;
|
||||
async Task<RenamePlan> SlowPlan(CancellationToken stop)
|
||||
@@ -687,22 +687,22 @@ public sealed class ReorganizeTests(TestContext context)
|
||||
var again = plans.GetOrPlanAsync(key, SlowPlan);
|
||||
var meanwhile = plans.GetOrPlanAsync(key, SlowPlan);
|
||||
// Being made, it is not the one let go for other plans.
|
||||
for (var i = 0; i < ReorganizePlanCache.Capacity; i++)
|
||||
await plans.GetOrPlanAsync(ReorganizePlanCache.Key.Of(key.ArchiveId, "stamp", [new Guid(i, 0, 0, new byte[8])], templates), PlanAtOnce);
|
||||
for (var i = 0; i < PlanCache.Capacity; i++)
|
||||
await plans.GetOrPlanAsync(ReorganizePlanKey.Of(key.ArchiveId, "stamp", [new Guid(i, 0, 0, new byte[8])], templates), PlanAtOnce);
|
||||
release.SetResult();
|
||||
var plan = await again;
|
||||
Assert.True(ReferenceEquals(plan, await meanwhile), "one plan for both");
|
||||
Assert.True(ReferenceEquals(plan, await plans.GetOrPlanAsync(key, SlowPlan)), "kept");
|
||||
Assert.Equal(1 + ReorganizePlanCache.Capacity, made);
|
||||
Assert.Equal(ReorganizePlanCache.Capacity, plans.Count);
|
||||
Assert.Equal(1 + PlanCache.Capacity, made);
|
||||
Assert.Equal(PlanCache.Capacity, plans.Count);
|
||||
|
||||
// Its timer lets it go at the end of its lifetime, with no request asking.
|
||||
clock.Advance(ReorganizePlanCache.Lifetime - TimeSpan.FromSeconds(1));
|
||||
Assert.Equal(ReorganizePlanCache.Capacity, plans.Count);
|
||||
clock.Advance(PlanCache.Lifetime - TimeSpan.FromSeconds(1));
|
||||
Assert.Equal(PlanCache.Capacity, plans.Count);
|
||||
clock.Advance(TimeSpan.FromSeconds(1));
|
||||
Assert.Equal(0, plans.Count);
|
||||
await plans.GetOrPlanAsync(key, SlowPlan);
|
||||
Assert.Equal(2 + ReorganizePlanCache.Capacity, made);
|
||||
Assert.Equal(2 + PlanCache.Capacity, made);
|
||||
|
||||
Task<RenamePlan> PlanAtOnce(CancellationToken stop)
|
||||
{
|
||||
@@ -715,10 +715,10 @@ public sealed class ReorganizeTests(TestContext context)
|
||||
public async Task PlansAreMadeAFewAtOnceAndStopWhenLetGoOrPastTheirTime()
|
||||
{
|
||||
var clock = new SteppedTimeProvider();
|
||||
using var plans = new ReorganizePlanCache(clock);
|
||||
using var plans = new PlanCache(clock);
|
||||
var archive = Guid.NewGuid();
|
||||
var templates = new RenameTemplates(Folder: "moved/{date}");
|
||||
ReorganizePlanCache.Key Key(int i) => ReorganizePlanCache.Key.Of(archive, "stamp", [new Guid(i, 0, 0, new byte[8])], templates);
|
||||
ReorganizePlanKey Key(int i) => ReorganizePlanKey.Of(archive, "stamp", [new Guid(i, 0, 0, new byte[8])], templates);
|
||||
var release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
var started = new ConcurrentDictionary<int, bool>();
|
||||
var stopped = new ConcurrentDictionary<int, bool>();
|
||||
@@ -739,16 +739,16 @@ public sealed class ReorganizeTests(TestContext context)
|
||||
};
|
||||
|
||||
// A few are made at once; the others wait for their turn.
|
||||
var asked = Enumerable.Range(0, ReorganizePlanCache.Capacity).Select(i => plans.GetOrPlanAsync(Key(i), Plan(i))).ToList();
|
||||
await EventuallyAsync(() => started.Count == ReorganizePlanCache.Concurrency);
|
||||
var asked = Enumerable.Range(0, PlanCache.Capacity).Select(i => plans.GetOrPlanAsync(Key(i), Plan(i))).ToList();
|
||||
await EventuallyAsync(() => started.Count == PlanCache.Concurrency);
|
||||
await Task.Delay(200);
|
||||
Assert.SequenceEqual(Enumerable.Range(0, ReorganizePlanCache.Concurrency), started.Keys.Order());
|
||||
Assert.SequenceEqual(Enumerable.Range(0, PlanCache.Concurrency), started.Keys.Order());
|
||||
|
||||
// One more lets go the plan asked first, being made with none kept: it stops, its waiter is told, the next one starts.
|
||||
var newest = plans.GetOrPlanAsync(Key(ReorganizePlanCache.Capacity), Plan(ReorganizePlanCache.Capacity));
|
||||
var newest = plans.GetOrPlanAsync(Key(PlanCache.Capacity), Plan(PlanCache.Capacity));
|
||||
await Assert.ThrowsAsync<OperationCanceledException>(() => asked[0]);
|
||||
await EventuallyAsync(() => stopped.ContainsKey(0) && started.ContainsKey(ReorganizePlanCache.Concurrency));
|
||||
Assert.Equal(ReorganizePlanCache.Capacity, plans.Count);
|
||||
await EventuallyAsync(() => stopped.ContainsKey(0) && started.ContainsKey(PlanCache.Concurrency));
|
||||
Assert.Equal(PlanCache.Capacity, plans.Count);
|
||||
release.SetResult();
|
||||
foreach (var plan in asked.Skip(1).Append(newest)) await plan;
|
||||
Assert.SequenceEqual([0], stopped.Keys);
|
||||
@@ -765,7 +765,7 @@ public sealed class ReorganizeTests(TestContext context)
|
||||
var stuck = plans.GetOrPlanAsync(Key(21), Hanging(21));
|
||||
await EventuallyAsync(() => started.ContainsKey(20) && started.ContainsKey(21));
|
||||
var waiting = plans.GetOrPlanAsync(Key(22), Plan(22));
|
||||
clock.Advance(ReorganizePlanCache.PlanTimeout - TimeSpan.FromSeconds(1));
|
||||
clock.Advance(PlanCache.PlanTimeout - TimeSpan.FromSeconds(1));
|
||||
Assert.False(hung.IsCompleted, "still in time");
|
||||
clock.Advance(TimeSpan.FromSeconds(1));
|
||||
foreach (var plan in new[] { hung, stuck, waiting }) await Assert.ThrowsAsync<OperationCanceledException>(() => plan);
|
||||
|
||||
@@ -66,6 +66,19 @@ public sealed partial class AdminConvertTests(TestContext context)
|
||||
Assert.Contains("if (plan !== shownPlan && (plan?.key !== shownPlan.key || plan.fingerprint !== shownPlan.fingerprint)) {", dialog);
|
||||
Assert.Contains(@"showError(t(""Admin.Convert.PlanChanged""))", dialog);
|
||||
Assert.True(!dialog.Contains("plan !== shownPlan) return;", StringComparison.Ordinal), "a confirmed apply is dropped when the same plan is answered again");
|
||||
// Until a mode is chosen the preview is asked in Replace's words (the photos stay as they are); New sequence asks
|
||||
// again and says what the new sequences lack, in the rows, the totals and the confirmation.
|
||||
Assert.Contains(@"const PREVIEW_MODE = ""replace"";", dialog);
|
||||
Assert.Contains(@"t(copies ? ""Admin.Convert.TotalsNew"" : ""Admin.Convert.Totals""", dialog);
|
||||
Assert.Contains(@"plural(""Admin.Convert.ConfirmNewNotCopied"", notCopied)", dialog);
|
||||
// A dry run the server is still making is asked again (a proxy cut the request off, or the server let the plan go)
|
||||
// while the totals say how long it takes; a confirmation the server let go plans again, one that changed says so,
|
||||
// and one with no answer is not sent again (its job may be queued).
|
||||
Assert.Contains(@"cutOff(err, Date.now() - sent) || (err?.code === ""plan_dropped"" && dropped++ < DROPPED_RETRIES)", dialog);
|
||||
Assert.Contains(@"t(""Admin.Convert.PlanningFor"", formatDuration(seconds))", dialog);
|
||||
Assert.Contains(@"[""plan_changed"", ""plan_dropped"", ""not_enough_space"", ""nothing_to_convert""].includes(err?.code)", dialog);
|
||||
Assert.Contains(@"notice = err.code === ""plan_changed"" ? t(""Admin.Convert.PlanChanged"") : errorText(err);", dialog);
|
||||
Assert.Contains(@"showError(t(""Admin.Convert.NoAnswer""));", dialog);
|
||||
// The panel runs under a strict CSP: no inline handlers or styles.
|
||||
Assert.True(!Regex.IsMatch(dialog, @"\son[a-z]+="""), "inline handler");
|
||||
Assert.True(!dialog.Contains(" style=", StringComparison.Ordinal), "inline style");
|
||||
@@ -105,24 +118,24 @@ public sealed partial class AdminConvertTests(TestContext context)
|
||||
var browser = await Browser.SignInAsync(server);
|
||||
var italian = CultureInfo.GetCultureInfo("it");
|
||||
|
||||
// The body the dialog writes: the selection's ids, the format, the quality for a JPEG only, the mode (the preview's
|
||||
// while none is chosen), then dryRun.
|
||||
// The body the dialog writes: the selection's ids, the format, the quality for a JPEG only, the mode (the preview's,
|
||||
// Replace, while none is chosen), then dryRun.
|
||||
StringContent Body(string target, string quality, string mode, string tail) =>
|
||||
Json($$"""{"all":false,"sequenceIds":["{{id}}"],"target":"{{target}}","quality":{{quality}},"mode":"{{mode}}",{{tail}}}""");
|
||||
string preview;
|
||||
using (var dry = await browser.SendAsync(HttpMethod.Post, Url, Body("linear-dng", "null", "new-sequence", "\"dryRun\":true")))
|
||||
string fingerprint;
|
||||
using (var dry = await browser.SendAsync(HttpMethod.Post, Url, Body("linear-dng", "null", "replace", "\"dryRun\":true")))
|
||||
{
|
||||
await dry.ExpectAsync(HttpStatusCode.OK);
|
||||
using var plan = JsonDocument.Parse(await dry.Content.ReadAsStringAsync());
|
||||
var root = plan.RootElement;
|
||||
Assert.Equal(("linear-dng", "new-sequence"), (root.GetProperty("target").GetString(), root.GetProperty("mode").GetString()));
|
||||
Assert.Equal(("linear-dng", "replace"), (root.GetProperty("target").GetString(), root.GetProperty("mode").GetString()));
|
||||
Assert.Equal((1, 2, 0), (root.GetProperty("sequences").GetInt32(), root.GetProperty("photos").GetInt32(), root.GetProperty("skipped").GetInt32()));
|
||||
Assert.True(root.GetProperty("bytesBefore").GetInt64() > 0 && root.GetProperty("bytesAfter").GetInt64() > 0, "sizes");
|
||||
var space = root.GetProperty("space");
|
||||
Assert.True(space.GetProperty("enough").GetBoolean(), "enough");
|
||||
Assert.False(string.IsNullOrWhiteSpace(space.GetProperty("message").GetString()));
|
||||
preview = root.GetProperty("fingerprint").GetString()!;
|
||||
Assert.Equal(64, preview.Length);
|
||||
fingerprint = root.GetProperty("fingerprint").GetString()!;
|
||||
Assert.Equal(64, fingerprint.Length);
|
||||
var step = root.GetProperty("steps").EnumerateArray().Single();
|
||||
Assert.Equal(id, step.GetProperty("sequenceId").GetGuid());
|
||||
Assert.True(step.GetProperty("folder").GetString()!.Length > 0, "folder");
|
||||
@@ -132,14 +145,13 @@ public sealed partial class AdminConvertTests(TestContext context)
|
||||
Assert.Equal((0, 0), (step.GetProperty("skipped").GetArrayLength(), step.GetProperty("blocked").GetArrayLength()));
|
||||
}
|
||||
|
||||
// Replace chosen: the same photos, another fingerprint, so the dialog plans again before Apply.
|
||||
string fingerprint;
|
||||
using (var dry = await browser.SendAsync(HttpMethod.Post, Url, Body("linear-dng", "null", "replace", "\"dryRun\":true")))
|
||||
// New sequence chosen: the same photos, another fingerprint, so the dialog asks again before Apply (the server
|
||||
// answers from the plan it kept).
|
||||
using (var dry = await browser.SendAsync(HttpMethod.Post, Url, Body("linear-dng", "null", "new-sequence", "\"dryRun\":true")))
|
||||
{
|
||||
using var plan = JsonDocument.Parse(await dry.Content.ReadAsStringAsync());
|
||||
Assert.Equal(2, plan.RootElement.GetProperty("photos").GetInt32());
|
||||
fingerprint = plan.RootElement.GetProperty("fingerprint").GetString()!;
|
||||
Assert.NotEqual(preview, fingerprint);
|
||||
Assert.Equal(("new-sequence", 2), (plan.RootElement.GetProperty("mode").GetString(), plan.RootElement.GetProperty("photos").GetInt32()));
|
||||
Assert.NotEqual(fingerprint, plan.RootElement.GetProperty("fingerprint").GetString());
|
||||
}
|
||||
|
||||
// A quality the server refuses names its field: the dialog marks Quality.
|
||||
@@ -154,6 +166,14 @@ public sealed partial class AdminConvertTests(TestContext context)
|
||||
Assert.Equal(("same_format", 2), (skipped.GetProperty("reason").GetString(), skipped.GetProperty("photos").GetInt32()));
|
||||
Assert.Equal(Loc.Get("Archive.Convert.Skipped.SameFormat", italian), skipped.GetProperty("message").GetString());
|
||||
}
|
||||
// Into a new sequence, the same photos are not copied into it, and the table says so.
|
||||
using (var same = await browser.SendAsync(HttpMethod.Post, Url, Body("jpeg", "90", "new-sequence", "\"dryRun\":true")))
|
||||
{
|
||||
using var plan = JsonDocument.Parse(await same.Content.ReadAsStringAsync());
|
||||
var skipped = plan.RootElement.GetProperty("steps")[0].GetProperty("skipped").EnumerateArray().Single();
|
||||
Assert.Equal(("same_format", 2), (skipped.GetProperty("reason").GetString(), skipped.GetProperty("photos").GetInt32()));
|
||||
Assert.Equal(Loc.Get("Archive.Convert.NotCopied.SameFormat", italian), skipped.GetProperty("message").GetString());
|
||||
}
|
||||
// A plan that no longer matches is refused with the one there is now: the dialog plans again.
|
||||
using (var stale = await browser.SendAsync(HttpMethod.Post, Url, Body("linear-dng", "null", "replace", $"\"dryRun\":false,\"fingerprint\":\"{new string('0', 64)}\"")))
|
||||
Assert.Equal(fingerprint, (await stale.ExpectErrorAsync(HttpStatusCode.Conflict, "plan_changed")).Details);
|
||||
|
||||
Reference in new issue
Block a user