fix: projects and the workspace copy follow phase 4's folder and conversion rules

Phase 5a was written on an early phase 4a state. On top of phase 4b it now
keeps the rules every writer of a sequence's folder follows, and keeps
project versions intact through Reorganize and conversions:

- ProjectService writes project sidecars, project.json and the settings
  only under the sequence's lease and only in a folder that holds the
  sequence (SequenceFolders.Holds). A folder another index moved away, or
  one another sequence was moved into, is refused with
  SequenceFolderMovedException: reads answer from the index, so no
  project is taken for gone, nothing is written into the other sequence's
  folder and the old folder is never created again. The home reports it
  as out of reach. Full scans and rebuilds reconcile projects the same
  way, and a failure there is logged instead of failing the scan.
- Stored versions park positions by photo and now follow the encodings a
  Replace conversion (or its undo) joined through the sequence's Replaced
  tombstones (PhotoAlias, HomePhotos): Main's mirror, renders of a
  version and workspace media place them on the converted frames.
  ProjectContent.Rebind/SameChoices name parked photos as the frames hold
  them now, so a conversion or a hash since is no "external" Main version
  and no spurious "Modified" in a workspace, and merges see one photo.
- The workspace copy job only reads the archive: it no longer runs the
  archive's proxy pass, makes missing proxies in the workspace, reads
  only a folder that holds the sequence, starts again when the folder
  moved or files were renamed during the copy, resumes from a checkpoint
  of its frames version, and refuses a workspace folder inside any
  library before it writes anything (Errors.Workspace.InsideLibrary).
  The working state keeps its positions by frame where frames stay.
- ArchiveProjectHome looks for a project or version the index does not
  know yet in the sequence's sidecars first (an index taken back from a
  backup, a sequence back from the trash), instead of calling it gone.

Tests cover the moved and taken folder, conversions in the archive and in
the workspace, the copy refusing a workspace in a library, resuming on
other frames, the alias lookup and the rebinding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Alby96andClaude Opus 5.5 committed 2026-11-02 10:14:39 +01:00
1 parent b9d1c396a1
commit 50e28fd63d
18 files changed
+596 -91

No files matched your search

+3 -3
View File
@@ -36,7 +36,7 @@ Dependencies point one way: `Core` ← `Core.Windows`, `Data`, `Protocol` ← `S
| `Advice` | `Advisor`: recommendations with localisable reasons, apply/optimise, sequence presets |
| `Motion` | FFT, phase correlation, pyramids, optical flow, motion blur, stabiliser, virtual camera, interpolation, stacking |
| `Math` (namespace `Chronos.Core.Mathematics`) | Splines, LOWESS, statistics, solar position, 3×3 colour matrices |
| `Project` | `ProjectSettings` (Auto/manual values; members a newer Chronos wrote kept in `Extra` and written back; one of a newer settings format read with `FromJson(json, allowNewer: true)`, `IsNewerFormat`: read-only), export presets, `ProjectBundle` (the `.chronos` project file: sequence reference and settings; format 2 adds the `workspace` section of a workspace's working state: project, name, Main, home, base version, media; a file without it is still written as format 1), `ProjectContent` (the portable form of project versions: every frame position parked by photo with `TranslateFramePositions`, placed back on any copy's frames; canonical text, content hash = SHA-256 of its UTF-8 bytes) |
| `Project` | `ProjectSettings` (Auto/manual values; members a newer Chronos wrote kept in `Extra` and written back; one of a newer settings format read with `FromJson(json, allowNewer: true)`, `IsNewerFormat`: read-only), export presets, `ProjectBundle` (the `.chronos` project file: sequence reference and settings; format 2 adds the `workspace` section of a workspace's working state: project, name, Main, home, base version, media; a file without it is still written as format 1), `ProjectContent` (the portable form of project versions: every frame position parked by photo with `TranslateFramePositions`, placed back on any copy's frames; canonical text, content hash = SHA-256 of its UTF-8 bytes; `Rebind`/`SameChoices` name each parked photo as the frames hold it now, so a photo hashed or converted since is no change of the choices), `FramePhotos` (the photos of one copy's frames; a photo named by an encoding a conversion replaced, or its undo, is found through `PhotoAlias` links) |
| `Render` | Render plan (timing modes), preparation pass, geometry, `RenderPipeline`, self-test suite |
| `Export` | Preset resolution, encoder selection with fallbacks, output naming |
| `Video` (+ `Mp4`, `Avi`, `Bitstream`, `Yuv`, `Managed`) | `IVideoEncoderProvider`, MP4 muxer and inspector, AVI-MJPEG, NV12/P010, managed encoders (MJPEG-AVI, GIF, APNG, image sequences) |
@@ -61,8 +61,8 @@ Dependencies point one way: `Core` ← `Core.Windows`, `Data`, `Protocol` ← `S
| `Housekeeping` | `BackupJobHandler`, `CleanupJobHandler`, `ICleanupStep` with the shared steps (`CleanupSteps`: job history, audit log, trash report that never deletes); hosts add their own steps through `ArchiveServiceOptions.CleanupSteps` |
| `Archiving` | `ArchiveService` (create/adopt/open, archive settings written to `archive.json` then the index, `ArchiveSettings.CheckTemplates` and `TemplateRuleException` holding a template given anew to the template rules; `UnderLeaseAsync`: a write under a sequence's lease with its row read under it), `ArchiveIndexer` (scan, rebuild; a sequence found in another folder takes its exports' paths along; a full scan or a rebuild takes the projects' sidecars back too), `ArchiveIngest` (uploads and imports, named through `FrameNaming`), `ImportService`, `ArchiveVerifier`, `ArchiveImages` (thumbnail and proxy passes, several photos at once through `OrderedWork`, which hands results back in frame order), `FrameTiers` (decode tier, `PreviewOnly`/`Undecodable` flags and preview size of frames, written to sidecar and index by the light analysis, deep analysis, renders and image passes), `GprCompanions` (the JPEG of each GoPro GPR, found in the index when an import kept it in another sequence), `ArchiveMaintenance`, `MassRenamer` (mass rename and Reorganize: the dry run reads each sidecar for a file template; each step checked under its leases before anything of it moves (where its sequences are, what is inside and at its new folder, a sidecar of a newer format, the title it was planned from when it changes it or names the folder or files after it), then the folder, with the sequences of the plan an earlier step left as they were inside it (taken along, their title and file names kept), every index row (the frame paths and render paths of the exports inside it, a file or an image-sequence folder, its own unless the export is still at the old place and another sequence's when it moved with it, follow it in one transaction; a sidecar that cannot be read or written then still has its row follow, and fails the step), then the files under the plan's journal in `journal/` (a sequence whose photos changed since, such as one still receiving them, or whose files are found changed or in use or have their new names taken keeps its file names, the renames being pinned by digest, and the step and the plan go on); the journal keeps the tally of the whole plan, and a plan its job held (paused, cancelled or failed) is left to that job, only its half-done file renames completed at an archive open, which completes any other plan an interruption left and tells each sequence changed; the file renames of a step: sidecar, a sequence whose sidecar cannot be read staying as it is with every folder around it, index (`FrameRepository.RenameFilesAsync`) and the sequence's journal), `SequenceFolders` (`Holds`: the folder the index has for a sequence is there and its sidecar names no other sequence; every writer into a sequence's folder asks it), `SequenceTrash` (delete into the archive's hidden trash with a dry run, restore; the folder moves back when the index step fails; lists sequence and photo entries; `SequenceTrash.Empty`: the emptying's dry run, fingerprint and erase, each entry checked against its fingerprint again under its lease), `FrameTrash` (photos of a sequence into `trash/photos` with a dry run, a journal that undoes an interrupted deletion whose entry stays unlisted until its sidecar is written, tombstones (a photo never hashed is hashed as it goes; one whose original is missing goes without a file or tombstone; a frame that stays and may be a twin is hashed, its hash kept), positions parked and placed back, proxies and thumbnails moved to follow their frames; `FrameTrash.Restore`: restore re-hashes each file and says why each kept photo stayed (`KeptReason`), under a journal of its own into the sequence that another writer settles only under the entry's lease, into a new sequence through the ingest when theirs is gone, each original staged inside its own entry and the record rewritten after each photo filed; a replaced entry swaps back while its converted frames are unchanged; `FrameTrash.Replace`: the files of frames swapped in place, same ids and indices, the old files into one `Replaced` entry with their replacements' hashes and a copy of `analysis.json`, tombstones aliasing old content to new, under `.chronos/replace.pending.json`, which the next writer undoes or completes), `MassConverter` (conversion's dry run with busy, offline and unreadable sequences and the space against `ArchiveServiceOptions.FreeSpace`; the job's stage A converts into `staging/convert-<job>/` with `OrderedWork`, each staged file appended to `journal/convert-<job>.staged`; stage B per sequence, all or nothing: replace through `FrameTrash.ReplaceAsync` (the sequence's folder read again under its lease, as every writer does), or a new sequence through the ingest with the source's tags, rating, notes and project; each sequence replaced or made told as it completes), job handlers |
| `Processing` | `ProcessingService`, `SequenceAnalyser`, `DeepReports` (a saved deep report while it still describes the frames), `SequenceRender`, `ProjectStore`, `ProjectBundles` (a `.chronos` file for a sequence, and the sequence a file names), `RenderNodeClient`, job handlers |
| `Projects` | `ProjectService` (`ArchiveService.Projects`: the projects of this PC's sequences as a home, under the sequence's lease: list, create, rename, delete into `projects/.trash`, history, push with fast-forward / idempotent repeat / divergence / hash and portability checks; each call first catches up: the sidecars into the index (`ProjectIndexer`, also run by full scans and rebuilds; a version file a push left past its record adopted, a stray one moved to `versions/.stray`), Main made from the sequence's settings, settings another path wrote appended to Main as an `external` version, a mirror a crash cut short completed or merged; a push to Main mirrored into `project.json` and the settings with the settings version check), `ProjectSidecars` (record and immutable version files, the settings as a JSON string checked against their hash), `ProjectMerge` (three-way merge member by member, pending frames merged photo by photo, conflicts, `Replay` of local commits on a moved head), `IProjectHome` and `ArchiveProjectHome` (the home a workspace pushes to and pulls from; the server's comes in phase 5b) |
| `Workspaces` | `WorkspaceStore` (a folder outside the library: `workspace.json`, one folder per project, no index), `WorkspaceProject` (`project.chronos` working state, `media/` with `media.json`, `proxies/<frame id>.jpg` and `analysis.json`, `versions/`, `sync.json` for a restore or a conflict, `exports/`, the lease `.lock.lease`), `WorkspaceSync` (checkout, open, save, commit, push, pull with replay and merge, conflicts settled by keep mine, take server or save both, restore, status), `WorkspaceCopyJobHandler` (`WorkspaceJobs.Copy`, registered by the desktop only: proxies copied hash-verified and resumable, the working state's positions moved by photo onto the new frames) |
| `Projects` | `ProjectService` (`ArchiveService.Projects`: the projects of this PC's sequences as a home, under the sequence's lease and only in a folder that holds the sequence (`SequenceFolders.Holds`; otherwise `SequenceFolderMovedException`: reads answer from the index, writes are refused): list, create, rename, delete into `projects/.trash`, history, push with fast-forward / idempotent repeat / divergence / hash and portability checks; each call first catches up: the sidecars into the index (`ProjectIndexer`, also run by full scans and rebuilds; a version file a push left past its record adopted, a stray one moved to `versions/.stray`), Main made from the sequence's settings, settings another path wrote appended to Main as an `external` version, a mirror a crash cut short completed or merged; a push to Main mirrored into `project.json` and the settings with the settings version check), `ProjectSidecars` (record and immutable version files, the settings as a JSON string checked against their hash), `ProjectMerge` (three-way merge member by member, pending frames merged photo by photo, conflicts, `Replay` of local commits on a moved head), `IProjectHome` and `ArchiveProjectHome` (the home a workspace pushes to and pulls from, a project or version the index lacks looked for in the sidecars first; the server's comes in phase 5b), `HomePhotos` (a sequence's photos as positions name them, with the encodings its Replaced tombstones join) |
| `Workspaces` | `WorkspaceStore` (a folder outside the library: `workspace.json`, one folder per project, no index), `WorkspaceProject` (`project.chronos` working state, `media/` with `media.json`, `proxies/<frame id>.jpg` and `analysis.json`, `versions/`, `sync.json` for a restore or a conflict, `exports/`, the lease `.lock.lease`), `WorkspaceSync` (checkout, open, save, commit, push, pull with replay and merge, conflicts settled by keep mine, take server or save both, restore, status), `WorkspaceCopyJobHandler` (`WorkspaceJobs.Copy`, registered by the desktop only: the archive only read, from a folder that holds the sequence; a workspace inside a library refused; the archive's current proxies copied, the others made in the workspace, each hash-verified, resumable from a checkpoint of its frames version; the encodings conversions joined kept in `media.json`; the working state's positions moved by frame where frames stay, by photo where they came or went) |
| `Sync` | `SyncService` (sync table of one server: compare, link by content, settle conflicts, enqueue transfers), `SyncRules` (status, settings merge and differences, no I/O), `SyncTombstones` (photos either side removed are not offered back, a replaced photo counts as its replacement), `SyncPositions` (the settings' frame positions translated by photo to the frames of the copy they go to), `SyncLinkStore` (`sync_links`, `sync_state`), `SyncTransfers` and `SyncJobHandler` (resumable, hash-verified uploads and downloads as jobs; downloads commit through the archive ingest), `SyncAgent` (background comparison, server events, backoff, offline proxies), `ProxyCache` and `RemoteSequence` (server sequences opened with proxies), `ISyncRemote` (the server calls; `ChronosSyncRemote` over `ChronosClient`) |
### 2.3 Chronos.App areas
+1
View File
@@ -53,6 +53,7 @@
<data name="Errors.Workspace.Damaged" xml:space="preserve"><value>A file of this project in the workspace cannot be read; the log names it.</value></data>
<data name="Errors.Workspace.EditNotStored" xml:space="preserve"><value>The last edit could not be stored yet; it is tried again, try once more in a moment.</value></data>
<data name="Errors.Workspace.HomeCheck" xml:space="preserve"><value>The library's copy of this project does not match what was sent; nothing was removed.</value></data>
<data name="Errors.Workspace.InsideLibrary" xml:space="preserve"><value>The workspace folder is inside a library; nothing is copied into it.</value></data>
<data name="Errors.Workspace.InvalidName" xml:space="preserve"><value>A project name has 1 to 60 characters.</value></data>
<data name="Errors.Workspace.MainKept" xml:space="preserve"><value>The Main project holds the sequence's own settings: it cannot be deleted.</value></data>
<data name="Errors.Workspace.MediaChanged" xml:space="preserve"><value>The workspace's photos of this project changed; the editor reads them again.</value></data>
+1
View File
@@ -53,6 +53,7 @@
<data name="Errors.Workspace.Damaged" xml:space="preserve"><value>Un file di questo progetto nello spazio di lavoro non è leggibile; il log lo indica.</value></data>
<data name="Errors.Workspace.EditNotStored" xml:space="preserve"><value>L'ultima modifica non è ancora stata memorizzata; viene ritentata, riprova tra un momento.</value></data>
<data name="Errors.Workspace.HomeCheck" xml:space="preserve"><value>La copia di questo progetto nella libreria non corrisponde a quanto inviato; non è stato rimosso nulla.</value></data>
<data name="Errors.Workspace.InsideLibrary" xml:space="preserve"><value>La cartella dello spazio di lavoro è dentro una libreria; non vi viene copiato nulla.</value></data>
<data name="Errors.Workspace.InvalidName" xml:space="preserve"><value>Il nome di un progetto ha da 1 a 60 caratteri.</value></data>
<data name="Errors.Workspace.MainKept" xml:space="preserve"><value>Il progetto Principale contiene le impostazioni della sequenza: non può essere eliminato.</value></data>
<data name="Errors.Workspace.MediaChanged" xml:space="preserve"><value>Le foto di questo progetto nello spazio di lavoro sono cambiate; l'editor le rilegge.</value></data>
+74 -1
View File
@@ -35,10 +35,27 @@ public sealed class FramePhotos
private readonly Dictionary<string, List<int>> _bySha = new(StringComparer.Ordinal);
private readonly Dictionary<(long Size, long Time), List<int>> _bySizeAndTime = [];
/// <summary>Each SHA-256 an alias links, and the one its photo is known by among them (empty without aliases).</summary>
private readonly Dictionary<string, string> _canonical;
/// <summary>The frames holding each linked photo, by the SHA-256 it is known by (<see cref="_canonical"/>).</summary>
private readonly Dictionary<string, List<int>> _byCanonical = new(StringComparer.Ordinal);
/// <param name="frames">Each frame's index and photo, in any order.</param>
public FramePhotos(IEnumerable<(int Index, FramePhoto Photo)> frames)
: this(frames, null)
{
}
/// <param name="frames">Each frame's index and photo, in any order.</param>
/// <param name="aliases">
/// Encodings of one photo (owner review 2, R7: a conversion replaced a frame's file, its undo put it back): a photo
/// named by an encoding no frame holds is held by the frames holding another encoding the links join to it.
/// </param>
public FramePhotos(IEnumerable<(int Index, FramePhoto Photo)> frames, IEnumerable<PhotoAlias>? aliases)
{
ArgumentNullException.ThrowIfNull(frames);
_canonical = PhotoAlias.Classes(aliases);
var list = frames.Where(f => f.Index >= 0).ToList();
_byIndex = new FramePhoto?[list.Count == 0 ? 0 : list.Max(f => f.Index) + 1];
foreach (var (index, photo) in list) _byIndex[index] = photo;
@@ -50,6 +67,11 @@ public sealed class FramePhotos
var sha = photo.Sha256!.ToLowerInvariant();
if (!_bySha.TryGetValue(sha, out var copies)) _bySha[sha] = copies = [];
copies.Add(i);
if (_canonical.TryGetValue(sha, out var known))
{
if (!_byCanonical.TryGetValue(known, out var linked)) _byCanonical[known] = linked = [];
linked.Add(i);
}
}
if (photo.CapturedAt is not { } time) continue;
var key = (photo.Size, FramePhoto.Millisecond(time));
@@ -91,7 +113,13 @@ public sealed class FramePhotos
/// </summary>
private List<int> Holding(FramePhoto photo)
{
if (photo.IsHashed && _bySha.TryGetValue(photo.Sha256!.ToLowerInvariant(), out var hashed)) return hashed;
if (photo.IsHashed)
{
var sha = photo.Sha256!.ToLowerInvariant();
if (_bySha.TryGetValue(sha, out var hashed)) return hashed;
// Another encoding of the photo (a conversion, or its undo) is what the frames hold now.
if (_canonical.TryGetValue(sha, out var known) && _byCanonical.TryGetValue(known, out var linked)) return linked;
}
var holding = new List<int>(1);
if (photo.CapturedAt is not { } time || !_bySizeAndTime.TryGetValue((photo.Size, FramePhoto.Millisecond(time)), out var candidates)) return holding;
foreach (var index in candidates)
@@ -114,3 +142,48 @@ public sealed class FramePhotos
return true;
}
}
/// <summary>
/// Two encodings of one photo (owner review 2, R7): a frame's file holding <see cref="Sha256"/> that a conversion replaced
/// by one holding <see cref="Replacement"/> (a tombstone of reason "replaced"), or the undo of one. Links join in either
/// direction and through chains (a conversion of a conversion), so a position parked on any encoding finds the frame that
/// holds another (<see cref="FramePhotos"/>).
/// </summary>
public sealed record PhotoAlias(string Sha256, string Replacement)
{
/// <summary>
/// Each SHA-256 the links join (union-find), lower-case, mapped to the ordinal-smallest of its photo, so the result does
/// not depend on the order of the links. Empty without links.
/// </summary>
internal static Dictionary<string, string> Classes(IEnumerable<PhotoAlias>? aliases)
{
var parent = new Dictionary<string, string>(StringComparer.Ordinal);
if (aliases is null) return parent;
foreach (var alias in aliases)
{
if (alias is null || string.IsNullOrEmpty(alias.Sha256) || string.IsNullOrEmpty(alias.Replacement)) continue;
var (a, b) = (alias.Sha256.ToLowerInvariant(), alias.Replacement.ToLowerInvariant());
if (a == b) continue;
parent.TryAdd(a, a);
parent.TryAdd(b, b);
var (x, y) = (Find(a), Find(b));
if (x == y) continue;
if (string.CompareOrdinal(x, y) < 0) parent[y] = x;
else parent[x] = y;
}
return parent.Keys.ToList().ToDictionary(k => k, Find, StringComparer.Ordinal);
string Find(string sha)
{
var root = sha;
while (parent[root] != root) root = parent[root];
while (parent[sha] != root)
{
var next = parent[sha];
parent[sha] = root;
sha = next;
}
return root;
}
}
}
@@ -54,6 +54,58 @@ public static class ProjectContent
return ProjectSettings.TranslateFramePositions(portableJson, null, frames) ?? portableJson;
}
/// <summary>
/// The portable text with each pending frame whose photo <paramref name="frames"/> hold named as they hold it now: an
/// encoding a conversion replaced by the one the frame holds (<see cref="PhotoAlias"/>, also its undo), a photo parked
/// before it was hashed by its SHA-256, size and capture time. The same choices on the same photos then read as the same
/// text (<see cref="SameChoices"/>), whichever encoding or hashing state they were saved against; a placement finds the
/// same frames either way. Canonical when anything is renamed; the text as it is otherwise, and for a newer format.
/// </summary>
/// <exception cref="JsonException">The text is not a project object.</exception>
public static string Rebind(string portableJson, FramePhotos frames)
{
ArgumentNullException.ThrowIfNull(portableJson);
ArgumentNullException.ThrowIfNull(frames);
if (frames.Count == 0 || !portableJson.Contains("pendingFrames", StringComparison.OrdinalIgnoreCase)) return portableJson;
if (FormatOf(portableJson) > ProjectSettings.CurrentVersion) return portableJson;
if (JsonNode.Parse(portableJson, new JsonNodeOptions { PropertyNameCaseInsensitive = true }) is not JsonObject project
|| project["pendingFrames"] is not JsonArray pending) return portableJson;
var renamed = false;
foreach (var entry in pending.OfType<JsonObject>())
{
var sha = entry["sha256"] is JsonValue s && s.TryGetValue<string>(out var text) ? text : null;
var size = entry["size"] is JsonValue z && z.TryGetValue<long>(out var bytes) ? bytes : -1;
DateTime? captured = entry["capturedAt"] is JsonValue t && t.TryGetValue<DateTime>(out var time) ? time : null;
if (string.IsNullOrEmpty(sha) && (captured is null || size < 0)) continue;
var occurrence = entry["occurrence"] is JsonValue o && o.TryGetValue<int>(out var n) && n > 0 ? n : 0;
// Only a frame that is hashed names a photo better than the entry does.
if (frames.IndexOf(new FramePhoto(sha, size, captured), occurrence) is not { } at || frames[at] is not { IsHashed: true } held) continue;
var sameTime = held.CapturedAt is { } heldAt
? captured is { } entryAt && FramePhoto.Millisecond(heldAt) == FramePhoto.Millisecond(entryAt)
: captured is null;
if (string.Equals(held.Sha256, sha, StringComparison.OrdinalIgnoreCase) && held.Size == size && sameTime) continue;
entry["sha256"] = held.Sha256!.ToLowerInvariant();
entry["size"] = held.Size;
if (held.CapturedAt is { } heldTime) entry["capturedAt"] = heldTime;
else entry.Remove("capturedAt");
renamed = true;
}
return renamed ? Canonical(project.ToJsonString()) : portableJson;
}
/// <summary>
/// Whether two portable texts hold the same choices on the frames of <paramref name="frames"/>: the same text, or the
/// same once their pending frames are named as the frames hold their photos now (<see cref="Rebind"/>).
/// </summary>
/// <exception cref="JsonException">A text is not a project object.</exception>
public static bool SameChoices(string portable, string other, FramePhotos frames)
{
ArgumentNullException.ThrowIfNull(portable);
ArgumentNullException.ThrowIfNull(other);
ArgumentNullException.ThrowIfNull(frames);
return portable == other || (frames.Count > 0 && Rebind(portable, frames) == Rebind(other, frames));
}
/// <summary>
/// The canonical text of a project: read (a newer Chronos's too, its unknown members kept) and written again, its
/// pending frames in a fixed order (by photo) and their capture times as wall-clock values.
+16 -5
View File
@@ -345,17 +345,28 @@ public sealed class ArchiveIndexer
private SequenceStatus Uncovered => _writer.Options.ImagesAfterScan ? CoveringStatus : SequenceStatus.None;
/// <summary>
/// The projects of a sequence brought in line with their sidecars (<see cref="ProjectIndexer"/>), under its lease; nothing
/// for a sequence without a projects folder (its Main is made when a project is first asked for).
/// The projects of a sequence brought in line with their sidecars (<see cref="ProjectIndexer"/>), under its lease, in the
/// folder the index has for it then, which must hold it (<see cref="SequenceFolders.Holds"/>: another sequence's projects
/// are never taken, and a folder another index moved away never marks the projects gone). Nothing for a sequence without
/// a projects folder (its Main is made when a project is first asked for). A failure is logged and leaves the folder
/// indexed: the next project operation on the sequence catches up again.
/// </summary>
private async Task ReconcileProjectsAsync(ArchivePaths paths, Guid sequenceId, CancellationToken cancellationToken)
{
if (await _db.Sequences.GetAsync(sequenceId, cancellationToken).ConfigureAwait(false) is not { } row
|| !Directory.Exists(paths.Sequence(row.Folder).ProjectsDirectory)) return;
await using (await _writer.LockAsync(paths, sequenceId, cancellationToken).ConfigureAwait(false))
try
{
var folder = await SequenceWriter.FolderAsync(_db, paths, row, cancellationToken).ConfigureAwait(false);
await _projects.ReconcileAsync(folder, sequenceId, cancellationToken).ConfigureAwait(false);
await using (await _writer.LockAsync(paths, sequenceId, cancellationToken).ConfigureAwait(false))
{
var folder = await SequenceWriter.FolderAsync(_db, paths, row, cancellationToken).ConfigureAwait(false);
if (!SequenceFolders.Holds(folder, sequenceId) || !Directory.Exists(folder.ProjectsDirectory)) return;
await _projects.ReconcileAsync(folder, sequenceId, cancellationToken).ConfigureAwait(false);
}
}
catch (Exception ex) when (ex is IOException or InvalidDataException or UnauthorizedAccessException)
{
Log.Warning("projects", $"The projects of {row.Folder} could not be read back; the next project operation tries again", ex);
}
}
@@ -1,7 +1,14 @@
using Chronos.Core.Archiving;
using Chronos.Core.Localization;
namespace Chronos.Data.Archiving;
/// <summary>
/// The folder the index has for a sequence does not hold it (<see cref="SequenceFolders.Holds"/>): another index of the
/// archive moved it away, maybe another sequence in; this index's next scan finds both. Nothing was written.
/// </summary>
public sealed class SequenceFolderMovedException() : IOException(Loc.L("Errors.Archive.FolderMoved"));
/// <summary>Whether a sequence is in the folder the index has for it, as writers that file into that folder ask.</summary>
public static class SequenceFolders
{
@@ -50,7 +50,8 @@ internal sealed class RenderJobHandler(ChronosDatabase db, ProcessingOptions opt
if (options.Host == EncoderHost.Server) export = export with { Folder = null };
var embedded = parameters.SettingsJson ?? await VersionSettingsAsync(parameters, sequence, cancellation).ConfigureAwait(false);
var frames = await SequenceFrames.LoadAsync(db, archive, sequence, parameters.UseProxies, options.Archive.Decoder, cancellation).ConfigureAwait(false);
var project = (embedded is not null ? ProjectStore.Parse(Placed(embedded, frames)) : ProjectStore.Read(sequence, folder)) with { Export = export };
var aliases = embedded is not null ? await Projects.HomePhotos.AliasesAsync(db, sequence.Id, cancellation).ConfigureAwait(false) : [];
var project = (embedded is not null ? ProjectStore.Parse(Placed(embedded, frames, aliases)) : ProjectStore.Read(sequence, folder)) with { Export = export };
var settingsJson = project.ToJson();
using var names = new OutputNames();
@@ -138,14 +139,15 @@ internal sealed class RenderJobHandler(ChronosDatabase db, ProcessingOptions opt
/// <summary>
/// Settings whose positions wait for their photos (a project version's portable form, settings synced from another copy)
/// with them placed on the frames rendered; others as they are.
/// with them placed on the frames rendered, a photo parked on an encoding a conversion replaced since found through
/// <paramref name="aliases"/> (the sequence's tombstones of reason "replaced"); others as they are.
/// </summary>
internal static string Placed(string settingsJson, SequenceFrames frames)
internal static string Placed(string settingsJson, SequenceFrames frames, IReadOnlyList<PhotoAlias>? aliases = null)
{
if (!settingsJson.Contains("pendingFrames", StringComparison.OrdinalIgnoreCase)) return settingsJson;
try
{
return ProjectContent.Place(settingsJson, new FramePhotos(frames.Frames.Select(f => (f.Index, new FramePhoto(f.Sha256, f.Size, f.CapturedAt)))));
return ProjectContent.Place(settingsJson, new FramePhotos(frames.Frames.Select(f => (f.Index, new FramePhoto(f.Sha256, f.Size, f.CapturedAt))), aliases));
}
catch (System.Text.Json.JsonException)
{
+25
View File
@@ -0,0 +1,25 @@
using Chronos.Core.Model;
using Chronos.Core.Project;
using Chronos.Data.Repositories;
namespace Chronos.Data.Projects;
/// <summary>
/// The photos of a sequence's frames at its home as project positions name them (owner review 2, R8), with the encodings
/// a conversion joined (R7): a version parked on a photo before its file was converted, or after a conversion was undone,
/// still finds its frame.
/// </summary>
internal static class HomePhotos
{
/// <summary>The links between encodings of the sequence's photos: its tombstones of reason "replaced".</summary>
public static async Task<IReadOnlyList<PhotoAlias>> AliasesAsync(ChronosRepositories db, Guid sequenceId, CancellationToken cancellationToken) =>
[
.. (await db.FrameRemovals.ListBySequenceAsync(sequenceId, cancellationToken).ConfigureAwait(false))
.Where(r => r.Reason == RemovalReason.Replaced && !string.IsNullOrEmpty(r.ReplacementSha256))
.Select(r => new PhotoAlias(r.Sha256, r.ReplacementSha256!)),
];
/// <summary>The photos of <paramref name="frames"/> (rows of the index, any order) with <paramref name="aliases"/>.</summary>
public static FramePhotos Of(IEnumerable<Frame> frames, IReadOnlyList<PhotoAlias> aliases) =>
new(frames.Select(f => (f.Index, new FramePhoto(f.Sha256, f.Size, f.CapturedAt))), aliases);
}
+31 -5
View File
@@ -1,6 +1,7 @@
using Chronos.Core.Archiving;
using Chronos.Core.Model;
using Chronos.Core.Project;
using Chronos.Data.Archiving;
namespace Chronos.Data.Projects;
@@ -78,8 +79,17 @@ public sealed class ArchiveProjectHome(ChronosDatabase db, ProjectService projec
return await projects.HeadAsync(projectId, cancellationToken).ConfigureAwait(false);
});
public async Task<ProjectVersion?> GetVersionAsync(Guid projectId, Guid versionId, CancellationToken cancellationToken) =>
await projects.GetVersionAsync(versionId, cancellationToken).ConfigureAwait(false) is { } version && version.ProjectId == projectId ? version : null;
public Task<ProjectVersion?> GetVersionAsync(Guid projectId, Guid versionId, CancellationToken cancellationToken) =>
Reach(async () =>
{
if (await projects.GetVersionAsync(versionId, cancellationToken).ConfigureAwait(false) is not { } version)
{
// A version the index does not know yet may be in the sequence's sidecars (see Own): they are read first.
await projects.ListAsync(sequenceId, cancellationToken).ConfigureAwait(false);
version = await projects.GetVersionAsync(versionId, cancellationToken).ConfigureAwait(false);
}
return version is not null && version.ProjectId == projectId ? version : null;
});
public Task<IReadOnlyList<ProjectVersion>> VersionsAfterAsync(Guid projectId, int afterNumber, int limit, CancellationToken cancellationToken) =>
Reach(async () =>
@@ -104,11 +114,22 @@ public sealed class ArchiveProjectHome(ChronosDatabase db, ProjectService projec
public Task<HomeFrames> FramesAsync(CancellationToken cancellationToken) => projects.FramesAsync(sequenceId, cancellationToken);
/// <summary>The project is one of this sequence's.</summary>
private async Task<SequenceProject> Own(Guid projectId, CancellationToken cancellationToken) =>
await projects.GetAsync(projectId, cancellationToken).ConfigureAwait(false) is { } project && project.SequenceId == sequenceId
/// <summary>
/// The project is one of this sequence's. One the index does not know yet (an index taken back from an older backup, a
/// sequence back from the trash, a project another program sharing the archive made) is looked for in the sequence's
/// sidecars first, which every listing brings into the index.
/// </summary>
private async Task<SequenceProject> Own(Guid projectId, CancellationToken cancellationToken)
{
if (await projects.GetAsync(projectId, cancellationToken).ConfigureAwait(false) is not { } project)
{
await projects.ListAsync(sequenceId, cancellationToken).ConfigureAwait(false);
project = await projects.GetAsync(projectId, cancellationToken).ConfigureAwait(false);
}
return project is not null && project.SequenceId == sequenceId
? project
: throw new ProjectException(ProjectError.NotFound, $"Project {projectId} is not a project of sequence {sequenceId}.");
}
/// <summary>An archive offline, read-only or busy is a home out of reach for now.</summary>
private static async Task<T> Reach<T>(Func<Task<T>> call)
@@ -125,5 +146,10 @@ public sealed class ArchiveProjectHome(ChronosDatabase db, ProjectService projec
{
throw new HomeUnavailableException(ex.Message, ex);
}
catch (SequenceFolderMovedException ex)
{
// Another index of the archive moved the sequence: this one finds it at its next scan.
throw new HomeUnavailableException(ex.Message, ex);
}
}
}
+21 -8
View File
@@ -317,13 +317,20 @@ public sealed class ProjectService
{
await UnderLeaseAsync(sequenceId, (_, _) => Task.FromResult(true), cancellationToken, ReadWait).ConfigureAwait(false);
}
catch (Exception ex) when (ex is ArchiveOfflineException or LeaseUnavailableException)
catch (Exception ex) when (ex is ArchiveOfflineException or LeaseUnavailableException or SequenceFolderMovedException)
{
Log.Info("projects", $"The projects of sequence {sequenceId} are read from the index: {ex.Message}");
}
}
/// <summary>Runs <paramref name="work"/> under the sequence's lease, after catching its projects up.</summary>
/// <summary>
/// Runs <paramref name="work"/> under the sequence's lease, after catching its projects up, in the folder the index has
/// for the sequence as read under the lease (a Reorganize moves it under the same lease), which must hold it.
/// </summary>
/// <exception cref="SequenceFolderMovedException">
/// The folder does not hold the sequence (another index of the archive moved it, maybe another sequence in): nothing is
/// read from it or written into it, so no project is taken for gone and no file lands in another sequence's folder.
/// </exception>
private async Task<T> UnderLeaseAsync<T>(Guid sequenceId, Func<LeaseContext, CancellationToken, Task<T>> work, CancellationToken cancellationToken,
TimeSpan? wait = null)
{
@@ -334,7 +341,9 @@ public sealed class ProjectService
await using (await _writer.LockAsync(paths, sequenceId, wait, cancellationToken).ConfigureAwait(false))
{
var row = await _db.Sequences.GetAsync(sequenceId, cancellationToken).ConfigureAwait(false) ?? throw SequenceMissing(sequenceId);
var context = new LeaseContext(paths.Sequence(row.Folder), row);
var folder = paths.Sequence(row.Folder);
if (!SequenceFolders.Holds(folder, sequenceId)) throw new SequenceFolderMovedException();
var context = new LeaseContext(folder, row);
await CatchUpAsync(context, cancellationToken).ConfigureAwait(false);
return await work(context, cancellationToken).ConfigureAwait(false);
}
@@ -359,7 +368,8 @@ public sealed class ProjectService
await _db.Projects.SetMirroredAsync(main.Id, main.HeadVersionId, context.Row.SettingsVersion, cancellationToken).ConfigureAwait(false);
return;
}
var sameAsHead = ProjectContent.Hash(current) == head.ContentHash;
// The head parks its photos as the frames held them when it was made: a conversion or a hash since names them anew.
var sameAsHead = ProjectContent.SameChoices(head.SettingsJson, current, frames.Photos);
if (main.MirroredVersionId == main.HeadVersionId || main.MirroredVersionId is null)
{
// Only the sequence's settings version moved (settings another path wrote, or only its title or tags), or the index
@@ -375,14 +385,16 @@ public sealed class ProjectService
return;
}
var mirrored = await _db.Projects.GetVersionAsync(main.MirroredVersionId.Value, cancellationToken).ConfigureAwait(false);
if (main.MirroredSettingsVersion == context.Row.SettingsVersion || mirrored is null || ProjectContent.Hash(current) == mirrored.ContentHash)
if (main.MirroredSettingsVersion == context.Row.SettingsVersion || mirrored is null || ProjectContent.SameChoices(mirrored.SettingsJson, current, frames.Photos))
{
// Nobody changed the settings since the last mirror: the mirror completes.
await MirrorAsync(context, main, head, cancellationToken).ConfigureAwait(false);
return;
}
// Both moved: the settings written meanwhile merge with the head (the version made in a workspace wins where both changed).
var merged = ProjectMerge.Merge(ProjectMerge.Read(mirrored.SettingsJson), ProjectMerge.Read(head.SettingsJson), ProjectMerge.Read(current));
// Both moved: the settings written meanwhile merge with the head (the version made in a workspace wins where both changed),
// each photo named as the frames hold it now so the same photo merges as one.
var merged = ProjectMerge.Merge(ProjectMerge.Read(ProjectContent.Rebind(mirrored.SettingsJson, frames.Photos)),
ProjectMerge.Read(ProjectContent.Rebind(head.SettingsJson, frames.Photos)), ProjectMerge.Read(ProjectContent.Rebind(current, frames.Photos)));
var version = await AppendDerivedAsync(context, main, head, ProjectContent.Canonical(merged.ToJson()), ProjectVersionKind.Merge, cancellationToken)
.ConfigureAwait(false);
main = (await _db.Projects.GetAsync(main.Id, cancellationToken).ConfigureAwait(false))!;
@@ -550,7 +562,8 @@ public sealed class ProjectService
if (page.Count < PageSize) break;
after = page[^1].Index;
}
return (frames, new FramePhotos(frames.Select(f => (f.Index, new FramePhoto(f.Sha256, f.Size, f.CapturedAt)))));
// With the encodings conversions joined: a version parked on a photo before its file was converted still finds its frame.
return (frames, HomePhotos.Of(frames, await HomePhotos.AliasesAsync(_db, sequenceId, cancellationToken).ConfigureAwait(false)));
}
private async Task EnsureNameFreeAsync(Guid sequenceId, Guid? projectId, string name, CancellationToken cancellationToken)
@@ -10,6 +10,7 @@ using Chronos.Core.Project;
using Chronos.Data.Archiving;
using Chronos.Data.Archiving.Jobs;
using Chronos.Data.Jobs;
using Chronos.Data.Projects;
namespace Chronos.Data.Workspaces;
@@ -37,12 +38,17 @@ public static class WorkspaceJobs
/// <summary>
/// <see cref="JobKind.WorkspaceCopy"/> for a sequence of this PC's archive (owner review 2, R8): its proxies into the
/// workspace project's <c>media/proxies/&lt;frame id&gt;.jpg</c>, the archive's own when current, the missing ones made
/// first by the archive's proxy pass (made here from the original when the archive makes none), each written atomically
/// and read back to check its SHA-256; then the deep report (<c>media/analysis.json</c>) and, last, <c>media/media.json</c>.
/// Frames changed at the home during the copy make it start again (three times at most). The working state's positions
/// follow the new frames by photo. Checkpoint: <c>proxies:&lt;frame&gt;</c> during the archive's pass, then
/// <c>copy:&lt;frame&gt;</c>. Originals come in phase 5c: asked for, the job fails saying so.
/// workspace project's <c>media/proxies/&lt;frame id&gt;.jpg</c>, the archive's own when current, else made here from the
/// original (at the archive's proxy size, or the default one when the archive makes none), each written atomically and read
/// back to check its SHA-256; then the deep report (<c>media/analysis.json</c>) and, last, <c>media/media.json</c> with the
/// encodings conversions joined at the home. The job only reads the archive: it never writes there, not even proxies (the
/// archive's own pass makes those), refuses a workspace folder inside any library of the index before it writes anything,
/// and reads only a folder that holds the sequence (<see cref="SequenceFolders.Holds"/>).
/// Frames changed at the home during the copy, or the folder moved (a Reorganize), make it start again (three times at
/// most; a folder still moving fails it, to be retried). The working state's positions follow the new frames: by frame
/// where a frame stays (one whose file a conversion replaced keeps its positions), by photo where frames came or went.
/// Checkpoint: <c>copy:&lt;frame&gt;:&lt;frames version&gt;</c>; a run resumed on other frames checks every file again.
/// Originals come in phase 5c: asked for, the job fails saying so.
/// </summary>
internal sealed class WorkspaceCopyJobHandler(ChronosDatabase db, ArchiveService archives) : IJobHandler
{
@@ -62,35 +68,29 @@ internal sealed class WorkspaceCopyJobHandler(ChronosDatabase db, ArchiveService
if (context.Job.TargetIds is not [var sequenceId] || await db.Sequences.GetAsync(sequenceId, ct).ConfigureAwait(false) is not { } sequence
|| await db.Archives.GetAsync(sequence.ArchiveId, ct).ConfigureAwait(false) is not { } archive)
throw new InvalidOperationException(Loc.L("Archive.Log.SequenceMissing", context.Job.TargetIds.FirstOrDefault()));
// Asked before the store is opened, which makes its folder: editing happens outside every library (owner review 2,
// R8), and a workspace inside one would put its copies among that library's own files.
if (await InsideALibraryAsync(parameters.WorkspaceRoot, ct).ConfigureAwait(false))
throw new InvalidOperationException(Loc.L("Errors.Workspace.InsideLibrary"));
var store = new WorkspaceStore(parameters.WorkspaceRoot);
var project = store.Find(parameters.ProjectId) ?? throw new InvalidOperationException(Loc.L("Errors.Workspace.NotCheckedOut"));
if (project.ReadBundle().Workspace is not { Home: { IsArchive: true } home } || home.SequenceId != sequenceId)
throw new InvalidOperationException(Loc.L("Errors.Workspace.OtherHome"));
var paths = new ArchivePaths(archive.RootPath);
var (stage, next) = Parse(context.Checkpoint);
// Read only: a read-only archive is copied from like any other.
if (ArchiveVolume.Probe(paths) == ArchiveAvailability.Offline) throw new ArchiveOfflineException(archive.RootPath, ArchiveAvailability.Offline);
var (copied, checkpointVersion) = Parse(context.Checkpoint);
for (var pass = 1; ; pass++)
{
var (row, frames) = await FramesAsync(sequenceId, ct).ConfigureAwait(false);
var policy = ManifestStore.ReadArchive(paths.ManifestPath).ProxyPolicy;
var makesProxies = policy.LongEdge != ProxyPolicy.Disabled && ArchiveVolume.Probe(paths) == ArchiveAvailability.Online;
var total = (long)frames.Count * (makesProxies ? 2 : 1);
long offset = 0;
if (makesProxies)
{
if (stage == Stage.Proxies && (row.Status & SequenceStatus.ProxiesBuilt) == 0)
{
var result = await archives.Images.BuildProxiesAsync(archive, row, force: false, next,
(frame, _) => context.CheckpointAsync(Checkpoint(Stage.Proxies, frame), frame, total), ct).ConfigureAwait(false);
context.Log(Loc.L("Archive.Log.ProxiesDone", row.DisplayTitle, result.Built, result.Reused, result.Undecodable, result.Waiting));
next = 0;
}
offset = frames.Count;
}
// Frames before a copy checkpoint were copied by an earlier run.
var copied = stage == Stage.Copy ? next : 0;
(stage, next) = (Stage.Copy, 0);
var folder = paths.Sequence(row.Folder);
// Another index moved the sequence away, maybe another one in: its files are not this sequence's photos.
if (!SequenceFolders.Holds(folder, sequenceId)) throw new SequenceFolderMovedException();
// Files kept from an earlier run are this frames version's only (a conversion keeps a frame's id, not its photo).
if (checkpointVersion is { } version && version != row.FramesVersion) copied = 0;
var total = (long)frames.Count;
var tally = new Tally();
var entries = new WorkspaceFrame[frames.Count];
Directory.CreateDirectory(project.ProxiesDirectory);
@@ -101,25 +101,32 @@ internal sealed class WorkspaceCopyJobHandler(ChronosDatabase db, ArchiveService
entries[i] = Copy(project, folder, paths, frames[i], policy, done: i < copied, tally, ct);
if (clock.Elapsed >= CheckpointEvery)
{
await context.CheckpointAsync(Checkpoint(Stage.Copy, i + 1), offset + i + 1, total).ConfigureAwait(false);
await context.CheckpointAsync(Checkpoint(Stage.Copy, i + 1, row.FramesVersion), i + 1, total).ConfigureAwait(false);
clock.Restart();
}
else
{
await context.ReportAsync(offset + i + 1, total).ConfigureAwait(false);
await context.ReportAsync(i + 1, total).ConfigureAwait(false);
}
}
var analysis = CopyAnalysis(folder, project, frames.Count);
var after = await db.Sequences.GetAsync(sequenceId, ct).ConfigureAwait(false)
?? throw new InvalidOperationException(Loc.L("Archive.Log.SequenceMissing", sequenceId));
if (after.FramesVersion != row.FramesVersion && pass < Passes)
var (after, now) = await FramesAsync(sequenceId, ct).ConfigureAwait(false);
var moved = after.Folder != row.Folder || !SequenceFolders.Holds(paths.Sequence(after.Folder), sequenceId) || Renamed(frames, now, entries);
if (after.FramesVersion != row.FramesVersion || moved)
{
context.Log(Loc.L("Archive.Log.WorkspaceFramesChanged", row.DisplayTitle));
(stage, next) = (Stage.Proxies, 0);
continue;
// The files read may be of other frames, or missing where a Reorganize took them: read them again.
if (pass < Passes)
{
context.Log(Loc.L("Archive.Log.WorkspaceFramesChanged", row.DisplayTitle));
(copied, checkpointVersion) = (0, null);
continue;
}
// Frames still arriving are copied as they were read (Update media takes the rest); a folder still moving is not.
if (moved) throw new SequenceFolderMovedException();
}
var aliases = await HomePhotos.AliasesAsync(db, sequenceId, ct).ConfigureAwait(false);
var media = new WorkspaceMedia(WorkspaceMedia.FormatName, WorkspaceMedia.CurrentFormatVersion, BundleMedia.Proxies, sequenceId, row.FramesVersion, entries,
DateTimeOffset.UtcNow, analysis);
DateTimeOffset.UtcNow, analysis, aliases.Count > 0 ? aliases : null);
// The working state is the editor's meanwhile: it is held only while the media index and the positions move.
using (await project.LockAsync(LockWait, ct).ConfigureAwait(false))
{
@@ -127,11 +134,42 @@ internal sealed class WorkspaceCopyJobHandler(ChronosDatabase db, ArchiveService
RemoveStale(project, media);
}
context.Log(Loc.L("Archive.Log.WorkspaceCopied", row.DisplayTitle, tally.Copied, tally.Made, tally.Kept, tally.Missing));
await context.CheckpointAsync(Checkpoint(Stage.Done, frames.Count), total, total).ConfigureAwait(false);
await context.CheckpointAsync(Checkpoint(Stage.Done, frames.Count, row.FramesVersion), total, total).ConfigureAwait(false);
return;
}
}
/// <summary>Whether <paramref name="folder"/> is an archive's root of this index or lies inside one.</summary>
private async Task<bool> InsideALibraryAsync(string folder, CancellationToken cancellationToken)
{
var path = Plain(Path.GetFullPath(folder));
foreach (var archive in await db.Archives.ListAsync(cancellationToken).ConfigureAwait(false))
{
if (DataLocation.IsUnder(path, Plain(Path.GetFullPath(archive.RootPath)))) return true;
}
return false;
// Without the long-path prefix, which only one of the two may carry.
static string Plain(string path) =>
path.StartsWith(@"\\?\UNC\", StringComparison.Ordinal) ? @"\\" + path[8..]
: path.StartsWith(@"\\?\", StringComparison.Ordinal) ? path[4..]
: path;
}
/// <summary>
/// A frame the copy found no file for whose original the index has under another name now (a Reorganize renamed the
/// files during the copy, keeping the frames version): the copy runs again.
/// </summary>
private static bool Renamed(List<Frame> read, List<Frame> now, WorkspaceFrame[] entries)
{
var paths = now.ToDictionary(f => f.Id, f => f.RelativePath);
for (var i = 0; i < read.Count; i++)
{
if (entries[i].File is null && paths.TryGetValue(read[i].Id, out var path) && path != read[i].RelativePath) return true;
}
return false;
}
/// <summary>
/// The frame's file in the workspace: kept when an earlier run copied it (before the checkpoint, or the same bytes as the
/// archive's proxy), else the archive's current proxy, else one made from the original; none for a frame without either.
@@ -219,7 +257,9 @@ internal sealed class WorkspaceCopyJobHandler(ChronosDatabase db, ArchiveService
/// <summary>
/// The media index and the working state moved onto it: first the working state with every position parked by photo
/// (valid whatever the frames), then the index, then the working state placed on the new frames. A crash in between
/// leaves positions that wait for their photos, which the next open places.
/// leaves positions that wait for their photos, which the next open places. A frame the new media still hold is parked
/// as the photo they hold for it, so one whose file a conversion replaced (the same frame, another SHA-256) keeps its
/// positions, as the archive's own settings do (<see cref="ProjectPositions"/>).
/// </summary>
private static void Commit(WorkspaceProject project, ProjectBundle bundle, WorkspaceMedia media, Sequence row)
{
@@ -230,7 +270,9 @@ internal sealed class WorkspaceCopyJobHandler(ChronosDatabase db, ArchiveService
// Without the media index the positions were made against, they are taken as naming these frames.
if (!working.IsNewerFormat && old is not null)
{
var portable = ProjectSettings.FromJson(ProjectContent.ToPortable(working, old.Photos), allowNewer: true);
var held = media.Frames.ToDictionary(f => f.Id);
var before = new FramePhotos(old.Frames.Select(f => (f.Index, Photo(held.TryGetValue(f.Id, out var now) ? now : f))), old.Aliases);
var portable = ProjectSettings.FromJson(ProjectContent.ToPortable(working, before), allowNewer: true);
project.WriteBundle(bundle with { Project = portable, Workspace = workspace, Sequence = sequence });
project.WriteMedia(media);
working = ProjectSettings.FromJson(ProjectContent.Place(portable.ToJson(), media.Photos), allowNewer: true);
@@ -242,6 +284,8 @@ internal sealed class WorkspaceCopyJobHandler(ChronosDatabase db, ArchiveService
project.WriteBundle(bundle with { Project = working, Workspace = workspace, Sequence = sequence, Saved = DateTimeOffset.UtcNow });
}
private static FramePhoto Photo(WorkspaceFrame frame) => new(frame.Sha256, frame.Size, frame.CapturedAt);
/// <summary>Proxies of frames the sequence no longer has: regenerable copies, removed so the media folder holds the frames listed.</summary>
private static void RemoveStale(WorkspaceProject project, WorkspaceMedia media)
{
@@ -313,21 +357,26 @@ internal sealed class WorkspaceCopyJobHandler(ChronosDatabase db, ArchiveService
private enum Stage
{
Proxies,
Copy,
Done,
}
private static string Checkpoint(Stage stage, int frame) =>
string.Create(CultureInfo.InvariantCulture, $"{stage.ToString().ToLowerInvariant()}:{frame}");
private static string Checkpoint(Stage stage, int frame, int framesVersion) =>
string.Create(CultureInfo.InvariantCulture, $"{stage.ToString().ToLowerInvariant()}:{frame}:{framesVersion}");
private static (Stage Stage, int Next) Parse(string? checkpoint)
/// <summary>
/// The frames copied by an earlier run (<c>copy:&lt;frame&gt;[:&lt;frames version&gt;]</c>) and the frames version they
/// were of (null when the checkpoint does not say); nothing for a finished run, or another stage (an earlier build's
/// archive proxy pass), whose files the copy checks again.
/// </summary>
private static (int Copied, int? FramesVersion) Parse(string? checkpoint)
{
var parts = checkpoint?.Split(':');
return parts is [var name, var number] && Enum.TryParse<Stage>(name, ignoreCase: true, out var stage)
&& int.TryParse(number, NumberStyles.None, CultureInfo.InvariantCulture, out var next)
? (stage == Stage.Done ? Stage.Copy : stage, stage == Stage.Done ? 0 : next)
: (Stage.Proxies, 0);
if (parts is not ([_, _] or [_, _, _]) || !Enum.TryParse<Stage>(parts[0], ignoreCase: true, out var stage) || stage != Stage.Copy
|| !int.TryParse(parts[1], NumberStyles.None, CultureInfo.InvariantCulture, out var next))
return (0, null);
int? version = parts.Length == 3 && int.TryParse(parts[2], NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture, out var v) ? v : null;
return (next, version);
}
private sealed class Tally
@@ -90,15 +90,17 @@ public sealed record WorkspaceFrame(
/// (<see cref="BundleMedia.Proxies"/>; originals in phase 5c), the sequence and its frames version at the home when they
/// were copied, and every frame in index order. Written last by a copy, so a copy cut short leaves the previous index.
/// </summary>
/// <param name="Aliases">The encodings of the sequence's photos a conversion joined at the home (R7, its tombstones of reason
/// "replaced") when the media were copied: a version parked on a photo before its file was converted still finds its frame.</param>
public sealed record WorkspaceMedia(string Format, int FormatVersion, string Kind, Guid SequenceId, int FramesVersion, IReadOnlyList<WorkspaceFrame> Frames,
DateTimeOffset Copied, string? AnalysisSha256 = null)
DateTimeOffset Copied, string? AnalysisSha256 = null, IReadOnlyList<PhotoAlias>? Aliases = null)
{
public const string FormatName = "chronos-workspace-media";
public const int CurrentFormatVersion = 1;
/// <summary>The frames' photos in index order, as project positions name them.</summary>
/// <summary>The frames' photos in index order, as project positions name them, with the encodings conversions joined.</summary>
[JsonIgnore]
public FramePhotos Photos => new(Frames.Select(f => (f.Index, new FramePhoto(f.Sha256, f.Size, f.CapturedAt))));
public FramePhotos Photos => new(Frames.Select(f => (f.Index, new FramePhoto(f.Sha256, f.Size, f.CapturedAt))), Aliases);
/// <summary>The frames held, as the processing engine reads them.</summary>
public IReadOnlyList<Frame> ToFrames() => [.. Frames.Select(f => f.ToFrame(SequenceId))];
+21 -6
View File
@@ -359,13 +359,15 @@ public sealed class WorkspaceSync(WorkspaceStore store, string clientName)
locals = [.. locals.Skip(ours.Count)];
var media = project.ReadMedia();
var photos = media?.Photos ?? NoFrames;
var basisSettings = basis is null ? new ProjectSettings() : ProjectMerge.Read(basis.SettingsJson);
var headSettings = ProjectMerge.Read(head.SettingsJson);
var tip = tipVersion is null ? basisSettings : ProjectMerge.Read(tipVersion.SettingsJson);
// Each side names a photo as the frames held it when it was saved: named as the media hold it now (a conversion or a
// hash since), the same photo merges as one.
var basisSettings = basis is null ? new ProjectSettings() : ProjectMerge.Read(ProjectContent.Rebind(basis.SettingsJson, photos));
var headSettings = ProjectMerge.Read(ProjectContent.Rebind(head.SettingsJson, photos));
var tip = tipVersion is null ? basisSettings : ProjectMerge.Read(ProjectContent.Rebind(tipVersion.SettingsJson, photos));
var working = ProjectMerge.Read(Portable(bundle.Project, media) ?? throw MediaMissing(project.ProjectId));
// "Take the server's" everywhere: the paths are those the merge finds, so it runs once to find them.
var choice = takeTheirs is EveryPath ? null : takeTheirs;
List<ProjectSettings> mine = [.. locals.Select(v => ProjectMerge.Read(v.SettingsJson))];
List<ProjectSettings> mine = [.. locals.Select(v => ProjectMerge.Read(ProjectContent.Rebind(v.SettingsJson, photos)))];
var replay = ProjectMerge.Replay(basisSettings, mine, headSettings, choice);
if (takeTheirs is EveryPath) replay = ProjectMerge.Replay(basisSettings, mine, headSettings, Paths(replay.Conflicts));
var newTip = replay.Commits.Count > 0 ? replay.Commits[^1] : headSettings;
@@ -469,7 +471,7 @@ public sealed class WorkspaceSync(WorkspaceStore store, string clientName)
var (tipId, tipNumber, tipHash) = locals.Count > 0
? (locals[^1].Id, locals[^1].Number, locals[^1].ContentHash)
: (workspace.Base?.VersionId, workspace.Base?.Number ?? 0, workspace.Base?.ContentHash);
if (hash == tipHash) return null;
if (hash == tipHash || SameAsTip(project, bundle, locals, portable, media)) return null;
var sync = project.ReadSync();
var version = new ProjectVersion
{
@@ -509,7 +511,8 @@ public sealed class WorkspaceSync(WorkspaceStore store, string clientName)
var conflict = project.ReadSync().Conflict;
var locals = Unpushed(project, bundle);
var tipHash = locals.Count > 0 ? locals[^1].ContentHash : workspace.Base?.ContentHash;
var modified = !bundle.IsReadOnly && (Portable(bundle.Project, media) is not { } portable || ProjectContent.Hash(portable) != tipHash);
var modified = !bundle.IsReadOnly && (Portable(bundle.Project, media) is not { } portable
|| (ProjectContent.Hash(portable) != tipHash && !SameAsTip(project, bundle, locals, portable, media)));
ProjectVersion? head = null;
var offline = false;
if (home is not null)
@@ -536,6 +539,18 @@ public sealed class WorkspaceSync(WorkspaceStore store, string clientName)
return new WorkspaceStatus(state, modified, locals.Count, behind, offline, media is null, bundle.IsReadOnly, conflict, workspace.Base, head);
}
/// <summary>
/// Whether the working state's portable form holds what the last version does (the last local one, else the base),
/// once each names its photos as the media hold them now: a version saved before the media were copied again after a
/// conversion or a hash at the home parks the same photos under their former names.
/// </summary>
private static bool SameAsTip(WorkspaceProject project, ProjectBundle bundle, List<ProjectVersion> locals, string portable, WorkspaceMedia? media)
{
if (media is null) return false;
var tip = locals.Count > 0 ? locals[^1] : bundle.Workspace!.Base is { } basis ? project.Versions().FirstOrDefault(v => v.Id == basis.VersionId) : null;
return tip is not null && ProjectContent.SameChoices(tip.SettingsJson, portable, media.Photos);
}
/// <summary>The local versions after the base, oldest first, each the child of the one before (the base's for the first).</summary>
private static List<ProjectVersion> Unpushed(WorkspaceProject project, ProjectBundle bundle)
{
@@ -90,6 +90,51 @@ public sealed class ProjectContentTests
Assert.SequenceEqual([2], ProjectSettings.FromJson(ProjectContent.Place(portable, hashed)).ExcludedFrames);
}
[Test]
public void APhotoParkedOnAnotherEncodingFindsTheFrameHoldingItsReplacement()
{
// Owner review 2, R7: a conversion replaced photo 2 by 20 and a later one 20 by 21; the undo of another gave 3 back for 30.
PhotoAlias[] links = [new(Photo(20).Sha256!, Photo(21).Sha256!), new(Photo(2).Sha256!.ToUpperInvariant(), Photo(20).Sha256!), new(Photo(30).Sha256!, Photo(3).Sha256!)];
var converted = new FramePhotos([(0, Photo(1)), (1, Photo(21)), (2, Photo(3))], links);
var portable = ProjectContent.ToPortable(new ProjectSettings { ExcludedFrames = [1], HeroFrame = 2 }, Frames(Photo(1), Photo(2), Photo(30)));
var placed = ProjectSettings.FromJson(ProjectContent.Place(portable, converted));
Assert.SequenceEqual([1], placed.ExcludedFrames);
Assert.Equal(2, placed.HeroFrame);
Assert.Null(placed.PendingFrames);
// The links joined in any order, either way: every encoding is the same frame.
var reversed = new FramePhotos([(0, Photo(1)), (1, Photo(21)), (2, Photo(3))], links.Reverse().Select(l => new PhotoAlias(l.Replacement, l.Sha256)));
foreach (var photos in new[] { converted, reversed })
Assert.SequenceEqual(new int?[] { 1, 1, 1, 2, 2 }, new[] { Photo(2), Photo(20), Photo(21), Photo(3), Photo(30) }.Select(p => photos.IndexOf(p)));
Assert.Null(converted.IndexOf(Photo(4)));
// Without the links the photos are not there: the positions wait for them.
var waiting = ProjectSettings.FromJson(ProjectContent.Place(portable, Frames(Photo(1), Photo(21), Photo(3))));
Assert.Null(waiting.HeroFrame);
Assert.Equal(2, waiting.PendingFrames!.Count);
}
[Test]
public void ChoicesSavedOnPhotosAsTheyWereReadAsTheSameOnceRenamed()
{
// Saved while photo 1 was not hashed and photo 2 not converted yet; since, 1 was hashed and 2 converted into 20.
var before = Frames(Photo(1, hashed: false), Photo(2), Photo(3));
var now = new FramePhotos([(0, Photo(1)), (1, Photo(20)), (2, Photo(3))], [new PhotoAlias(Photo(2).Sha256!, Photo(20).Sha256!)]);
var saved = ProjectContent.ToPortable(new ProjectSettings { ExcludedFrames = [0, 1] }, before);
var current = ProjectContent.ToPortable(new ProjectSettings { ExcludedFrames = [0, 1] }, now);
Assert.NotEqual(saved, current);
Assert.Equal(current, ProjectContent.Rebind(saved, now));
Assert.True(ProjectContent.SameChoices(saved, current, now));
Assert.False(ProjectContent.SameChoices(saved, ProjectContent.ToPortable(new ProjectSettings { ExcludedFrames = [1] }, now), now));
// Named as the frames hold them already, or held by no frame: the text as it is.
Assert.Equal(current, ProjectContent.Rebind(current, now));
Assert.Equal(saved, ProjectContent.Rebind(saved, Frames(Photo(7))));
Assert.False(ProjectContent.SameChoices(saved, current, Frames(Photo(7))));
// A newer Chronos's settings are its own to rename.
var newer = """{ "version": 2, "pendingFrames": [ { "size": 1001, "capturedAt": "2026-09-28T18:00:10", "excluded": true } ] }""";
Assert.Equal(newer, ProjectContent.Rebind(newer, now));
}
[Test]
public void TheSameSettingsGiveTheSameTextAndHashWhateverTheFrames()
{
@@ -33,6 +33,34 @@ public abstract class ProjectServiceTests(TestContext context) : DatabaseTest(co
internal static FramePhotos Photos(IEnumerable<Frame> frames) => new(frames.Select(f => (f.Index, new FramePhoto(f.Sha256, f.Size, f.CapturedAt))));
/// <summary>
/// The files of the frames at <paramref name="indices"/> replaced by other encodings, as a Replace conversion does (owner
/// review 2, R7, <see cref="FrameTrash.ReplaceAsync"/>): the same frames, other files and SHA-256.
/// </summary>
internal static async Task ConvertAsync(ArchiveFixture fx, Guid sequenceId, IReadOnlyList<int> indices, int seed)
{
var sequence = (await fx.Db.Sequences.GetAsync(sequenceId))!;
var manifest = ManifestStore.ReadSequence(fx.Paths.Sequence(sequence.Folder).ManifestPath);
var staging = Path.Combine(fx.Paths.StagingDirectory, "convert-test");
Directory.CreateDirectory(staging);
var swaps = new List<FrameSwap>();
foreach (var i in indices)
{
var frame = manifest.Frames[i];
var bytes = Chronos.Data.Tests.Archiving.Photos.Jpeg(frame.CapturedAt, seed + i);
var path = Path.Combine(staging, $"{Guid.NewGuid():N}.jpeg");
File.WriteAllBytes(path, bytes);
var sha = Convert.ToHexStringLower(System.Security.Cryptography.SHA256.HashData(bytes));
swaps.Add(new FrameSwap(frame.Id, fx.Paths.ToRelative(path), [], (current, _) => Path.ChangeExtension(current.File, ".jpeg"), current => current with
{
Sha256 = sha,
Size = bytes.Length,
Source = new FrameSource { Sha256 = current.Sha256, Format = current.Format, File = current.File, Size = current.Size },
}));
}
Assert.NotNull(await fx.Service.FrameTrash.ReplaceAsync(fx.Archive, sequenceId, null, swaps, null, null, null, null, fx.Ct));
}
/// <summary>The sequence's settings written as an older code path writes them (the editor, an analysis): index first, with its version check.</summary>
internal static async Task<Sequence> WriteSettingsAsync(ArchiveFixture fx, Guid sequenceId, ProjectSettings settings)
{
@@ -278,6 +306,34 @@ public abstract class ProjectServiceTests(TestContext context) : DatabaseTest(co
Assert.Equal(mainPush.Id, (await fx.Db.Projects.GetAsync(main.Id))!.MirroredVersionId);
}
[Test]
public async Task AProjectTheIndexDoesNotKnowYetIsFoundInItsSidecarsThroughItsHome()
{
var fx = await FixtureAsync();
var (sequence, frames) = await UploadedAsync(fx, 3);
var vertical = await fx.Service.Projects.CreateAsync(sequence.Id, "Vertical", cancellationToken: fx.Ct);
var head = (await fx.Service.Projects.PushAsync(vertical.Id,
Draft(await fx.Service.Projects.HeadAsync(vertical.Id, fx.Ct), new ProjectSettings { ExcludedFrames = [1] }, frames), fx.Ct)).Version!;
// An index taken back from an older backup (or a sequence back from the trash) holds the sequence, not its projects yet:
// a workspace that has one checked out still finds it, and its versions.
await fx.ReopenWithEmptyIndexAsync();
await fx.ScanAsync();
Assert.Null(await fx.Db.Projects.GetAsync(vertical.Id));
var version = await new ArchiveProjectHome(fx.Db, fx.Service.Projects, sequence.Id).GetVersionAsync(vertical.Id, head.Id, fx.Ct);
Assert.Equal(head.ContentHash, Assert.NotNull(version).ContentHash);
await fx.ReopenWithEmptyIndexAsync();
await fx.ScanAsync();
var home = new ArchiveProjectHome(fx.Db, fx.Service.Projects, sequence.Id);
Assert.Equal(head.Id, (await home.HeadAsync(vertical.Id, fx.Ct)).Id);
// A project of another sequence is still none of this home's.
var other = (await UploadedAsync(fx, 1, seed: 300)).Sequence;
var square = await fx.Service.Projects.CreateAsync(other.Id, "Square", cancellationToken: fx.Ct);
Assert.Equal(ProjectError.NotFound, (await Assert.ThrowsAsync<ProjectException>(() => home.HeadAsync(square.Id, fx.Ct))).Reason);
Assert.Null(await home.GetVersionAsync(square.Id, head.Id, fx.Ct));
}
[Test]
public async Task AVersionFileAPushLeftBehindItsRecordIsAdoptedAndAStrayOneSetAside()
{
@@ -327,14 +383,75 @@ public abstract class ProjectServiceTests(TestContext context) : DatabaseTest(co
var (sequence, frames) = await UploadedAsync(fx, 3);
var projects = fx.Service.Projects;
var main = await projects.MainAsync(sequence.Id, fx.Ct);
var moved = "2026/Moved " + Guid.NewGuid().ToString("N")[..6];
Directory.CreateDirectory(Path.GetDirectoryName(fx.Paths.Sequence(moved).Folder)!);
Directory.Move(fx.Paths.Sequence(sequence.Folder).Folder, fx.Paths.Sequence(moved).Folder);
await fx.Db.Sequences.RelocateAsync(sequence.Id, moved, sequence.Title, fx.Ct);
var vertical = await projects.CreateAsync(sequence.Id, "Vertical", cancellationToken: fx.Ct);
// A Reorganize (phase 4a) moves the whole folder under the sequence's lease: the projects' sidecars go with it.
var plan = await fx.Service.Renamer.PlanAsync(fx.Archive, [sequence.Id], "Moved/{title}", null, fx.Ct);
await fx.Service.Renamer.ApplyAsync(fx.Archive, plan, cancellationToken: fx.Ct);
var reorganized = (await fx.Db.Sequences.GetAsync(sequence.Id))!;
Assert.NotEqual(sequence.Folder, reorganized.Folder);
Assert.FileExists(fx.Paths.Sequence(reorganized.Folder).ProjectRecordPath(vertical.Id));
Assert.SequenceEqual([main.Id, vertical.Id], (await projects.ListAsync(sequence.Id, fx.Ct)).Select(p => p.Id));
var pushed = await projects.PushAsync(main.Id, Draft(await projects.HeadAsync(main.Id, fx.Ct), new ProjectSettings { ExcludedFrames = [1] }, frames), fx.Ct);
Assert.Equal(PushOutcome.Stored, pushed.Outcome);
Assert.FileExists(fx.Paths.Sequence(moved).VersionPath(main.Id, 2, pushed.Version!.Id));
Assert.FileExists(fx.Paths.Sequence(reorganized.Folder).VersionPath(main.Id, 2, pushed.Version!.Id));
// Another index of the archive moved it away: until this index finds it there, its projects are read from the index (none
// taken for gone) and nothing is written where it was, which is never made again. A workspace hears its home is out of
// reach for now, and reads its head from the index meanwhile.
var away = "2026/Away " + Guid.NewGuid().ToString("N")[..6];
Directory.CreateDirectory(Path.GetDirectoryName(fx.Paths.Sequence(away).Folder)!);
Directory.Move(fx.Paths.Sequence(reorganized.Folder).Folder, fx.Paths.Sequence(away).Folder);
Assert.SequenceEqual([main.Id, vertical.Id], (await projects.ListAsync(sequence.Id, fx.Ct)).Select(p => p.Id));
var head = await projects.HeadAsync(main.Id, fx.Ct);
await Assert.ThrowsAsync<SequenceFolderMovedException>(() => projects.PushAsync(main.Id, Draft(head, new ProjectSettings { ExcludedFrames = [2] }, frames), fx.Ct));
var home = new ArchiveProjectHome(fx.Db, projects, sequence.Id);
await Assert.ThrowsAsync<HomeUnavailableException>(() => home.PushAsync(main.Id, Draft(head, new ProjectSettings { ExcludedFrames = [2] }, frames), fx.Ct));
await Assert.ThrowsAsync<HomeUnavailableException>(() => home.CreateAsync("Square", null, null, fx.Ct));
Assert.Equal(head.Id, (await home.HeadAsync(main.Id, fx.Ct)).Id);
Assert.False(Directory.Exists(fx.Paths.Sequence(reorganized.Folder).Folder));
// Another sequence moved into the folder it left is no home of its projects either: none is read from it or written there.
var other = (await UploadedAsync(fx, 1, seed: 300)).Sequence;
Directory.Move(fx.Paths.Sequence(other.Folder).Folder, fx.Paths.Sequence(reorganized.Folder).Folder);
await Assert.ThrowsAsync<SequenceFolderMovedException>(() => projects.RenameAsync(vertical.Id, "Vertical 2", fx.Ct));
Assert.False(Directory.Exists(fx.Paths.Sequence(reorganized.Folder).ProjectsDirectory));
Assert.SequenceEqual([main.Id, vertical.Id], (await projects.ListAsync(sequence.Id, fx.Ct)).Select(p => p.Id));
await fx.Db.Sequences.RelocateAsync(sequence.Id, away, reorganized.Title, fx.Ct);
Assert.Equal(PushOutcome.Stored, (await projects.PushAsync(main.Id, Draft(head, new ProjectSettings { ExcludedFrames = [2] }, frames), fx.Ct)).Outcome);
Assert.Null((await projects.GetAsync(vertical.Id, fx.Ct))!.DeletedAt);
}
[Test]
public async Task AVersionKeepsItsPositionsWhenItsPhotosAreConverted()
{
var fx = await FixtureAsync();
var (sequence, frames) = await UploadedAsync(fx, 4);
var projects = fx.Service.Projects;
await WriteSettingsAsync(fx, sequence.Id, new ProjectSettings { ExcludedFrames = [1], HeroFrame = 2 });
var main = await projects.MainAsync(sequence.Id, fx.Ct);
var head = await projects.HeadAsync(main.Id, fx.Ct);
// A Replace conversion (phase 4b) gives frames 1 and 2 other files: the same frames, other SHA-256.
await ConvertAsync(fx, sequence.Id, [1, 2], 700);
var home = await projects.FramesAsync(sequence.Id, fx.Ct);
Assert.NotEqual(frames[1].Sha256, home.Frames[1].Sha256);
Assert.Equal(frames[1].Id, home.Frames[1].Id);
// The version parked the photos as they were: it still names the converted frames.
var placed = ProjectSettings.FromJson(ProjectContent.Place(head.SettingsJson, home.Photos));
Assert.SequenceEqual([1], placed.ExcludedFrames);
Assert.Equal((int?)2, placed.HeroFrame);
// The sequence's settings written again with the same choices are no outside change: Main keeps its head.
var row = (await fx.Db.Sequences.GetAsync(sequence.Id))!;
await WriteSettingsAsync(fx, sequence.Id, ProjectSettings.FromJson(row.SettingsJson!));
Assert.Equal(head.Id, (await projects.HeadAsync(main.Id, fx.Ct)).Id);
// A version pushed on top of it is mirrored onto the converted frames.
var pushed = await projects.PushAsync(main.Id, Draft(head, new ProjectSettings { ExcludedFrames = [1, 3], HeroFrame = 2 }, home.Frames), fx.Ct);
Assert.Equal(PushOutcome.Stored, pushed.Outcome);
var mirrored = ProjectSettings.FromJson((await fx.Db.Sequences.GetAsync(sequence.Id))!.SettingsJson!);
Assert.SequenceEqual([1, 3], mirrored.ExcludedFrames);
Assert.Equal((int?)2, mirrored.HeroFrame);
}
}
@@ -54,10 +54,14 @@ internal sealed class WorkspaceFixture : IAsyncDisposable
}
/// <summary>Runs a workspace copy job to its end.</summary>
public async Task<Job> CopyAsync(WorkspaceStore store, Guid projectId, JobState expected = JobState.Done, string? checkpoint = null, string media = "proxies")
public Task<Job> CopyAsync(WorkspaceStore store, Guid projectId, JobState expected = JobState.Done, string? checkpoint = null, string media = "proxies") =>
CopyAsync(store.Root, projectId, expected, checkpoint, media);
/// <summary>Runs a workspace copy job into the workspace at <paramref name="root"/> to its end.</summary>
public async Task<Job> CopyAsync(string root, Guid projectId, JobState expected = JobState.Done, string? checkpoint = null, string media = "proxies")
{
var engine = await EngineAsync();
var request = WorkspaceJobs.Copy(Sequence, store.Root, projectId, media);
var request = WorkspaceJobs.Copy(Sequence, root, projectId, media);
var job = checkpoint is null ? await engine.EnqueueAsync(request, cancellationToken: Ct) : await Archive.EnqueueWithCheckpointAsync(engine, request, checkpoint);
return await Archive.WaitForAsync(job.Id, expected, (int)TestTiming.Budget(60));
}
@@ -79,17 +79,27 @@ public abstract class WorkspaceTests(TestContext context) : DatabaseTest(context
Assert.Equal(WorkspaceState.MediaRemoved, (await sync.StatusAsync(main.Id, fx.Home, fx.Ct)).State);
Assert.Null((await sync.OpenAsync(main.Id, fx.Ct)).Sequence);
// The archive has no proxies yet: the copy makes them in the workspace and writes nothing into the archive.
var folder = fx.Archive.Paths.Sequence(fx.Sequence.Folder);
var job = await fx.CopyAsync(store, main.Id);
Assert.Contains(Loc.L("Archive.Log.WorkspaceCopied", fx.Sequence.DisplayTitle, 5, 0, 0, 0), job.LogTail!);
Assert.Contains(Loc.L("Archive.Log.WorkspaceCopied", fx.Sequence.DisplayTitle, 0, 5, 0, 0), job.LogTail!);
Assert.False(fx.Frames.Any(f => File.Exists(folder.Proxy(f.Index))));
var media = Assert.NotNull(project.ReadMedia());
Assert.Equal((BundleMedia.Proxies, fx.Sequence.Id, fx.Frames.Count), (media.Kind, media.SequenceId, media.Frames.Count));
foreach (var (frame, entry) in fx.Frames.Zip(media.Frames))
{
Assert.Equal((frame.Id, frame.Sha256), (entry.Id, entry.Sha256));
var file = project.MediaFile(entry.File!);
Assert.Equal(entry.FileSha256, Sha(file));
// The archive's own proxy, copied byte for byte.
Assert.Equal(Sha(fx.Archive.Paths.Sequence(fx.Sequence.Folder).Proxy(frame.Index)), entry.FileSha256);
Assert.Equal(entry.FileSha256, Sha(project.MediaFile(entry.File!)));
}
// Once the archive's own pass made its proxies, those are what the workspace holds, byte for byte.
await fx.Archive.Service.Images.BuildProxiesAsync(fx.Archive.Archive, fx.Sequence, force: false, 0, (_, _) => ValueTask.CompletedTask, fx.Ct);
await fx.CopyAsync(store, main.Id);
media = Assert.NotNull(project.ReadMedia());
foreach (var (frame, entry) in fx.Frames.Zip(media.Frames))
{
Assert.Equal(entry.FileSha256, Sha(project.MediaFile(entry.File!)));
Assert.Equal(Sha(folder.Proxy(frame.Index)), entry.FileSha256);
}
var session = await sync.OpenAsync(main.Id, fx.Ct);
var opened = Assert.NotNull(session.Sequence);
@@ -116,6 +126,8 @@ public abstract class WorkspaceTests(TestContext context) : DatabaseTest(context
var (store, sync) = fx.Workspace();
var main = await fx.Archive.Service.Projects.MainAsync(fx.Sequence.Id, fx.Ct);
var project = await sync.CheckoutAsync(fx.Home, main.Id, cancellationToken: fx.Ct);
// The archive's own proxies, which the copy takes as they are.
await fx.Archive.Service.Images.BuildProxiesAsync(fx.Archive.Archive, fx.Sequence, force: false, 0, (_, _) => ValueTask.CompletedTask, fx.Ct);
await fx.CopyAsync(store, main.Id);
var copied = project.ReadMedia()!;
@@ -127,6 +139,56 @@ public abstract class WorkspaceTests(TestContext context) : DatabaseTest(context
var media = Assert.NotNull(project.ReadMedia());
Assert.SequenceEqual(copied.Frames.Select(f => f.FileSha256), media.Frames.Select(f => f.FileSha256));
Assert.Equal(copied.Frames[3].FileSha256, Sha(project.MediaFile(media.Frames[3].File!)));
// A run stopped on other frames (a conversion keeps a frame's id, not its photo) leaves no file unchecked.
File.WriteAllText(project.MediaFile(copied.Frames[0].File!), "not a photo either");
job = await fx.CopyAsync(store, main.Id, checkpoint: $"copy:{copied.Frames.Count}:{fx.Sequence.FramesVersion - 1}");
Assert.Contains(Loc.L("Archive.Log.WorkspaceCopied", fx.Sequence.DisplayTitle, 1, 0, 4, 0), job.LogTail!);
Assert.Equal(copied.Frames[0].FileSha256, Sha(project.MediaFile(Assert.NotNull(project.ReadMedia()).Frames[0].File!)));
}
[Test]
public async Task AWorkspaceInsideALibraryGetsNothingCopied()
{
var fx = await FixtureAsync(2);
var main = await fx.Archive.Service.Projects.MainAsync(fx.Sequence.Id, fx.Ct);
var inside = Path.Combine(fx.Archive.Paths.Root, "workspace");
var job = await fx.CopyAsync(inside, main.Id, JobState.Failed);
Assert.Equal(Loc.L("Errors.Workspace.InsideLibrary"), job.Error);
Assert.False(Directory.Exists(inside));
}
[Test]
public async Task PhotosConvertedAtTheHomeKeepTheirPositionsInTheWorkspace()
{
var fx = await FixtureAsync(4);
var (store, sync) = fx.Workspace();
var id = (await fx.CheckoutAsync(sync)).ProjectId;
await EditAsync(sync, id, s => s with { ExcludedFrames = [1], HeroFrame = 2 }, fx.Ct);
var saved = Assert.NotNull(await sync.CommitAsync(id, cancellationToken: fx.Ct));
await sync.PushAsync(id, fx.Home, fx.Ct);
await EditAsync(sync, id, s => s with { ExcludedFrames = [] }, fx.Ct);
await sync.CommitAsync(id, cancellationToken: fx.Ct);
await sync.PushAsync(id, fx.Home, fx.Ct);
await EditAsync(sync, id, s => s with { ExcludedFrames = [1] }, fx.Ct);
// A Replace conversion gives frames 1 and 2 other files (the same frames); Update media copies them again.
await ProjectServiceTests.ConvertAsync(fx.Archive, fx.Sequence.Id, [1, 2], 700);
fx.Sequence = (await fx.Db.Sequences.GetAsync(fx.Sequence.Id))!;
await fx.CopyAsync(store, id);
var session = await sync.OpenAsync(id, fx.Ct);
Assert.NotEqual(fx.Frames[1].Sha256, session.Media!.Frames[1].Sha256);
Assert.SequenceEqual([1], session.Working.ExcludedFrames);
Assert.Equal((int?)2, session.Working.HeroFrame);
// The versions saved before name the photos as they were: still the same choices, and they restore onto the frames.
Assert.Equal(WorkspaceState.Modified, (await sync.StatusAsync(id, fx.Home, fx.Ct)).State);
await EditAsync(sync, id, s => s with { ExcludedFrames = [] }, fx.Ct);
Assert.Equal(WorkspaceState.InSync, (await sync.StatusAsync(id, fx.Home, fx.Ct)).State);
Assert.Null(await sync.CommitAsync(id, cancellationToken: fx.Ct));
var restored = await sync.RestoreAsync(id, saved.Id, fx.Home, fx.Ct);
Assert.SequenceEqual([1], restored.ExcludedFrames);
Assert.Equal((int?)2, restored.HeroFrame);
}
[Test]