fix(reorganize): take along a sequence left inside a moving folder; keep a plan whose archive is offline

A selected sequence nested in another selected sequence's moving folder moves
out first. When the templates no longer gave it its folder (its first photo
gone, or an earlier one filed), it stayed inside, and the outer step then found
it as a stranger: StaleRenamePlanException at every retry, the plan held forever.

- A moving folder now takes along the sequences of earlier steps that stayed
  inside it (their index row still there): locked, moved and recorded with it,
  keeping their title and file names. They are reported TitlesKept instead of
  Dropped; a carried sequence whose carrier stayed is simply changed (or
  TitlesKept when it was to change more than its folder); Newer stays Newer.
  RefuseStaleAsync exempts exactly those, so a held plan either fits or is given up.
- MoveJobHandler: a journal not found on an offline archive tells nothing. A
  failed job keeps its plan:<id> checkpoint instead of becoming 'applied', and a
  retry while the archive is offline fails with "archive offline" instead of
  logging ReorganizedEarlier and ending done.
- The Fit.Names warning now says what happens to the folder (moved already,
  carried, or not moved to the planned one); stale ApplyAsync, RefuseStaleAsync,
  FitAsync and job docs corrected.

Tests: the nested case (fresh run and held retry, three levels), the same through
a held job, a carrier named otherwise with a carried sequence, Fit.Names on a
resumed step whose folder moved (held and not), and an offline retry that keeps
its plan and completes it once the archive is back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit c5ffc87f1a3900442920693763fd04d28cb43aee)
This commit is contained in:
Alby96 committed 2026-10-27 19:46:16 +01:00
1 parent 0ff5852f60
commit d0da224de5
3 files changed
+332 -50

No files matched your search

@@ -17,12 +17,15 @@ namespace Chronos.Data.Archiving.Jobs;
/// sequences kept). Paused, cancelled or failed, the job stops between two sequences and holds the plan
/// (<see cref="MassRenamer.Hold"/>), which an archive open then leaves to it (completing only file renames a step left
/// half-way); resumed or retried, it completes the rest when it still fits. A plan whose journal is gone completed
/// meanwhile (at an archive open). A job retried once its plan is over plans again, and applies only a plan that is still
/// the one confirmed. Sequences left as they were, or that kept
/// their title or file names, because they changed after the plan (one still receiving photos keeps its file names, and
/// its folder and title too when the templates now give it others), a newer Chronos wrote their sidecar, a photo was in
/// use or its new name taken, or the folder around them stayed, are logged and fail the job at the end, after the others
/// changed. Each sequence changed is told to <see cref="ArchiveServiceOptions.SequenceChanged"/> (<see cref="Change"/>)
/// meanwhile (at an archive open), unless the archive cannot be reached: then no journal tells nothing, and the job fails
/// as the archive does, keeping its checkpoint (so does a job that fails while the archive is offline and its plan's
/// journal cannot be found). A job retried once its plan is over plans again, and applies only a plan that is still
/// the one confirmed. Sequences left as they were, or that kept their title or file names, because they changed after
/// the plan (one still receiving photos keeps its file names, and its title too when the templates now give it another
/// folder or title, its folder then moving only before an interruption or with a folder around it), a newer Chronos
/// wrote their sidecar, a photo was in use or its new name taken, or the folder around them stayed, are logged and fail
/// the job at the end, after the others changed. Each sequence changed is told to
/// <see cref="ArchiveServiceOptions.SequenceChanged"/> (<see cref="Change"/>)
/// once its step is journaled, by this run or by an archive open that completed a plan an interruption left; those of a
/// step that failed half-way (its folder moved, then a sidecar could not be read) are told at once. The log counts the
/// whole plan, over every run of the job.
@@ -52,8 +55,12 @@ internal sealed class MoveJobHandler(ChronosDatabase db, MassRenamer renamer, Fu
{
if (planId is { } resumed)
{
if (RenameJournal.Read(paths, resumed) is not { } journal || await ResumeAsync(context, archive, journal).ConfigureAwait(false) is not { } done)
var journal = RenameJournal.Read(paths, resumed);
if (journal is null || await ResumeAsync(context, archive, journal).ConfigureAwait(false) is not { } done)
{
// No journal on an archive that cannot be reached tells nothing: the job fails as the archive does, and
// keeps its checkpoint for a retry.
if (ArchiveVolume.Probe(paths) == ArchiveAvailability.Offline) throw new ArchiveOfflineException(paths.Root, ArchiveAvailability.Offline);
context.Log(Loc.L("Archive.Log.ReorganizedEarlier"));
await context.ReportAsync(context.Total, context.Total).ConfigureAwait(false);
return;
@@ -134,14 +141,17 @@ internal sealed class MoveJobHandler(ChronosDatabase db, MassRenamer renamer, Fu
/// <summary>
/// Holds the plan of a failed run and logs what its completed steps did; a hold that cannot be written is logged. False
/// when the plan has no journal any more (given up as no longer matching the archive).
/// when the plan has no journal any more on an archive that can be reached (given up as no longer matching the
/// archive). On one that cannot (an unplugged drive, a dropped share), a journal not found tells nothing: the checkpoint
/// stays, so a retry resumes the plan once the archive is back.
/// </summary>
private static bool HoldFailed(JobContext context, Archive archive, Guid planId)
{
var paths = new ArchivePaths(archive.RootPath);
try
{
if (!MassRenamer.Hold(archive, planId)) return false;
if (RenameJournal.Read(new ArchivePaths(archive.RootPath), planId) is not { } journal) return true;
if (!MassRenamer.Hold(archive, planId)) return ArchiveVolume.Probe(paths) == ArchiveAvailability.Offline;
if (RenameJournal.Read(paths, planId) is not { } journal) return true;
Log.Info("archive", $"Reorganize {planId} held by its failed job {context.Job.Id}");
LogDone(context, journal.Plan, RenameResult.Of(journal.Done));
}
+119 -40
View File
@@ -31,6 +31,10 @@ public sealed class StaleRenamePlanException(Guid sequenceId)
/// names stayed because a newer Chronos wrote their sidecar since (<see cref="Newer"/>, only when resuming; their folder
/// moved, or, planned a new title, they stayed as they were when it had not moved yet), and that stayed as they were
/// because the folder around them that was to carry them stayed, for a reason of its own (<see cref="CarriedKept"/>).
/// A sequence left as it was inside a folder that a later step of the plan moves goes with that folder and keeps its title
/// and file names: it is then among the <see cref="TitlesKept"/> (still among the <see cref="Newer"/> when a newer
/// Chronos keeps them), or only among the <see cref="Changed"/> when it was to move with a folder around it that stayed
/// and nothing more of it was to change.
/// </summary>
public sealed record RenameResult(IReadOnlyList<Guid> Changed, int Files, IReadOnlyList<Guid> Dropped, IReadOnlyList<Guid> FilesKept,
IReadOnlyList<Guid> TitlesKept, IReadOnlyList<Guid> FilesBusy, IReadOnlyList<Guid> Newer, IReadOnlyList<Guid> CarriedKept)
@@ -81,7 +85,8 @@ public sealed class MassRenamer
/// <summary>
/// Its photos changed since the plan so that the templates now give it another folder or title (a photo filed before
/// the first, or the first one gone): its folder (unless it moved already), title and file names stay.
/// the first, or the first one gone): its title and file names stay, and its folder moves only when an interrupted run
/// moved it already, or with a folder around it.
/// </summary>
Names,
@@ -164,9 +169,11 @@ public sealed class MassRenamer
/// <summary>
/// Applies a plan; each completed step is journaled before the next starts. A sequence whose photos changed since the
/// plan (one still receiving them) refuses nothing: it keeps its file names (<see cref="RenameResult.FilesKept"/>), its
/// folder and title change as planned while the templates still give them from its first photo as it is now (otherwise
/// it stays as it was: <see cref="RenameResult.Dropped"/>), and the steps after it go on.
/// plan (one still receiving them) refuses nothing, and the steps after it go on: while the templates still give its
/// folder and title from its first photo as it is now, they change as planned and only its file names stay
/// (<see cref="RenameResult.FilesKept"/>); otherwise its title and file names stay too, and so does its folder unless
/// it moved already (<see cref="RenameResult.Dropped"/>, or <see cref="RenameResult.TitlesKept"/>). A sequence left as it
/// was inside a folder that a later step moves goes with that folder (<see cref="RenameResult.TitlesKept"/>).
/// </summary>
/// <exception cref="StaleRenamePlanException">A sequence is no longer where the plan expects it, a folder that moves holds
/// a sequence the plan does not take along, something took a folder the plan moves one into, or a title it changes, or
@@ -333,12 +340,16 @@ public sealed class MassRenamer
{
if (completed.Contains(step.SequenceId)) continue;
cancellationToken.ThrowIfCancellationRequested();
// A folder that moves carries the sequences inside it: they are locked, moved and recorded with it.
IReadOnlyList<RenameStep> carried = step.MovesFolder && !step.Carried
? [.. steps.Where(c => c.Carried && !completed.Contains(c.SequenceId) && Carrier(steps, c) == step)]
: [];
// A folder that moves carries the sequences inside it: they are locked, moved and recorded with it, those the plan
// carries in it and those an earlier step left inside it.
IReadOnlyList<RenameStep> carried = [], along = [];
if (step.MovesFolder && !step.Carried)
{
carried = [.. steps.Where(c => c.Carried && !completed.Contains(c.SequenceId) && Carrier(steps, c) == step)];
along = await LeftInsideAsync(steps, step, completed, cancellationToken).ConfigureAwait(false);
}
var before = result.Changed.Count;
await ApplyStepAsync(archive, paths, journal.Plan, step, carried, resuming, result, cancellationToken).ConfigureAwait(false);
await ApplyStepAsync(archive, paths, journal.Plan, step, carried, along, resuming, result, cancellationToken).ConfigureAwait(false);
completed.Add(step.SequenceId);
foreach (var c in carried) completed.Add(c.SequenceId);
journal = journal with { Completed = [.. completed], Done = result.Record() };
@@ -352,11 +363,12 @@ public sealed class MassRenamer
/// <summary>
/// A plan made before another change is refused before anything moves: a sequence moved, a title it changes or names a
/// folder or files after was edited, one was indexed inside a folder that moves (a scan adopted a subfolder copied in
/// since the dry run), something took a folder it moves one into (<see cref="TargetTaken"/>), or a sidecar it rewrites is
/// of a newer format. A step a failed run left half-way (its folder at its new place) still fits. Photos changed since
/// (a sequence still receiving them) refuse nothing: under its leases that step keeps their names. Read without the
/// leases, each step checks the same again under its own (<see cref="FitAsync"/>), so a held plan resumed by its job
/// either fits here or is given up.
/// since the dry run; a sequence of the plan an earlier step left where it was goes with the folder, <see cref="LeftInsideAsync"/>),
/// something took a folder it moves one into (<see cref="TargetTaken"/>), or a sidecar it rewrites is of a newer format.
/// A step a failed run left half-way (its folder at its new place) still fits. Photos changed since (a sequence still
/// receiving them) refuse nothing: under its leases that step keeps their file names, and its title too when the
/// templates now give it another folder or title (<see cref="Fit.Names"/>; <see cref="FitAsync"/>). Read without the
/// leases, each step checks the same again under its own, so a held plan resumed by its job either fits here or is given up.
/// </summary>
/// <exception cref="NewerFormatException">A sidecar the plan rewrites is of a newer format.</exception>
private async Task RefuseStaleAsync(Archive archive, ArchivePaths paths, RenamePlan plan, IReadOnlyList<RenameStep> remaining, CancellationToken cancellationToken)
@@ -381,9 +393,15 @@ public sealed class MassRenamer
throw new NewerFormatException(sequence.ManifestPath, manifest.SchemaVersion);
if (step.UsesTitle && !TitleStands(step, row, manifest)) throw new StaleRenamePlanException(step.SequenceId);
}
var planned = plan.Steps.Select(s => s.SequenceId).ToHashSet();
if (StrangerInside(locations, remaining.Where(s => s.MovesFolder && !s.Carried), planned) is { } stranger)
throw new StaleRenamePlanException(stranger.Id);
// Inside a folder that moves, only the sequences of its remaining steps and those an earlier step left where they were.
var pending = remaining.Select(s => s.SequenceId).ToHashSet();
var done = plan.Steps.Where(s => s.Changes && !pending.Contains(s.SequenceId)).ToDictionary(s => s.SequenceId);
foreach (var move in remaining.Where(s => s.MovesFolder && !s.Carried))
{
var stranger = StrangerInside(locations, move,
l => pending.Contains(l.Id) || (done.TryGetValue(l.Id, out var left) && IsLeftInside(move, left, l.Folder)));
if (stranger is not null) throw new StaleRenamePlanException(stranger.Id);
}
}
/// <summary>The sidecar of a sequence (read without its lease); null when it cannot be read or is another sequence's.</summary>
@@ -404,6 +422,39 @@ public sealed class MassRenamer
private static RenameStep? Carrier(IReadOnlyList<RenameStep> steps, RenameStep carried) =>
steps.Where(m => !m.Carried && m.MovesFolder && RenamePlanner.IsInside(m.FromFolder, carried.FromFolder)).MaxBy(m => m.FromFolder.Length);
/// <summary>
/// The sequences of completed steps that stayed inside the folder of the moving step <paramref name="step"/>: their own
/// folder, title or names no longer fitting the plan, or the folder that was to carry them having stayed. The index
/// still has them there (or where the folder took them, when an interrupted run moved it already). The folder takes
/// them along as it takes those it carries (<see cref="Along"/>), so no index row is left behind it, and no plan fails
/// at every retry for a sequence of its own.
/// </summary>
private async Task<IReadOnlyList<RenameStep>> LeftInsideAsync(IReadOnlyList<RenameStep> steps, RenameStep step, IReadOnlySet<Guid> completed,
CancellationToken cancellationToken)
{
var along = new List<RenameStep>();
foreach (var left in steps.Where(s => completed.Contains(s.SequenceId) && RenamePlanner.IsInside(step.FromFolder, s.FromFolder)))
{
if (await _db.Sequences.GetAsync(left.SequenceId, cancellationToken).ConfigureAwait(false) is { } row && IsLeftInside(step, left, row.Folder))
along.Add(Along(step, left));
}
return along;
}
/// <summary>
/// Whether the sequence of the completed step <paramref name="left"/>, which the index has at <paramref name="folder"/>,
/// stayed inside the folder of <paramref name="move"/>: at its own folder, or where that folder took it (<see cref="Along"/>).
/// </summary>
private static bool IsLeftInside(RenameStep move, RenameStep left, string folder) =>
RenamePlanner.IsInside(move.FromFolder, left.FromFolder) && (folder == left.FromFolder || folder == Along(move, left).ToFolder);
/// <summary>
/// How the folder of <paramref name="carrier"/> takes along the sequence of <paramref name="left"/>, which an earlier step
/// left inside it: carried, its title and file names as they are.
/// </summary>
private static RenameStep Along(RenameStep carrier, RenameStep left) =>
new(left.SequenceId, left.FromFolder, carrier.ToFolder + left.FromFolder[carrier.FromFolder.Length..], left.FromTitle, left.FromTitle, Carried: true);
/// <summary>Where a step's sequence is now: its folder moves first, so a resumed step may find it at its new place already.</summary>
private static SequencePaths Where(ArchivePaths paths, RenameStep step)
{
@@ -422,18 +473,21 @@ public sealed class MassRenamer
/// before anything of it moves (where its sequences are, what is inside and at its new folder, then each sidecar,
/// settled: <see cref="FitAsync"/>; a sequence whose photos changed since keeps its file names, and its folder and title
/// too when the templates now give it others). A folder that stays keeps the sequences it was to carry
/// (<see cref="RenameResult.CarriedKept"/>). Then the folder moves, every index row follows it (a sidecar that cannot be
/// read or written there fails the step once the others did and every sequence of the step was told), and only then
/// are the files renamed; a sequence whose files are found changed or in use there keeps its file names, and the step
/// goes to its end whatever the caller cancels.
/// (<see cref="RenameResult.CarriedKept"/>), and those an earlier step left inside it keep what they were told. Then the
/// folder moves, every index row follows it (a sidecar that cannot be read or written there fails the step once the
/// others did and every sequence of the step was told), and only then are the files renamed; a sequence whose files are
/// found changed or in use there keeps its file names, and the step goes to its end whatever the caller cancels.
/// </summary>
/// <param name="carried">The steps the plan carries in the folder of <paramref name="step"/>.</param>
/// <param name="along">The sequences earlier steps left inside that folder (<see cref="LeftInsideAsync"/>): they move with
/// it, keeping their title and file names, and are no longer told they stayed (<see cref="Tally.TakenAlong"/>).</param>
/// <param name="resuming">A step whose sequence the index no longer has where the plan says, or that no longer fits it,
/// is dropped (or keeps what no longer fits) instead of failing the rename: its sidecar still carries the id, so the
/// next scan indexes it wherever it ended up.</param>
private async Task ApplyStepAsync(Archive archive, ArchivePaths paths, RenamePlan plan, RenameStep step, IReadOnlyList<RenameStep> carried,
bool resuming, Tally result, CancellationToken cancellationToken)
IReadOnlyList<RenameStep> along, bool resuming, Tally result, CancellationToken cancellationToken)
{
var ids = carried.Select(c => c.SequenceId).Append(step.SequenceId).Order().ToList();
var ids = carried.Concat(along).Select(c => c.SequenceId).Append(step.SequenceId).Order().ToList();
var locks = new List<IAsyncDisposable>(ids.Count);
var leases = new List<LeaseFile>(ids.Count);
try
@@ -460,15 +514,21 @@ public sealed class MassRenamer
if (await StandsAsync(c, resuming, cancellationToken).ConfigureAwait(false)) follow.Add(c);
else result.Dropped.Add(c.SequenceId);
}
// Those an earlier step left inside go with the folder while the index still has them there; why they stayed was told.
var left = new List<RenameStep>(along.Count);
foreach (var a in along)
{
if (await StandsAsync(a, resuming, cancellationToken).ConfigureAwait(false)) left.Add(a);
}
IReadOnlyList<SequenceLocation> locations = step.MovesFolder ? await _db.Sequences.ListLocationsAsync(archive.Id, cancellationToken).ConfigureAwait(false) : [];
if (!CarriesAllInside(paths, step, carried, locations, resuming) || !TargetFree(paths, step, locations, resuming))
if (!CarriesAllInside(paths, step, [.. carried, .. along], locations, resuming) || !TargetFree(paths, step, locations, resuming))
{
result.Dropped.Add(step.SequenceId);
result.CarriedKept.AddRange(follow.Select(c => c.SequenceId));
return;
}
var fits = new List<(RenameStep Step, Fit Fit)>(follow.Count + 1);
foreach (var s in follow.Prepend(step)) fits.Add((s, await FitAsync(paths, plan, s, Settle(paths, s), resuming, cancellationToken).ConfigureAwait(false)));
var fits = new List<(RenameStep Step, Fit Fit)>(follow.Count + left.Count + 1);
foreach (var s in follow.Concat(left).Prepend(step)) fits.Add((s, await FitAsync(paths, plan, s, Settle(paths, s), resuming, cancellationToken).ConfigureAwait(false)));
// Its folder or title no longer fits (its title edited since, the templates naming it otherwise now, or a new title
// planned in a sidecar of a newer format) while nothing of it moved: it stays as it was, with the sequences it would carry.
var first = fits[0].Fit;
@@ -510,6 +570,7 @@ public sealed class MassRenamer
else if (fit == Fit.Newer) result.Newer.Add(s.SequenceId);
if (s.MovesFolder || (s.Retitles && TakesTitle(fit)) || renamed) result.Changed.Add(s.SequenceId);
}
foreach (var a in left) result.TakenAlong(plan.Steps.First(p => p.SequenceId == a.SequenceId));
}
finally
{
@@ -542,8 +603,8 @@ public sealed class MassRenamer
/// a folder or title is given only while the templates still give it from the first photo as it is now
/// (<see cref="NamesStand"/>), and the files are renamed only as the plan renames them (<see cref="FilesStand"/>). When
/// not resuming, a newer format or an edited title refuses the step; photos changed since (a sequence still receiving
/// them) never do: its files keep their names, and so do its folder and title when the templates now give it others,
/// and the rest of the step and of the plan goes on.
/// them) never do: its files keep their names, and so does its title when the templates now give it another folder or
/// title (<see cref="Fit.Names"/>), and the rest of the step and of the plan goes on.
/// </summary>
/// <exception cref="NewerFormatException">Not resuming, and the step would rewrite a sidecar of a newer format.</exception>
/// <exception cref="StaleRenamePlanException">Not resuming, and its title was edited since the plan.</exception>
@@ -567,7 +628,13 @@ public sealed class MassRenamer
}
if (!NamesStand(plan, step, manifest))
{
Log.Warning("archive", $"{sequence.RelativeFolder} keeps its folder, title and file names: its photos changed after the plan was made, "
// Its folder: moved already, carried with the folder around it, or not moved to the planned one (a folder around
// it may still take it along).
var folder = !step.MovesFolder ? string.Empty
: Moved(paths, step) ? ", and its folder moved already"
: step.Carried ? ", and its folder moves only with the folder around it"
: $", and its folder does not move to {step.ToFolder}";
Log.Warning("archive", $"{sequence.RelativeFolder} keeps its title and file names{folder}: its photos changed after the plan was made, "
+ "and the templates now name it otherwise");
return Fit.Names;
}
@@ -640,17 +707,19 @@ public sealed class MassRenamer
}
/// <summary>
/// Whether every sequence the index has inside the folder of a moving step moves with it as planned, checked again under
/// the step's leases just before the folder moves. One indexed there since the dry run would move without its index
/// row following, and was never shown in the plan: the step is refused, or dropped when resuming a folder that has not
/// moved yet (a folder moved before the interruption goes on; the next scan indexes the newcomer where it ended up).
/// Whether every sequence the index has inside the folder of a moving step moves with it (<paramref name="carried"/>:
/// those the plan carries, and those an earlier step left there), checked again under the step's leases just before the
/// folder moves. One indexed there since the dry run would move without its index row following, and was never shown
/// in the plan: the step is refused, or dropped when resuming a folder that has not moved yet (a folder moved before
/// the interruption goes on; the next scan indexes the newcomer where it ended up).
/// </summary>
/// <exception cref="StaleRenamePlanException">Not resuming, and such a sequence is inside the folder.</exception>
private static bool CarriesAllInside(ArchivePaths paths, RenameStep step, IReadOnlyList<RenameStep> carried, IReadOnlyList<SequenceLocation> locations,
bool resuming)
{
if (!step.MovesFolder) return true;
if (StrangerInside(locations, [step], carried.Select(c => c.SequenceId).ToHashSet()) is not { } stranger) return true;
var taken = carried.Select(c => c.SequenceId).ToHashSet();
if (StrangerInside(locations, step, l => taken.Contains(l.Id)) is not { } stranger) return true;
if (!resuming) throw new StaleRenamePlanException(stranger.Id);
if (Moved(paths, step)) return true;
Log.Warning("archive", $"Rename of {step.FromFolder} dropped: {stranger.Folder} was indexed inside it after the plan was made");
@@ -687,12 +756,9 @@ public sealed class MassRenamer
|| RenamePlanner.IsInside(step.ToFolder, l.Folder)));
}
/// <summary>A sequence of <paramref name="locations"/> inside the folder of one of <paramref name="moving"/> that is not in <paramref name="taken"/>; null when there is none.</summary>
private static SequenceLocation? StrangerInside(IReadOnlyList<SequenceLocation> locations, IEnumerable<RenameStep> moving, IReadOnlySet<Guid> taken)
{
var folders = moving.Select(s => s.FromFolder).ToList();
return folders.Count == 0 ? null : locations.FirstOrDefault(l => !taken.Contains(l.Id) && folders.Any(f => RenamePlanner.IsInside(f, l.Folder)));
}
/// <summary>A sequence of <paramref name="locations"/> inside the folder of <paramref name="moving"/> that it does not take along (<paramref name="taken"/>); null when there is none.</summary>
private static SequenceLocation? StrangerInside(IReadOnlyList<SequenceLocation> locations, RenameStep moving, Func<SequenceLocation, bool> taken) =>
locations.FirstOrDefault(l => RenamePlanner.IsInside(moving.FromFolder, l.Folder) && !taken(l));
/// <summary>
/// Sidecar title, then index (folder, title, frames' paths and the paths of renders whose export moved with the
@@ -852,6 +918,19 @@ public sealed class MassRenamer
public List<Guid> CarriedKept { get; } = [.. done.CarriedKept];
public int Files { get; set; } = done.Files;
/// <summary>
/// The sequence of <paramref name="planned"/>, left as it was by its own step or with the folder that was to carry it,
/// went with a folder around it that moved later: its folder changed, its title and file names stayed. Left as it
/// was because it changed since, it now kept its title and file names (<see cref="TitlesKept"/>); carried by a folder
/// that stayed, so too when it was to change more than its folder, otherwise it simply moved; a newer Chronos keeps
/// them (<see cref="Newer"/>) as it did.
/// </summary>
public void TakenAlong(RenameStep planned)
{
var id = planned.SequenceId;
if (Dropped.Remove(id) || (CarriedKept.Remove(id) && (planned.Retitles || planned.FileChanges > 0))) TitlesKept.Add(id);
}
public RenameTally Record() => new()
{
Changed = [.. Changed],
@@ -1157,4 +1157,197 @@ public sealed class ReorganizeTests(TestContext context)
Assert.SequenceEqual(before, await FilesAsync(fx, id));
Assert.Empty(RenameJournal.ListPending(fx.Paths));
}
/// <summary>"trips" with photos of its own, "trips/day2" inside it and "trips/day2/dawn" inside that, adopted by a scan; their ids in that order.</summary>
private static async Task<List<Guid>> NestedAsync(ArchiveFixture fx)
{
Photos.WriteSeries(Path.Combine(fx.Paths.Root, "trips"), Start, 2);
Photos.WriteSeries(Path.Combine(fx.Paths.Root, "trips", "day2"), Start.AddDays(1), 3, firstSeed: 10);
Photos.WriteSeries(Path.Combine(fx.Paths.Root, "trips", "day2", "dawn"), Start.AddDays(2), 2, firstSeed: 20);
await fx.ScanAsync();
var locations = (await fx.Db.Sequences.ListLocationsAsync(fx.Archive.Id)).OrderBy(l => l.Folder, StringComparer.Ordinal).ToList();
Assert.SequenceEqual(["trips", "trips/day2", "trips/day2/dawn"], locations.Select(l => l.Folder));
return [.. locations.Select(l => l.Id)];
}
[Theory]
[Case(false)]
[Case(true)]
public async Task ASequenceLeftAsItWasInsideAFolderThatMovesGoesWithIt(bool held)
{
await using var fx = await CreateAsync(context);
var ids = await NestedAsync(fx);
var (trips, day2, dawn) = (ids[0], ids[1], ids[2]);
var templates = new RenameTemplates(Folder: "moved/{date} {time}");
var plan = await fx.Service.PlanReorganizeAsync(fx.Archive.Id, [trips, day2], templates, fx.Ct);
// The inner one moves out first, with the one inside it; then the folder around it moves.
Assert.SequenceEqual([(day2, false), (trips, false), (dawn, true)], plan.Steps.Select(s => (s.SequenceId, s.Carried)));
var title = (await fx.Db.Sequences.GetAsync(day2))!.Title;
var files = new List<string>();
// Once the plan's own checks passed, the first photo of the inner one goes to the trash: the templates now name it after another.
using var pause = new CancellationTokenSource();
Func<int, IReadOnlyList<Guid>, ValueTask>? stepDone = null;
if (held)
{
stepDone = (_, _) =>
{
pause.Cancel();
return ValueTask.CompletedTask;
};
}
var run = fx.Service.Renamer.ApplyAsync(fx.Archive, plan, async _ =>
{
await TrashFirstPhotoAsync(fx, day2);
files.AddRange((await fx.Db.Frames.ListBySequenceAsync(day2)).Select(f => f.FileName));
}, stepDone, pause.Token);
RenameResult result;
if (held)
{
// Its job paused once the inner one stayed, with the one inside it, and held the plan; retried, it goes on.
await Assert.ThrowsAsync<OperationCanceledException>(() => run);
Assert.SequenceEqual(new[] { day2, dawn }.Order(), Assert.NotNull(RenameJournal.Read(fx.Paths, plan.Id)).Completed.Order());
Assert.True(MassRenamer.Hold(fx.Archive, plan.Id));
result = Assert.NotNull(await fx.Service.Renamer.ResumeAsync(fx.Archive, plan.Id, null, fx.Ct));
}
else
{
result = await run;
}
// It stays in the folder around it, which moves as planned and takes it along with the one inside it: no index row
// is left behind, its title and file names stay, and neither is said to have stayed as it was.
Assert.SequenceEqual([trips, day2, dawn], result.Changed);
Assert.SequenceEqual([day2], result.TitlesKept);
Assert.Empty(result.Dropped);
Assert.Empty(result.CarriedKept);
Assert.Equal(("moved/2026-09-28 18-00-00", "moved/2026-09-28 18-00-00/day2", "moved/2026-09-28 18-00-00/day2/dawn"),
(await FolderAsync(fx, trips), await FolderAsync(fx, day2), await FolderAsync(fx, dawn)));
Assert.Equal(title, (await fx.Db.Sequences.GetAsync(day2))!.Title);
Assert.Count(2, files);
Assert.SequenceEqual(files, (await fx.Db.Frames.ListBySequenceAsync(day2)).Select(f => f.FileName));
Assert.Empty(RenameJournal.ListPending(fx.Paths));
Assert.Equal(0, (await fx.ScanAsync()).Indexed);
}
[Test]
public async Task ARetriedJobTakesAlongASequenceLeftAsItWasInsideAFolderItMoves()
{
await using var fx = await CreateAsync(context);
var parent = await UploadAsync(fx, 2);
var folder = (await fx.Db.Sequences.GetAsync(parent))!.Folder;
Photos.WriteSeries(Path.Combine(fx.Paths.FromRelative(folder), "extra"), Start.AddDays(3), 3, firstSeed: 50);
await fx.ScanAsync();
var nested = (await fx.Db.Sequences.ListLocationsAsync(fx.Archive.Id)).Single(l => l.Folder == folder + "/extra").Id;
var templates = new RenameTemplates(Folder: "moved/{date} {time}");
var plan = await fx.Service.PlanReorganizeAsync(fx.Archive.Id, [parent, nested], templates, fx.Ct);
Assert.SequenceEqual([nested, parent], plan.Steps.Select(s => s.SequenceId));
var name = plan.Steps[0].ToTitle ?? plan.Steps[0].FromFolder;
// Its job held the plan before its first step (paused); the first photo of the inner one then went to the trash.
RenameJournal.Write(fx.Paths, new RenameJournal(plan, [], Held: true));
await TrashFirstPhotoAsync(fx, nested);
// Resumed, the job moves the folder around it as planned, with it inside, and fails for it alone: never for the plan.
var engine = await fx.StartEngineAsync();
var failed = await fx.WaitForAsync((await fx.EnqueueWithCheckpointAsync(engine, ArchiveJobs.Reorganize(fx.Archive, plan, templates),
"plan:" + plan.Id.ToString("N"))).Id, JobState.Failed);
Assert.Equal(Loc.Plural("Archive.Error.ReorganizeKept", 1), failed.Error);
Assert.Contains(Loc.L("Archive.Log.Reorganized", 2, 0), failed.LogTail);
Assert.Contains(Loc.L("Archive.Log.ReorganizeTitleKept", name), failed.LogTail);
Assert.False(failed.LogTail!.Contains(Loc.L("Archive.Log.ReorganizeDropped", name), StringComparison.Ordinal), failed.LogTail);
Assert.Equal(("moved/2026-09-28 18-00-00", "moved/2026-09-28 18-00-00/extra"), (await FolderAsync(fx, parent), await FolderAsync(fx, nested)));
Assert.Empty(RenameJournal.ListPending(fx.Paths));
}
[Test]
public async Task AFolderNamedAfterAPhotoNoLongerFirstStaysWithTheSequencesInside()
{
await using var fx = await CreateAsync(context);
var parent = await UploadAsync(fx, 3);
var folder = (await fx.Db.Sequences.GetAsync(parent))!.Folder;
Photos.WriteSeries(Path.Combine(fx.Paths.FromRelative(folder), "extra"), Start.AddDays(3), 2, firstSeed: 50);
await fx.ScanAsync();
var nested = (await fx.Db.Sequences.ListLocationsAsync(fx.Archive.Id)).Single(l => l.Folder == folder + "/extra").Id;
var plan = await fx.Service.PlanReorganizeAsync(fx.Archive.Id, [parent], new RenameTemplates(Folder: "moved/{date} {time}", File: "{n:000}"), fx.Ct);
Assert.SequenceEqual([(parent, false, 3), (nested, true, 0)], plan.Steps.Select(s => (s.SequenceId, s.Carried, s.FileChanges)));
var files = new List<string>();
// Once the plan's own checks passed, the first photo of the folder that carries the other goes to the trash.
var result = await fx.Service.Renamer.ApplyAsync(fx.Archive, plan, async _ =>
{
await TrashFirstPhotoAsync(fx, parent);
files.AddRange(await FilesAsync(fx, parent));
}, null, fx.Ct);
// It stays as it was, and the one inside stays with it, told why; the plan is over.
Assert.SequenceEqual([parent], result.Dropped);
Assert.SequenceEqual([nested], result.CarriedKept);
Assert.Empty(result.Changed);
Assert.Equal((folder, folder + "/extra"), (await FolderAsync(fx, parent), await FolderAsync(fx, nested)));
Assert.SequenceEqual(files, await FilesAsync(fx, parent));
Assert.Empty(RenameJournal.ListPending(fx.Paths));
}
[Theory]
[Case(false)]
[Case(true)]
public async Task AStepWhoseFolderMovedBeforeItsFirstPhotoWentKeepsItsTitleAndFileNames(bool held)
{
await using var fx = await CreateAsync(context);
var id = await UploadAsync(fx, 3);
var plan = await fx.Service.PlanReorganizeAsync(fx.Archive.Id, [id], new RenameTemplates(Folder: "moved/{date} {time}", File: "{n:000}", Title: "{time}"),
fx.Ct);
var step = plan.Steps.Single();
Assert.True(step is { ToFolder: "moved/2026-09-28 18-00-00", ToTitle: "18-00-00", FileChanges: 3 });
// The first photo went to the trash; a run of the plan was then interrupted (or failed, and its job held it) once
// the folder moved, before its index row followed.
await TrashFirstPhotoAsync(fx, id);
var before = await FilesAsync(fx, id);
RenameJournal.Write(fx.Paths, new RenameJournal(plan, [], Held: held));
RenamePlanner.MoveFolder(fx.Paths, step);
// Resumed, its index row follows the folder; the title and file names stay, the templates naming it after another photo now.
var result = Assert.NotNull(await fx.Service.Renamer.ResumeAsync(fx.Archive, plan.Id, null, fx.Ct));
Assert.SequenceEqual([id], result.TitlesKept);
Assert.SequenceEqual([id], result.Changed);
Assert.Empty(result.Dropped);
Assert.Equal(0, result.Files);
Assert.Equal("moved/2026-09-28 18-00-00", await FolderAsync(fx, id));
Assert.Null((await fx.Db.Sequences.GetAsync(id))!.Title);
Assert.SequenceEqual(before, await FilesAsync(fx, id));
Assert.Empty(RenameJournal.ListPending(fx.Paths));
}
[Test]
public async Task AJobRetriedWhileTheArchiveIsOfflineKeepsItsPlanForTheNextRetry()
{
var root = context.CreateTempDirectory("archive");
await using var fx = await CreateAsync(context, root);
var first = await UploadAsync(fx, 2);
var second = await UploadAsync(fx, 2, firstSeed: 20, firstSecond: 3600, key: "upload-2");
var templates = new RenameTemplates(File: "{n:000}");
var plan = await fx.Service.PlanReorganizeAsync(fx.Archive.Id, [first, second], templates, fx.Ct);
// Its job held the plan before its first step (paused); the drive was then unplugged.
RenameJournal.Write(fx.Paths, new RenameJournal(plan, [], Held: true));
var unplugged = root + "-unplugged";
Directory.Move(root, unplugged);
// Retried, the job fails as the archive does: no journal found there says nothing, and it keeps its plan.
var engine = await fx.StartEngineAsync();
var checkpoint = "plan:" + plan.Id.ToString("N");
var job = await fx.EnqueueWithCheckpointAsync(engine, ArchiveJobs.Reorganize(fx.Archive, plan, templates), checkpoint);
var failed = await fx.WaitForAsync(job.Id, JobState.Failed);
Assert.Equal(Loc.L("Archive.Error.Offline", fx.Paths.Root), failed.Error);
Assert.Equal(checkpoint, failed.Checkpoint);
Assert.False(failed.LogTail!.Contains(Loc.L("Archive.Log.ReorganizedEarlier"), StringComparison.Ordinal), failed.LogTail);
// Plugged in again, the next retry completes the plan.
Directory.Move(unplugged, root);
await engine.ResumeAsync(job.Id, fx.Ct);
var done = await fx.WaitForAsync(job.Id, JobState.Done);
Assert.Contains(Loc.L("Archive.Log.Reorganized", 2, 4), done.LogTail);
foreach (var id in new[] { first, second }) Assert.SequenceEqual(["originals/001.JPG", "originals/002.JPG"], await FilesAsync(fx, id));
Assert.Empty(RenameJournal.ListPending(fx.Paths));
}
}