fix(server): key kept Reorganize plans on what they read, plan apart from the request
A kept dry run was keyed on a stamp of every sequence row of the archive, updated_at and verified_at included, so any upload, ingest status write, verify or tag edit anywhere invalidated it and the confirm planned the whole selection again. The plan was also bound to the request, so a proxy cutting the request off cancelled it and a retry started over. - The stamp (SequenceRepository.GetStampAsync) now hashes only what the planner reads from the index: every row's id and folder (collisions and what a moving folder carries), the planned rows' title, first capture, camera, lens, photo count and frames version, and the titles of the rows inside a planned folder. With all, every row is planned. - ReorganizePlanCache makes a plan on the thread pool with the host's stopping token, once per key for concurrent requests; a request cut off only stops waiting, and the same request again waits for the plan or takes it. Refusals and failures are still not kept. - A kept plan is dropped by its own timer five minutes after it was made (and never used past that), instead of staying until the next request. - The protocol and endpoint docs say that plan_changed answers changes the index has; one made on disk alone is found by the job, which fails and changes nothing. The Capacity doc no longer claims a step per sequence. - Tests: what the stamp follows and ignores, a kept plan surviving an upload elsewhere and a verify, one plan for concurrent requests and after a cut-off, the timer, an all:true apply carrying a nested sequence beside a blocked-only one, the first sequence told before a pause. The no-archive-module job_unavailable test expected a folder the planner no longer plans without a sidecar; it now expects the unreadable step. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 7e01a4f85fe4b880ddd98a481fc8591649cec254)
This commit is contained in:
1 parent
6e99a4b269
commit
37800b28ef
7 files changed
+328
-80
No files matched your search
@@ -428,21 +428,40 @@ public sealed class SequenceRepository
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// A digest of the index rows of <paramref name="archiveId"/> (owner review 2, R7): another value once a sequence is
|
||||
/// added or removed, or once a row changes in what a Reorganize plans from (folder, title, first capture, camera, lens,
|
||||
/// frame count) or in what every write of it moves (its versions, verify time and time of change). A change made on
|
||||
/// disk alone, which the index does not have yet, leaves it as it was.
|
||||
/// A digest of what a Reorganize of <paramref name="sequenceIds"/> (null: every sequence) plans from in the index of
|
||||
/// <paramref name="archiveId"/> (owner review 2, R7): every sequence of the archive and its folder, which decide the
|
||||
/// folders that get " (2)" and what a moving folder carries; of the sequences planned, their title, first capture,
|
||||
/// camera, lens, photo count and frames version; of the others inside their folders, the title. Another value once one
|
||||
/// of these changes. Any other write (an upload into a sequence not planned, a verify, a status, tag or note change)
|
||||
/// leaves it as it was, and so does a change made on disk alone, which the index does not have.
|
||||
/// </summary>
|
||||
public async Task<string> GetStampAsync(Guid archiveId, CancellationToken cancellationToken = default)
|
||||
public async Task<string> GetStampAsync(Guid archiveId, IReadOnlyCollection<Guid>? sequenceIds, CancellationToken cancellationToken = default)
|
||||
{
|
||||
var rows = await _db.ListAsync(
|
||||
"SELECT id, folder, title, first_capture, camera_make, camera_model, lens, frame_count, frames_version, settings_version, verified_at, updated_at " +
|
||||
"FROM sequences WHERE archive_id = $1 ORDER BY id", [archiveId],
|
||||
r => string.Create(CultureInfo.InvariantCulture,
|
||||
$"{r.GetGuid(0):N}\u001f{r.GetString(1)}\u001f{r.GetStringOrNull(2)}\u001f{r.GetDateTimeOrNull(3):O}\u001f{r.GetStringOrNull(4)}\u001f" +
|
||||
$"{r.GetStringOrNull(5)}\u001f{r.GetStringOrNull(6)}\u001f{r.GetInt32(7)}\u001f{r.GetInt32(8)}\u001f{r.GetInt32(9)}\u001f" +
|
||||
$"{r.GetDateTimeOffsetOrNull(10):O}\u001f{r.GetDateTimeOffset(11):O}"), cancellationToken).ConfigureAwait(false);
|
||||
return Convert.ToHexStringLower(SHA256.HashData(Encoding.UTF8.GetBytes(string.Join('\n', rows))));
|
||||
"SELECT id, folder, title, first_capture, camera_make, camera_model, lens, frame_count, frames_version FROM sequences WHERE archive_id = $1 ORDER BY id",
|
||||
[archiveId], r => (Id: r.GetGuid(0), Folder: r.GetString(1), Title: r.GetStringOrNull(2), Read: string.Create(CultureInfo.InvariantCulture,
|
||||
$"{r.GetStringOrNull(2)}\u001f{r.GetDateTimeOrNull(3):O}\u001f{r.GetStringOrNull(4)}\u001f{r.GetStringOrNull(5)}\u001f{r.GetStringOrNull(6)}\u001f" +
|
||||
$"{r.GetInt32(7)}\u001f{r.GetInt32(8)}")), cancellationToken).ConfigureAwait(false);
|
||||
var selected = sequenceIds?.ToHashSet();
|
||||
var planned = new HashSet<string>(rows.Where(r => selected?.Contains(r.Id) != false && r.Folder.Length > 0).Select(r => r.Folder),
|
||||
StringComparer.OrdinalIgnoreCase);
|
||||
using var hash = IncrementalHash.CreateHash(HashAlgorithmName.SHA256);
|
||||
foreach (var row in rows)
|
||||
{
|
||||
var read = selected?.Contains(row.Id) != false ? row.Read : Inside(row.Folder) ? row.Title : null;
|
||||
hash.AppendData(Encoding.UTF8.GetBytes(string.Create(CultureInfo.InvariantCulture, $"{row.Id:N}\u001f{row.Folder}\u001e{read}\n")));
|
||||
}
|
||||
return Convert.ToHexStringLower(hash.GetHashAndReset());
|
||||
|
||||
// Whether a folder lies inside a planned sequence's folder (RenamePlanner.IsInside), checking each folder around it.
|
||||
bool Inside(string folder)
|
||||
{
|
||||
for (var slash = folder.LastIndexOf('/'); slash > 0; slash = folder.LastIndexOf('/', slash - 1))
|
||||
{
|
||||
if (planned.Contains(folder[..slash])) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
|
||||
Reference in new issue
Block a user