feat(archive): photo delete, tombstones, trash kinds and emptying, library API

Owner review 2, R6: the web panel manages the library without editing it, so
the engine and the server must delete single photos safely, keep them out of
sync afterwards, and let an administrator restore or finally erase the trash.

- Migration 0008: frame_removals (tombstones by sequence and SHA-256, reason,
  replacement); the sidecar gains "removed" (omitted when empty) and frames a
  reserved "source"; the indexer writes the tombstones on every sync, so a
  rebuild keeps them, and drops a hero frame that is gone.
- FrameTrash: dry run (busy, frames changed, offline, would empty), delete under
  the sequence lease with a write-ahead journal that the next writer undoes
  after a crash, files to trash/photos/<seq>-<time>/originals with a record,
  sidecar without the frames plus tombstones, index sync clearing analyses,
  thumbnails and proxies, proxies renamed to follow their frames, follow-up
  jobs. Restore re-hashes every file, keeps mismatches in the trash, puts the
  same frame records back and drops their tombstones, or files the photos into
  a new sequence through the ingest when theirs is gone.
- Project positions on vanished photos are translated by photo and wait as
  pending frames, so they come back with a restore.
- Trash entries have a kind (sequence, photos, replaced); emptying is a dry run
  with a fingerprint and a new EmptyTrash job (18) that erases exactly the
  planned entries, refuses a stale plan and erases replaced originals only
  while their replacements still hash right.
- Sync honours tombstones: nothing one side deleted is offered back, local
  files are never deleted, replaced photos count as their replacement; the
  server answers an upload of a tombstoned SHA-256 as already there.
- Server: sequence thumbnail, photo delete, bulk delete/retag/jobs, trash
  list/restore/empty endpoints with their error messages; a verify of
  sequences now targets their archive (POST /jobs failed before).
- Desktop trash dialog lists photo entries and empties the trash after a
  typed confirmation through the same job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 35bb56690a1d69eae57153d036e1409201cd73f7)
This commit is contained in:
Alby96 committed 2026-10-27 10:49:34 +01:00
1 parent 4b9ba04fe4
commit 51782a3ada
63 files changed
+3303 -60

No files matched your search

+8
View File
@@ -114,6 +114,14 @@ public static class JobEndpoints
return ApiErrors.BadRequest(context, "invalid_parameters", "parametersJson");
}
if (kind == JobKind.Verify)
{
// A verify of sequences targets their archive and names them in its parameters, as its handler reads it.
var verify = LibraryEndpoints.SequenceJob(kind, archive.Current, targets, label);
targets = verify.Targets;
queue = [(label, verify.ParametersJson)];
}
Job? first = null;
foreach (var (jobLabel, parameters) in queue)
{
+377
View File
@@ -0,0 +1,377 @@
using System.Globalization;
using Microsoft.AspNetCore.Mvc;
using Chronos.Core.Archiving;
using Chronos.Core.Imaging.Decoders;
using Chronos.Core.Model;
using Chronos.Data;
using Chronos.Data.Archiving;
using Chronos.Data.Archiving.Jobs;
using Chronos.Data.Jobs;
using Chronos.Data.Processing;
using Chronos.Data.Repositories;
using Chronos.Protocol;
using Chronos.Server.Events;
using Chronos.Server.Jobs;
using Chronos.Server.Security;
using Chronos.Server.Storage;
namespace Chronos.Server.Api;
/// <summary>
/// The library the admin panel manages without editing (owner review 2, R6): a sequence's cover thumbnail, deleting photos
/// of a sequence into the archive's trash (<see cref="FrameTrash"/>), and mass operations on many sequences at once —
/// delete into the trash, retag, queue analysis, proxies, thumbnails or a verify — each answering per sequence whether it
/// was done or skipped, and why.
/// </summary>
public static class LibraryEndpoints
{
/// <summary>Sequences one bulk request may name.</summary>
public const int MaxBulkIds = 1000;
/// <summary>Photos one delete request may name.</summary>
public const int MaxFrameIds = 100_000;
private const string Ok = "ok";
private const string Skipped = "skipped";
private const string Regenerable = "private, max-age=3600";
public static void Map(RouteGroupBuilder api)
{
var sequences = api.MapGroup("/sequences");
sequences.MapGet("/{id:guid}/thumb", ThumbAsync).Requires(Permissions.Read);
sequences.MapPost("/{id:guid}/frames/delete", DeleteFramesAsync).Requires(Permissions.Delete);
sequences.MapPost("/bulk/delete", BulkDeleteAsync).Requires(Permissions.Delete);
sequences.MapPost("/bulk/retag", BulkRetagAsync).Requires(Permissions.Edit);
sequences.MapPost("/bulk/jobs", BulkJobsAsync).Requires(Permissions.Edit);
}
/// <summary>The archive engine behind these operations: the host's, or one of its own (a test host that replaces the archive module).</summary>
internal static ArchiveService Engine(ArchiveService? archives, ChronosDatabase db) =>
archives ?? new ArchiveService(db, new ArchiveServiceOptions { Decoder = DecoderFactory.CreateServer(), Owner = ArchiveModule.Owner, ImagesAfterScan = true });
/// <summary>
/// The thumbnail of the sequence's hero frame, else of its first frame, streamed like <c>/frames/{id}/thumb</c> with an
/// ETag (304 when unchanged); 404 <c>not_found</c> (<c>thumbnail</c>) when the sequence has no frame or that thumbnail
/// is not built yet.
/// </summary>
private static async Task<IResult> ThumbAsync(HttpContext context, Guid id, ChronosDatabase db, ServerArchive archive, IFrameAssetLocator locator)
{
var ct = context.RequestAborted;
if (await db.Sequences.GetAsync(id, ct) is not { } sequence) return ApiErrors.NotFound(context, "sequence");
var frame = sequence.HeroFrameId is { } hero && await db.Frames.GetAsync(hero, ct) is { } chosen && chosen.SequenceId == id
? chosen
: (await db.Frames.ListBySequenceAsync(id, limit: 1, cancellationToken: ct)).FirstOrDefault();
if (frame is null) return ApiErrors.NotFound(context, "thumbnail");
var location = await locator.LocateAsync(archive.Current, sequence, frame, FrameAsset.Thumbnail, ct);
if (location is null || !ArchivePathGuard.IsInside(archive.Root, location.Path)) return ApiErrors.NotFound(context, "thumbnail");
var file = new FileInfo(location.Path);
if (!file.Exists || file.LinkTarget is not null) return ApiErrors.NotFound(context, "thumbnail");
var etag = string.Create(CultureInfo.InvariantCulture, $"s-{frame.Id:N}-{file.LastWriteTimeUtc.Ticks}-{location.Offset}");
return FileResponses.Serve(context, file, location.Offset, location.Length, location.ContentType, Regenerable, etag)
?? ApiErrors.NotFound(context, "thumbnail");
}
/// <summary>
/// Moves photos of a sequence into the archive's trash (<see cref="FrameTrash"/>; BRIEF §0.2 rule 8: nothing is erased),
/// or with <c>dryRun</c> says what would go. Owners and admins only, like deleting the sequence. 400 <c>bad_request</c>
/// (<c>frameIds</c>); 409 <c>frames_conflict</c> (the frames changed since <c>expectedFramesVersion</c>, or a photo named
/// is not the sequence's; details: the frames version now), <c>sequence_busy</c>, <c>would_empty</c> (every photo would
/// go: delete the sequence instead), <c>folder_in_use</c>; 503 <c>archive_offline</c>. Publishes <c>sequenceChanged</c>
/// <c>frames</c>; the thumbnails, proxies and analyses follow as jobs.
/// </summary>
private static async Task<IResult> DeleteFramesAsync(HttpContext context, Caller caller, Guid id, FrameDeleteRequest request, ChronosDatabase db,
[FromServices] ArchiveService? archives, JobEngineHost host, AuditLog audit, EventHub events, IHostApplicationLifetime lifetime)
{
var ct = context.RequestAborted;
if (request.FrameIds is not { Count: > 0 and <= MaxFrameIds } ids) return ApiErrors.BadRequest(context, "bad_request", "frameIds");
if (await db.Sequences.GetAsync(id, ct) is not { } sequence) return ApiErrors.NotFound(context, "sequence");
if (!caller.IsAdmin && sequence.OwnerUserId != caller.Id) return ApiErrors.Forbidden(context);
var engine = Engine(archives, db).FrameTrash;
FrameDeletePlan plan;
try
{
plan = await engine.PlanDeleteAsync(id, ids, request.ExpectedFramesVersion, ct);
}
catch (ArgumentException)
{
return ApiErrors.NotFound(context, "sequence");
}
if (plan.Blocked.Count > 0) return Refused(context, plan.Blocked[0], plan.Sequence);
if (request.DryRun)
return Results.Json(new FrameDeleteResultDto(id, plan.Frames.Count, plan.Bytes, plan.Sequence.FramesVersion, plan.Sequence.FrameCount, null, true),
ProtocolJson.Default.FrameDeleteResultDto);
FrameDeleteResult result;
try
{
// Once confirmed the deletion runs to the end even if the client goes away meanwhile.
result = await engine.DeleteAsync(plan, caller.User.Username, host.Engine, lifetime.ApplicationStopping);
}
catch (ArchiveOfflineException)
{
return Refused(context, FrameDeleteBlock.Offline, plan.Sequence);
}
catch (Exception ex) when (ex is StaleDeletePlanException or LeaseUnavailableException)
{
var again = await engine.PlanDeleteAsync(id, ids, plan.Sequence.FramesVersion, ct);
return Refused(context, again.Blocked.Count > 0 ? again.Blocked[0] : ex is LeaseUnavailableException ? FrameDeleteBlock.Busy : FrameDeleteBlock.FramesChanged,
again.Sequence);
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
// A photo is open in another program: nothing moved.
return ApiErrors.Conflict(context, "folder_in_use");
}
await audit.WriteAsync(caller, context, "frames_deleted", "sequence", id.ToString(), $"{plan.Frames.Count} · {result.Entry.Name}");
events.Publish(new SequenceChangedEvent(id, "frames", result.Sequence.FramesVersion, result.Sequence.SettingsVersion));
return Results.Json(new FrameDeleteResultDto(id, plan.Frames.Count, plan.Bytes, result.Sequence.FramesVersion, result.Sequence.FrameCount,
result.Entry.Name, false), ProtocolJson.Default.FrameDeleteResultDto);
}
private static IResult Refused(HttpContext context, FrameDeleteBlock reason, Sequence sequence) => reason switch
{
FrameDeleteBlock.Offline => ApiErrors.Result(context, StatusCodes.Status503ServiceUnavailable, "archive_offline"),
FrameDeleteBlock.Busy => ApiErrors.Conflict(context, "sequence_busy"),
FrameDeleteBlock.WouldEmpty => ApiErrors.Conflict(context, "would_empty"),
_ => ApiErrors.Conflict(context, "frames_conflict", sequence.FramesVersion.ToString(CultureInfo.InvariantCulture)),
};
/// <summary>
/// Moves many sequences into the trash as <c>DELETE /sequences/{id}</c> does each (or, with <c>dryRun</c>, plans them):
/// one item per id, <c>ok</c> with the trash folder, or <c>skipped</c> with the code that says why (<c>not_found</c>,
/// <c>forbidden</c> for a sequence of another user, <c>sequence_busy</c>, <c>archive_root</c>, <c>contains_sequences</c>,
/// <c>sidecar_mismatch</c>, <c>folder_missing</c>, <c>archive_offline</c>, <c>folder_in_use</c>). Each sequence moves whole
/// or not at all. 400 <c>bad_request</c> (<c>ids</c>).
/// </summary>
private static async Task<IResult> BulkDeleteAsync(HttpContext context, Caller caller, BulkDeleteRequest request, ChronosDatabase db,
[FromServices] ArchiveService? archives, AuditLog audit, EventHub events, IHostApplicationLifetime lifetime)
{
var ct = context.RequestAborted;
if (Ids(request.Ids) is not { } ids) return ApiErrors.BadRequest(context, "bad_request", "ids");
var trash = Engine(archives, db).Trash;
var items = new Dictionary<Guid, BulkItemResult>();
var allowed = new List<Sequence>();
foreach (var id in ids)
{
if (await db.Sequences.GetAsync(id, ct) is not { } sequence) items[id] = Skip(context, id, "not_found");
else if (!caller.IsAdmin && sequence.OwnerUserId != caller.Id) items[id] = Skip(context, id, "forbidden");
else allowed.Add(sequence);
}
var steps = new List<DeleteStep>();
foreach (var (sequence, reason, step) in await PlanEachAsync(trash, allowed, ct))
{
if (step is not null) steps.Add(step);
else items[sequence.Id] = Skip(context, sequence.Id, reason!);
}
var photos = 0;
long bytes = 0;
foreach (var step in steps)
{
var sequence = step.Sequence;
if (!request.DryRun)
{
try
{
await trash.DeleteAsync(new DeletePlan([step], []), caller.User.Username, null, lifetime.ApplicationStopping);
}
catch (ArchiveOfflineException)
{
items[sequence.Id] = Skip(context, sequence.Id, "archive_offline");
continue;
}
catch (Exception ex) when (ex is StaleDeletePlanException or LeaseUnavailableException)
{
items[sequence.Id] = Skip(context, sequence.Id, "sequence_busy");
continue;
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
items[sequence.Id] = Skip(context, sequence.Id, "folder_in_use");
continue;
}
await audit.WriteAsync(caller, context, "sequence_deleted", "sequence", sequence.Id.ToString(), sequence.DisplayTitle);
events.Publish(new SequenceChangedEvent(sequence.Id, "deleted", sequence.FramesVersion, sequence.SettingsVersion));
}
items[sequence.Id] = new BulkItemResult(sequence.Id, Ok, TrashFolder: step.TrashFolder);
photos += sequence.FrameCount;
bytes += step.Bytes;
}
return Results.Json(Result(ids, items, request.DryRun, photos, bytes), ProtocolJson.Default.BulkResultDto);
}
/// <summary>Each sequence's delete plan: its step, or the code of why it cannot go.</summary>
private static async Task<List<(Sequence Sequence, string? Reason, DeleteStep? Step)>> PlanEachAsync(SequenceTrash trash, List<Sequence> sequences,
CancellationToken ct)
{
var planned = new List<(Sequence, string?, DeleteStep?)>();
if (sequences.Count == 0) return planned;
DeletePlan plan;
try
{
plan = await trash.PlanAsync([.. sequences.Select(s => s.Id)], null, ct);
}
catch (ArgumentException) when (sequences.Count == 1)
{
return [(sequences[0], "not_found", null)];
}
catch (ArgumentException)
{
// One left the index meanwhile: planned one by one, so the others still go.
foreach (var sequence in sequences) planned.AddRange(await PlanEachAsync(trash, [sequence], ct));
return planned;
}
catch (ArchiveOfflineException)
{
return [.. sequences.Select(s => (s, (string?)"archive_offline", (DeleteStep?)null))];
}
foreach (var step in plan.Steps) planned.Add((step.Sequence, null, step));
foreach (var blocked in plan.Blocked) planned.Add((blocked.Sequence, BlockCode(blocked.Reason), null));
return planned;
}
/// <summary>The error code of a sequence that cannot go to the trash, as <c>DELETE /sequences/{id}</c> answers it.</summary>
internal static string BlockCode(DeleteBlock reason) => reason switch
{
DeleteBlock.Offline => "archive_offline",
DeleteBlock.Busy => "sequence_busy",
DeleteBlock.ArchiveRoot => "archive_root",
DeleteBlock.ContainsSequences => "contains_sequences",
DeleteBlock.ForeignSidecar => "sidecar_mismatch",
_ => "folder_missing",
};
/// <summary>
/// Adds and removes tags on many sequences, each with the same edit as a PATCH of its tags (case-insensitive, the first
/// spelling kept; settings version + 1 when they change). Skipped: <c>not_found</c>, <c>bad_request</c> (more than 50
/// tags then). 400 <c>bad_request</c> (<c>ids</c>, <c>tags</c>). Publishes <c>sequenceChanged</c> <c>settings</c>.
/// </summary>
private static async Task<IResult> BulkRetagAsync(HttpContext context, Caller caller, BulkRetagRequest request, ChronosDatabase db, AuditLog audit,
EventHub events)
{
var ct = context.RequestAborted;
if (Ids(request.Ids) is not { } ids) return ApiErrors.BadRequest(context, "bad_request", "ids");
if (Tags(request.Add) is not { } add || Tags(request.Remove) is not { } remove || add.Count + remove.Count == 0)
return ApiErrors.BadRequest(context, "bad_request", "tags");
var items = new Dictionary<Guid, BulkItemResult>();
foreach (var id in ids)
{
items[id] = await RetagAsync(context, caller, db, audit, events, id, add, remove, ct);
}
return Results.Json(Result(ids, items, false, 0, 0), ProtocolJson.Default.BulkResultDto);
}
private static async Task<BulkItemResult> RetagAsync(HttpContext context, Caller caller, ChronosDatabase db, AuditLog audit, EventHub events, Guid id,
IReadOnlyList<string> add, IReadOnlyList<string> remove, CancellationToken ct)
{
for (var attempt = 0; attempt < 3; attempt++)
{
if (await db.Sequences.GetAsync(id, ct) is not { } sequence) return Skip(context, id, "not_found");
var tags = sequence.Tags.Where(t => !remove.Contains(t, StringComparer.OrdinalIgnoreCase)).ToList();
tags.AddRange(add.Where(t => !tags.Contains(t, StringComparer.OrdinalIgnoreCase)));
if (tags.Count > SequenceEndpoints.MaxTags) return Skip(context, id, "bad_request");
if (tags.ToHashSet(StringComparer.OrdinalIgnoreCase).SetEquals(sequence.Tags)) return new BulkItemResult(id, Ok);
var result = await db.Sequences.EditAsync(id, new SequenceEdit { Tags = tags }, sequence.SettingsVersion, ct);
if (result.Outcome == SequenceEditOutcome.NotFound) return Skip(context, id, "not_found");
// Edited meanwhile: the tags apply again on top of that edit.
if (result.Outcome == SequenceEditOutcome.Conflict) continue;
var updated = result.Sequence!;
await audit.WriteAsync(caller, context, "sequence_updated", "sequence", id.ToString(), "tags");
events.Publish(new SequenceChangedEvent(id, "settings", updated.FramesVersion, updated.SettingsVersion));
return new BulkItemResult(id, Ok);
}
return Skip(context, id, "settings_conflict");
}
/// <summary>
/// Queues one job over many sequences: <c>analyse</c> (light analysis), <c>deep</c> (deep analysis), <c>proxies</c>,
/// <c>thumbnails</c> or <c>verify</c> (a verify of those sequences in their archive), at user priority. Sequences not in
/// the index are skipped (<c>not_found</c>). 202 with the job when one was queued; 400 <c>invalid_job_kind</c>,
/// <c>bad_request</c> (<c>ids</c>); 409 <c>job_unavailable</c> when this server cannot run that kind.
/// </summary>
private static async Task<IResult> BulkJobsAsync(HttpContext context, Caller caller, BulkJobsRequest request, ChronosDatabase db, ServerArchive archive,
JobEngineHost host, AuditLog audit)
{
var ct = context.RequestAborted;
if (Ids(request.Ids) is not { } ids) return ApiErrors.BadRequest(context, "bad_request", "ids");
if (BulkKind(request.Kind) is not { } kind) return ApiErrors.BadRequest(context, "invalid_job_kind");
if (!host.CanRun(kind)) return ApiErrors.Conflict(context, "job_unavailable", JobWire.Kind(kind));
var items = new Dictionary<Guid, BulkItemResult>();
var found = new List<Sequence>();
foreach (var id in ids)
{
if (await db.Sequences.GetAsync(id, ct) is { } sequence && sequence.ArchiveId == archive.Id)
{
found.Add(sequence);
items[id] = new BulkItemResult(id, Ok);
}
else
{
items[id] = Skip(context, id, "not_found");
}
}
if (found.Count == 0) return Results.Json(Result(ids, items, false, 0, 0), ProtocolJson.Default.BulkResultDto);
var targets = found.Select(s => s.Id).ToList();
var label = Label(found);
var job = await host.Engine.EnqueueAsync(SequenceJob(kind, archive.Current, targets, label) with { Priority = JobPriority.User }, caller.Id, ct);
await audit.WriteAsync(caller, context, "job_created", "job", job.Id.ToString(), $"{JobWire.Kind(kind)} · {label}");
return Results.Json(Result(ids, items, false, 0, 0) with { Job = Dto.Job(job) }, ProtocolJson.Default.BulkResultDto,
statusCode: StatusCodes.Status202Accepted);
}
/// <summary>The kinds a bulk job request names.</summary>
private static JobKind? BulkKind(string? kind) => kind?.Trim().ToLowerInvariant() switch
{
"analyse" => JobKind.LightAnalyse,
"deep" => JobKind.DeepAnalyse,
"proxies" => JobKind.Proxies,
"thumbnails" => JobKind.Thumbs,
"verify" => JobKind.Verify,
_ => null,
};
/// <summary>
/// The job of a sequence kind over <paramref name="sequenceIds"/>. A verify targets the archive and names the sequences
/// in its parameters (<see cref="ArchiveJobs.Verify"/>), as its handler reads it.
/// </summary>
internal static ArchiveJobRequest SequenceJob(JobKind kind, Archive archive, IReadOnlyList<Guid> sequenceIds, string label) => kind switch
{
JobKind.Verify => ArchiveJobs.Verify(archive, sequenceIds) with { Label = label },
JobKind.LightAnalyse => ProcessingJobs.LightAnalysis(sequenceIds, label),
JobKind.DeepAnalyse => ProcessingJobs.DeepAnalysis(sequenceIds, label),
JobKind.Proxies => ArchiveJobs.Proxies(sequenceIds, label),
JobKind.Thumbs => ArchiveJobs.Thumbnails(sequenceIds, label),
_ => throw new ArgumentOutOfRangeException(nameof(kind), kind, "Not a sequence job."),
};
/// <summary>"Harbour sunset", or the first three titles and "…".</summary>
internal static string Label(IReadOnlyList<Sequence> sequences) =>
sequences.Count == 1 ? sequences[0].DisplayTitle
: string.Join(", ", sequences.Take(3).Select(s => s.DisplayTitle)) + (sequences.Count > 3 ? " …" : string.Empty);
/// <summary>The ids of a bulk request, once each in their order; null when there are none or too many.</summary>
private static List<Guid>? Ids(IReadOnlyList<Guid>? ids) =>
ids is { Count: > 0 and <= MaxBulkIds } ? [.. ids.Distinct()] : null;
/// <summary>Tags as a request names them, trimmed, once each; null when one is blank or too long.</summary>
private static List<string>? Tags(IReadOnlyList<string>? tags)
{
if (tags is null) return [];
var list = new List<string>();
foreach (var tag in tags)
{
var trimmed = tag?.Trim();
if (string.IsNullOrEmpty(trimmed) || trimmed.Length > TagRepository.MaxLength) return null;
if (!list.Contains(trimmed, StringComparer.OrdinalIgnoreCase)) list.Add(trimmed);
}
return list.Count > SequenceEndpoints.MaxTags ? null : list;
}
private static BulkItemResult Skip(HttpContext context, Guid id, string code) => new(id, Skipped, code, ApiErrors.Message(context, code));
private static BulkResultDto Result(List<Guid> ids, Dictionary<Guid, BulkItemResult> items, bool dryRun, int photos, long bytes)
{
var ordered = ids.Select(id => items[id]).ToList();
var ok = ordered.Count(i => i.Outcome == Ok);
return new BulkResultDto(ordered, ok, ordered.Count - ok, dryRun, photos, bytes);
}
}
+5 -2
View File
@@ -25,7 +25,8 @@ public static class SequenceEndpoints
{
private const int MaxTitle = 200;
private const int MaxNotes = 2000;
private const int MaxTags = 50;
/// <summary>Tags a sequence may carry.</summary>
internal const int MaxTags = 50;
/// <summary>Listings of a sequence's frames read without a lock before one is read under its row lock (<see cref="SnapshotAsync"/>).</summary>
private const int SnapshotAttempts = 3;
@@ -238,9 +239,11 @@ public static class SequenceEndpoints
if (context.Request.Headers.IfNoneMatch == etag) return Results.StatusCode(StatusCodes.Status304NotModified);
var renders = await db.Renders.ListBySequenceAsync(id, ct);
// Tombstones change only with the frames (a deletion bumps the frames version), so the ETag covers them.
var removed = await db.FrameRemovals.ListBySequenceAsync(id, ct);
context.Response.Headers.ETag = etag;
return Results.Json(new SequenceManifest(id, sequence.FramesVersion, sequence.SettingsVersion, sequence.SettingsJson, sequence.Tags, frames,
[.. renders.Select(Dto.Render)], spec), ProtocolJson.Default.SequenceManifest);
[.. renders.Select(Dto.Render)], spec, removed.Count == 0 ? null : Data.Sync.SyncTombstones.ToDto(removed)), ProtocolJson.Default.SequenceManifest);
}
/// <summary>
+122
View File
@@ -0,0 +1,122 @@
using System.Text.RegularExpressions;
using Microsoft.AspNetCore.Mvc;
using Chronos.Core.Archiving;
using Chronos.Core.Model;
using Chronos.Data;
using Chronos.Data.Archiving;
using Chronos.Data.Jobs;
using Chronos.Protocol;
using Chronos.Server.Events;
using Chronos.Server.Jobs;
using Chronos.Server.Security;
using Chronos.Server.Storage;
namespace Chronos.Server.Api;
/// <summary>
/// <c>/api/v1/archive/trash</c> (owner review 2, R6; administrators of the archive): the entries of the archive's trash —
/// deleted sequences and photos deleted from sequences —, restoring one, and emptying it: a dry run that measures the
/// entries and fingerprints them, then an <c>empty-trash</c> job that erases exactly those entries and refuses when the
/// trash changed since (BRIEF §0.2 rule 8: the only erase of user files, always an explicit, confirmed action).
/// </summary>
public static partial class TrashEndpoints
{
public static void Map(RouteGroupBuilder api)
{
var trash = api.MapGroup("/archive/trash").Requires(Permissions.ManageArchive);
trash.MapGet("", ListAsync);
trash.MapPost("/{name}/restore", RestoreAsync);
trash.MapPost("/empty", EmptyAsync);
}
/// <summary>Whether <paramref name="name"/> has the shape of an entry name (a sequence id, a time, <c>photos-</c>): never a path.</summary>
public static bool IsEntryName(string? name) => name is not null && EntryName().IsMatch(name);
public static TrashEntryDto Entry(TrashEntry e) =>
new(e.Name, e.Kind.ToString().ToLowerInvariant(), e.SequenceId, e.Title, e.Folder, e.Frames, e.Files, e.Bytes, e.DeletedBy, e.DeletedAt);
/// <summary>The entries newest first with their total size; an archive offline lists none.</summary>
private static async Task<IResult> ListAsync(HttpContext context, ChronosDatabase db, ServerArchive archive, [FromServices] ArchiveService? archives)
{
var entries = await LibraryEndpoints.Engine(archives, db).Trash.ListAsync(archive.Current, context.RequestAborted);
return Results.Json(new TrashListDto([.. entries.Select(Entry)], entries.Sum(e => e.Bytes)), ProtocolJson.Default.TrashListDto);
}
/// <summary>
/// Brings an entry back: a sequence into its folder and the library, photos into their sequence (a new one when it is
/// gone), each photo re-hashed against its record first. 404 <c>not_found</c> (<c>trash_entry</c>); 409
/// <c>sequence_in_library</c> (that sequence is in the library already), <c>sequence_busy</c>, <c>folder_in_use</c>; 422
/// <c>trash_mismatch</c> (no file matches its record any more); 503 <c>archive_offline</c>. Publishes
/// <c>sequenceChanged</c>: <c>created</c> for a sequence, <c>frames</c> for photos.
/// </summary>
private static async Task<IResult> RestoreAsync(HttpContext context, Caller caller, string name, ChronosDatabase db, ServerArchive archive,
[FromServices] ArchiveService? archives, JobEngineHost host, AuditLog audit, EventHub events, IHostApplicationLifetime lifetime)
{
if (!IsEntryName(name)) return ApiErrors.NotFound(context, "trash_entry");
TrashRestoreResult result;
try
{
result = await LibraryEndpoints.Engine(archives, db).RestoreEntryAsync(archive.Id, name, host.Engine, lifetime.ApplicationStopping);
}
catch (DirectoryNotFoundException)
{
return ApiErrors.NotFound(context, "trash_entry");
}
catch (SequenceInLibraryException)
{
return ApiErrors.Conflict(context, "sequence_in_library");
}
catch (ArchiveOfflineException)
{
return ApiErrors.Result(context, StatusCodes.Status503ServiceUnavailable, "archive_offline");
}
catch (LeaseUnavailableException)
{
return ApiErrors.Conflict(context, "sequence_busy");
}
catch (InvalidDataException)
{
return ApiErrors.Result(context, StatusCodes.Status422UnprocessableEntity, "trash_mismatch");
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
return ApiErrors.Conflict(context, "folder_in_use");
}
var sequence = result.Sequence;
await audit.WriteAsync(caller, context, "trash_restored", "sequence", sequence.Id.ToString(), $"{name} · {sequence.DisplayTitle}");
events.Publish(new SequenceChangedEvent(sequence.Id, result.Kind == TrashKind.Sequence || result.NewSequence ? "created" : "frames",
sequence.FramesVersion, sequence.SettingsVersion));
return Results.Json(new TrashRestoreDto(name, result.Kind.ToString().ToLowerInvariant(), sequence.Id, result.Kind == TrashKind.Sequence ? sequence.FrameCount : result.Photos,
result.Kept, result.NewSequence), ProtocolJson.Default.TrashRestoreDto);
}
/// <summary>
/// Empties the trash, or the entries <c>names</c> names. With <c>dryRun</c> the plan: the entries measured on disk, their
/// totals and a fingerprint, nothing touched. Otherwise the request carries that fingerprint, and the server queues an
/// <c>empty-trash</c> job (202) that erases exactly those entries; the job fails, erasing nothing, when the trash changed
/// before it ran. 400 <c>bad_request</c> (<c>names</c>, <c>fingerprint</c>); 409 <c>trash_changed</c> (details: the
/// fingerprint now), <c>job_unavailable</c>.
/// </summary>
private static async Task<IResult> EmptyAsync(HttpContext context, Caller caller, TrashEmptyRequest request, ChronosDatabase db, ServerArchive archive,
[FromServices] ArchiveService? archives, JobEngineHost host, AuditLog audit)
{
var ct = context.RequestAborted;
if (request.Names is { } names && (names.Count == 0 || names.Any(n => !IsEntryName(n)))) return ApiErrors.BadRequest(context, "bad_request", "names");
var plan = await LibraryEndpoints.Engine(archives, db).PlanEmptyTrashAsync(archive.Id, request.Names, ct);
if (request.DryRun)
{
return Results.Json(new TrashEmptyPlanDto([.. plan.Entries.Select(Entry)], plan.Photos, plan.Files, plan.Bytes, plan.Fingerprint, plan.Unknown),
ProtocolJson.Default.TrashEmptyPlanDto);
}
if (string.IsNullOrWhiteSpace(request.Fingerprint)) return ApiErrors.BadRequest(context, "bad_request", "fingerprint");
if (plan.Fingerprint != request.Fingerprint || plan.Unknown.Count > 0) return ApiErrors.Conflict(context, "trash_changed", plan.Fingerprint);
if (!host.CanRun(JobKind.EmptyTrash)) return ApiErrors.Conflict(context, "job_unavailable", JobWire.Kind(JobKind.EmptyTrash));
var job = await host.Engine.EnqueueAsync(ArchiveJobs.EmptyTrash(archive.Current, plan), caller.Id, ct);
await audit.WriteAsync(caller, context, "trash_emptied", "job", job.Id.ToString(),
$"{plan.Entries.Count} · {plan.Bytes} · {string.Join(", ", plan.Entries.Take(5).Select(e => e.Name))}");
return Results.Json(Dto.Job(job), ProtocolJson.Default.JobDto, statusCode: StatusCodes.Status202Accepted);
}
[GeneratedRegex("^(photos-)?[0-9a-f]{32}(-[0-9]{14}(-[0-9]{1,6})?)?$", RegexOptions.CultureInvariant)]
private static partial Regex EntryName();
}
@@ -26,6 +26,7 @@ public sealed class ServerSettingsStore(ServerOptions options, ServerLogging log
[
JobKind.Render, JobKind.Import, JobKind.Scan, JobKind.Rebuild, JobKind.Verify, JobKind.LightAnalyse, JobKind.DeepAnalyse, JobKind.Thumbs,
JobKind.Proxies, JobKind.Maintenance, JobKind.Duplicates, JobKind.Orphans, JobKind.Adopt, JobKind.Move, JobKind.Backup, JobKind.Cleanup,
JobKind.EmptyTrash,
];
private readonly Lock _gate = new();
+2
View File
@@ -173,12 +173,14 @@ public static class ServerHost
AuthEndpoints.Map(api);
UserEndpoints.Map(api);
SequenceEndpoints.Map(api);
LibraryEndpoints.Map(api);
FileEndpoints.Map(api);
UploadEndpoints.Map(api);
JobEndpoints.Map(api);
ScheduleEndpoints.Map(api);
NodeEndpoints.Map(api);
ArchiveEndpoints.Map(api);
TrashEndpoints.Map(api);
AdminEndpoints.Map(api);
DiagnosticsEndpoints.Map(api);
EventEndpoints.Map(api);
@@ -56,6 +56,14 @@ public static class UploadRules
return intact.FirstOrDefault(f => f.SequenceId == target) ?? intact.FirstOrDefault();
}
/// <summary>
/// Whether <paramref name="target"/> tombstoned <paramref name="sha256"/> (owner review 2, R6): a photo deleted from that
/// sequence (or replaced by a converted file) that an upload must not bring back. The upload is answered as
/// <c>alreadyExists</c> without a frame, so a client neither sends the file nor treats the answer as a failure.
/// </summary>
public static async Task<bool> RemovedFromTargetAsync(ChronosDatabase db, string sha256, Guid? target, CancellationToken cancellationToken) =>
target is { } sequence && await db.FrameRemovals.IsRemovedAsync(sequence, sha256, cancellationToken).ConfigureAwait(false);
/// <summary>
/// The sequence an upload files into: its own, or the one its session key names (the user's own key without one);
/// null while that sequence does not exist yet.
+5 -1
View File
@@ -60,7 +60,11 @@ public sealed class UploadService(
var sha = request.Sha256.ToLowerInvariant();
var sessionKey = string.IsNullOrWhiteSpace(request.SessionKey) ? null : request.SessionKey.Trim();
var copies = await db.Frames.FindBySha256Async(sha, cancellationToken);
var target = copies.Count == 0 ? null : await UploadRules.TargetAsync(db, archive.Id, request.SequenceId, sessionKey, caller.Id, cancellationToken);
var target = await UploadRules.TargetAsync(db, archive.Id, request.SequenceId, sessionKey, caller.Id, cancellationToken);
// A photo deleted from the target sequence is not filed into it again: an old client syncing a copy that still holds
// it, or a watch folder sending it again, hears that the server has it (owner review 2, R6). A restore brings it back.
if (await UploadRules.RemovedFromTargetAsync(db, sha, target, cancellationToken))
return new StartUploadResponse(null, options.ChunkSizeBytes, [], AlreadyExists: true, null);
// Only a copy that is still whole stands for the upload: a client may delete its file on this answer.
if (UploadRules.StandIn(archive.Root, copies, target) is { } known)
return new StartUploadResponse(null, options.ChunkSizeBytes, [], AlreadyExists: true, known.Id);