fix(sync,upload): keep tombstones honest and let an explicit upload bring a photo back

Sync:
- A server frame that has not been hashed is known by content only once it is
  downloaded: one this PC deleted (or holds as its replacement) is now deleted
  from staging instead of filed again, which also dropped its tombstone.
- An unhashed local original that turns out to be a photo the server deleted
  is not uploaded, and the archive verify then writes the hashes into the
  index, so the row settles instead of offering Upload for good.
- A tombstone leaves a photo out of the comparison only when its side no longer
  holds that content, so a tombstone left beside a held copy does not keep the
  other side's copy missing forever.

Uploads:
- StartUploadRequest.Restore (optional, never sent by sync or older clients)
  lets an explicit user upload file a photo deleted from the sequence again,
  also after the trash was emptied; the completion repeats it with
  ?restore=true and the ingest drops the tombstone.
- The completion checks the tombstone again: a photo deleted while its bytes
  were on the way is not filed (409 photo_removed, upload cancelled); the
  protocol client reports that as not sent.
- A tombstone of content a frame of the target still holds refuses no repair.
- Hidden dot names (AppleDouble "._" files) are refused, as the archive
  scanner leaves them out.

Web upload dialog:
- Upload again on rows marked Deleted before; the tooltip no longer promises
  a trash restore that may be impossible.
- Sends a capture time: the Exif DateTimeOriginal of a JPEG or TIFF-based file
  (minimal in-house reader, capture-time.js), else the file's time, so photos
  without Exif keep their order.
- Esc is stopped at keydown during a run (Chromium allows one prevented cancel
  per activation); hidden dot files are left out; a completion refused while
  another one files the upload waits for it; the same photo picked under two
  names waits for its first copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit d051bcfc351f5a08500ca864a6508576a95492a4)
This commit is contained in:
Alby96 committed 2026-10-27 12:45:33 +01:00
1 parent eb607d47aa
commit 316c182b19
18 files changed
+753 -63

No files matched your search

@@ -1,4 +1,5 @@
using System.Net;
using System.Net.Http.Json;
using System.Security.Cryptography;
using Microsoft.Extensions.DependencyInjection;
using Chronos.Core.Archiving;
@@ -419,6 +420,59 @@ public sealed class ArchiveModuleTests(TestContext context)
Assert.Equal(photo.Length, (await me.ReadAsync(ProtocolJson.Default.UserDto)).UsedBytes);
}
[Test]
public async Task TheProtocolClientTakesAPhotoDeletedWhileOnItsWayAsNotSent()
{
// Owner review 2, R6: a repair upload whose photo the Library deletes meanwhile is refused at its completion
// (photo_removed, nothing filed); the protocol client reports it like a photo deleted before: null, not a failure.
await using var server = await TestServer.StartAsync(context, services: ArchiveModule.Register);
var (_, _, token) = await server.UserAsync("alberto", "editor");
var admin = await server.AdminAsync();
var path = Path.Combine(context.CreateTempDirectory("source"), "IMG_0001.JPG");
var photo = Photo(5);
await File.WriteAllBytesAsync(path, photo);
using var client = new ChronosClient(server.HttpBase, new CertificatePinning([]), "Chronos Uploader (tests)") { Token = token };
var sequence = await client.CreateSequenceAsync(new CreateSequenceRequest("Roof", null), context.Cancellation);
// A sequence keeps one photo at least: the other is the one deleted.
var keeper = Path.Combine(Path.GetDirectoryName(path)!, "IMG_0000.JPG");
await File.WriteAllBytesAsync(keeper, Photo(6));
var kept = Assert.NotNull(await client.UploadFileAsync(keeper, sequence.Id, null, ct: context.Cancellation));
var first = Assert.NotNull(await client.UploadFileAsync(path, sequence.Id, null, ct: context.Cancellation));
await UntilIdleAsync(server);
File.Delete(Path.Combine(server.Options.ArchivePath, Assert.NotNull(await server.Database.Frames.GetAsync(first.FrameId)).RelativePath));
using var deleting = new BeforeCompletion(async () =>
{
await UntilIdleAsync(server);
var current = Assert.NotNull(await server.Database.Sequences.GetAsync(sequence.Id));
using var delete = await admin.PostAsJsonAsync($"/api/v1/sequences/{sequence.Id}/frames/delete",
new FrameDeleteRequest([first.FrameId], current.FramesVersion, false), ProtocolJson.Default.FrameDeleteRequest);
await delete.ExpectAsync(HttpStatusCode.OK);
});
using var racing = new ChronosClient(server.HttpBase, new CertificatePinning([]), "Chronos Uploader (tests)", deleting) { Token = token };
Assert.Null(await racing.UploadFileAsync(path, sequence.Id, null, ct: context.Cancellation));
Assert.True(deleting.Ran, "the completion was reached");
Assert.SequenceEqual([kept.FrameId], (await server.Database.Frames.ListBySequenceAsync(sequence.Id)).Select(f => f.Id));
Assert.True(await server.Database.FrameRemovals.IsRemovedAsync(sequence.Id, Convert.ToHexStringLower(SHA256.HashData(photo))), "the tombstone stays");
}
/// <summary>Runs a step on the server before the first upload completion goes through, then sends it on.</summary>
private sealed class BeforeCompletion(Func<Task> step) : DelegatingHandler(new SocketsHttpHandler())
{
public bool Ran { get; private set; }
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
if (!Ran && request.Method == HttpMethod.Post && request.RequestUri?.AbsolutePath.EndsWith("/complete", StringComparison.Ordinal) == true)
{
Ran = true;
await step();
}
return await base.SendAsync(request, cancellationToken);
}
}
[Test]
public async Task AnIntactCopyElsewhereDoesNotStandForTheDamagedOneOfTheTargetSequence()
{