Files
Chronos/src/Chronos.Data/Workspaces/WorkspaceSync.cs
T
Alby96andClaude Opus 5.5 26bab759fe feat(app): workspace projects whose home is a server, pushed and pulled like this PC's archive
Owner review 2, R8, phase 5b (client part), ported onto the final phase 5a.
A Chronos Server with the "projects" capability is now the home of
workspace projects. A server sequence is edited in the workspace like a
local one, not inside the library, and its versions sync with the server
"like git".

- Data: ServerProjectHome is an IProjectHome over ChronosClient.
  - These are HomeUnavailableException, so the workspace shows Offline: a
    server out of reach, an expired sign-in, a 503 (lease, offline archive,
    projects_unreadable), a 409 folder_in_use, and a server without projects.
  - 404 sequence/project/version and refused names are ProjectException;
    other refusals keep their meaning.
  - A deleted project's head is 404 project, so final 5a's HeadAsync NotFound
    rule reads it as Gone. The original 5b's IsLiveAsync and
    WorkspaceStatus.Gone are dropped. IProjectHome gains rename and delete.
- Data: WorkspaceJobs.CopyFromServer downloads into the workspace, through
  the desktop's session, the proxies of one frames version (or the
  thumbnails standing in for them), with the aliases of the server's
  conversions. It is rebuilt on final 5a's copy handler:
  - the copy:frame:fv:folder checkpoint, EnsureSpace (from the server's proxy
    sizes), WritingMedia, and the cancelled-copy clean-up;
  - the editor's current file is never overwritten;
  - each file is hashed as it arrives and checked on disk, four at a time.
  It never writes this PC's archive. Originals are still refused until 5c.
- Data: SyncAgent passes on the server's projectChanged events, and after a
  lost event stream it reports that changes may have been missed.
- App: SyncHost gives the server home over one session client. The Server
  screen offers Open in workspace, disabled with its reason on older
  servers, whose sequences keep opening with proxies.
- App: Edit opens or checks out server projects with final 5a's refusals:
  unreadable copies, a disabled damaged or newer Main, a refused folder. The
  "sequence left the index" refusal now applies only to projects of this
  PC's archive.
- App: the picker and the chip follow projectChanged (Server newer, Not on
  server). Every state, conflict, delete, remove and failure text uses its
  ".Server" form when one exists (WorkspaceText.ForHome). That includes 5a's
  newer-format conflict, the save conflict, the taken-values toast, Gone and
  version-too-large.
- App: a render of a server project first offers to send what is not on the
  server, then renders the version the server holds. Offline, Gone, Damaged
  and missing media hold the render. Analyses and renders run as server jobs
  (ServerJobs, shared with RemoteSequenceSource).
- Pruning keeps final 5a's rule: a version is pruned only after HoldsAsync
  reads it back.
- Strings in en and it; ARCHITECTURE, DECISIONS and CHANGELOG.

Tests:
- ServerProjectHome error mapping, using a fake handler.
- Against the in-process server:
  - two workspaces on one project: fast-forward, replay and merge, keep mine,
    take server, save both, restore;
  - the copy job verifying, resuming and following new frames;
  - a project deleted on the server (Gone from its head; a push sends
    nothing);
  - an offline server.
- SyncAgent project events.
- App tests for the host, the project menu and renders of a server home.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 11:45:06 +01:00

1527 lines
98 KiB
C#

using System.Globalization;
using System.Text.Json;
using Chronos.Core.Archiving;
using Chronos.Core.Diagnostics;
using Chronos.Core.Model;
using Chronos.Core.Project;
using Chronos.Data.Projects;
using Chronos.Data.Sync;
namespace Chronos.Data.Workspaces;
/// <summary>Where a workspace project stands against its home (<see cref="WorkspaceStatus.State"/>), the most pressing first.</summary>
public enum WorkspaceState
{
/// <summary>Nothing to save, push or pull.</summary>
InSync,
/// <summary>The working state has changes no version holds yet (save a version).</summary>
Modified,
/// <summary>Versions saved here wait to be pushed (<see cref="WorkspaceStatus.Ahead"/>).</summary>
Ahead,
/// <summary>The home has versions this workspace has not pulled (<see cref="WorkspaceStatus.Behind"/>).</summary>
Behind,
/// <summary>A pull or push met changes both sides made to the same settings: keep mine, take the server's or save both.</summary>
Conflict,
/// <summary>The home cannot be reached now; work goes on here and syncs later.</summary>
Offline,
/// <summary>The workspace holds no media for the project (never copied, or removed): copy them again to edit.</summary>
MediaRemoved,
/// <summary>
/// The home answered that it no longer has the project or its sequence (the sequence left its index): nothing more can be
/// sent, and the local copy can only be removed, its work kept aside. Told by a status asked of the home, and by a sync
/// or a choice that met it (nothing here is changed by it).
/// </summary>
Gone,
/// <summary>
/// A version saved here cannot be read, or no longer follows from the base (<see cref="WorkspaceStatus.Damaged"/>):
/// nothing is saved, sent or taken while it is so, which would leave it and the versions after it out of the chain.
/// One the home holds takes the home's copy at the next sync.
/// </summary>
Damaged,
}
/// <summary>
/// A workspace project's state (owner review 2, R8): the leading <see cref="State"/> and every fact behind it. Without a
/// home to ask, <see cref="Behind"/> is 0 and <see cref="Offline"/> and <see cref="Gone"/> false.
/// </summary>
/// <param name="Base">The home's version the project was last in step with.</param>
/// <param name="HomeHead">The home's newest version, when it was asked.</param>
/// <param name="Gone">The home was asked and no longer has the project (<see cref="WorkspaceState.Gone"/>).</param>
public sealed record WorkspaceStatus(WorkspaceState State, bool Modified, int Ahead, int Behind, bool Offline, bool MediaRemoved, bool ReadOnly,
WorkspaceConflict? Conflict, BundleBase? Base, ProjectVersion? HomeHead)
{
/// <summary>
/// The media are behind the home's frames, as the home was asked: frames added, removed or converted there since the copy
/// (Update media copies them again), or photos the home names otherwise now (hashed, or joined to another encoding,
/// there since) whose names the media could not learn yet because another operation held the project. Every sync
/// teaches the media the home's names; false without a home asked.
/// </summary>
public bool MediaOutdated { get; init; }
/// <summary>Version files after the base that cannot be read or do not follow from it (<see cref="WorkspaceState.Damaged"/>).</summary>
public int Damaged { get; init; }
}
/// <summary>What a push, pull or resolve did: the versions it stored at the home and those it took from there, and where it left the project.</summary>
/// <param name="SavedAs">After <see cref="ConflictChoice.SaveBoth"/>: the new project at the home holding this workspace's versions.</param>
public sealed record WorkspaceSyncResult(WorkspaceStatus Status, IReadOnlyList<ProjectVersion> Pushed, IReadOnlyList<ProjectVersion> Pulled,
SequenceProject? SavedAs = null)
{
/// <summary>
/// After <see cref="ConflictChoice.TakeServer"/> settled a conflict: the working state as keeping this side's values on
/// the settings both changed would have left it (positions on the media's frames), so the editor takes the server's
/// values as one step its undo takes back, and only them; null otherwise.
/// </summary>
public ProjectSettings? Kept { get; init; }
}
/// <summary>How a conflict is settled (owner review 2, R8: only on real conflicts).</summary>
public enum ConflictChoice
{
/// <summary>This workspace's values where both changed (each member listed in <c>takeServer</c> takes the server's instead).</summary>
KeepMine,
/// <summary>The server's values wherever both changed.</summary>
TakeServer,
/// <summary>Both kept: this workspace's versions go to a new project at the home, and this project takes the server's.</summary>
SaveBoth,
}
/// <summary>The working state of a workspace project opened for editing.</summary>
/// <param name="Working">The settings, their positions placed on the media's frames.</param>
/// <param name="Sequence">The sequence over the media's files; null while the workspace holds no media.</param>
public sealed record WorkspaceSession(WorkspaceProject Project, ProjectBundle Bundle, ProjectSettings Working, WorkspaceMedia? Media, WorkspaceSequence? Sequence)
{
/// <summary>The settings come from a newer Chronos: shown, never saved.</summary>
public bool IsReadOnly => Bundle.IsReadOnly;
}
/// <summary>
/// A push the home refused for good (not a divergence: settings damaged on the way, of a newer format, larger than a
/// version may be (<see cref="PushOutcome.TooLarge"/>), or not this project's).
/// </summary>
public sealed class WorkspaceSyncException(PushOutcome outcome, string message) : IOException(message)
{
public PushOutcome Outcome { get; } = outcome;
}
/// <summary>
/// Sync of workspace projects with their homes, "like git" with one linear history and no branches (owner review 2, R8).
/// A workspace edits the working state (<see cref="SaveAsync"/>), saves versions of it locally (<see cref="CommitAsync"/>,
/// parent = the last one), pushes them in order (<see cref="PushAsync"/>) and pulls the home's (<see cref="PullAsync"/>).
/// When the home moved meanwhile, its new versions are pulled and each local commit is replayed on top of them
/// (<see cref="ProjectMerge.Replay"/>, new ids, the old files kept in <c>versions/.replaced</c>), and the working state is
/// merged too (<see cref="ProjectMerge.MergeWithConflicts"/>); settings both sides changed to different values stop
/// there as a conflict (<see cref="WorkspaceConflict"/>) for the user to settle (<see cref="ResolveAsync"/>), a choice
/// applying only to the conflicts the user was shown. Versions travel in the portable form (<see cref="ProjectContent"/>),
/// so frames inserted at the home never move a position; before a merge the media learn how the home names its photos now
/// (hashed or converted there since the copy), so every side names one photo alike. Settings of a newer Chronos are never
/// merged with this one's: a project with nothing of its own moves onto them read-only, any other waits on a conflict of
/// format that only Save both settles (<see cref="WorkspaceConflict.NewerFormat"/>). Every operation holds the project's
/// lease; nothing is ever deleted: a restore or a save-both first saves uncommitted changes as a version, and taking the
/// server's values keeps the working state they replace aside (<c>versions/.replaced</c>). A conflict lists this side's
/// values as the working state holds them, so it is asked and settled against what the user sees.
/// Every write sequence survives a cut at any step: the working state's base is the commit point. A push moves the base
/// before its local file reads as sent; a replay journals itself in <c>sync.json</c> (<see cref="WorkspaceReplay"/>), writes
/// the replayed versions, then the base, and only then sets the old ones aside; a save-both journals its copy and the ids
/// it sends under (<see cref="WorkspaceSaveBoth"/>). The next push, pull or resolve finishes or undoes what a cut left
/// (<see cref="SettleAsync"/>). A local version after the base that cannot be read, or does not follow from it, is never
/// taken for one that is not there: nothing is saved, sent or taken past it (<see cref="WorkspaceState.Damaged"/>) until it
/// reads again or takes the home's copy.
/// </summary>
public sealed class WorkspaceSync(WorkspaceStore store, string clientName)
{
/// <summary>Versions asked for at once when pulling.</summary>
private const int PullPage = 100;
/// <summary>Times a push pulls and replays when the home keeps moving under it.</summary>
private const int PushAttempts = 3;
private static readonly FramePhotos NoFrames = new([]);
public WorkspaceStore Store => store;
/// <summary>How long an operation waits for the project's lease (another program working on it).</summary>
public TimeSpan LockWait { get; init; } = TimeSpan.FromSeconds(30);
/// <summary>For tests: called with a step's name between the writes of a sync, where it may throw as a crash would.</summary>
internal Action<string>? Cut { get; init; }
private void Step(string name) => Cut?.Invoke(name);
/// <summary>
/// The project <paramref name="projectId"/> of <paramref name="home"/> brought into the workspace: its folder, the home's
/// head as the base and the working state (positions waiting for the media, which a
/// <see cref="JobKind.WorkspaceCopy"/> job copies: <see cref="WorkspaceJobs.Copy"/>). A project already here is left as
/// it is, but for its media's names of the photos (<see cref="WorkspaceMedia.NamedAsAt"/>).
/// </summary>
/// <exception cref="ProjectException">The home has no such project.</exception>
/// <exception cref="InvalidOperationException">A project of that id from another home is in the workspace.</exception>
public async Task<WorkspaceProject> CheckoutAsync(IProjectHome home, Guid projectId, string media = BundleMedia.Proxies, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(home);
var project = store.Project(projectId);
using (await project.LockAsync(LockWait, cancellationToken).ConfigureAwait(false))
{
var sequence = await home.SequenceAsync(cancellationToken).ConfigureAwait(false);
if (project.Exists)
{
var existing = project.ReadBundle();
if (existing.Workspace!.Home.SequenceId != sequence.Home.SequenceId)
throw new InvalidOperationException($"Project {projectId} in the workspace belongs to another sequence.");
// Opened again: its media learn how the home names its photos now, as a sync's do (damaged ones are copied again).
try
{
await MediaNamedAsAtHomeAsync(project, home, cancellationToken).ConfigureAwait(false);
}
catch (InvalidDataException ex)
{
Log.Warning("workspace", $"The media index of workspace project {projectId} cannot be read", ex);
}
return project;
}
var listed = (await home.ListAsync(cancellationToken).ConfigureAwait(false)).FirstOrDefault(p => p.Id == projectId)
?? throw new ProjectException(ProjectError.NotFound, $"The home has no project {projectId}.");
var head = await home.HeadAsync(projectId, cancellationToken).ConfigureAwait(false);
Verify(head);
var bundle = new ProjectBundle(sequence.Archive, sequence.Sequence, ProjectSettings.FromJson(head.SettingsJson, allowNewer: true), DateTimeOffset.UtcNow)
{
Workspace = new BundleWorkspace(projectId, listed.Name, listed.IsMain, sequence.Home, Base(head), new BundleMedia(media, 0)),
};
// The working state first: one larger than a project file holds is refused before anything is written
// (ProjectTooLargeException), never left as a file no later open could read. Its base is fetched again if a cut
// comes before the version's own file.
project.WriteBundle(bundle);
project.WriteVersion(head, replace: true);
Directory.CreateDirectory(project.ExportsDirectory);
return project;
}
}
/// <summary>
/// The working state for the editor: its settings placed on the media's frames, and the media's sequence. Both files are
/// read under the project's lease, as one state: a copy committing new media rewrites them in turn.
/// </summary>
/// <exception cref="InvalidOperationException">The project is not in the workspace.</exception>
/// <exception cref="Core.Archiving.LeaseUnavailableException">Another writer kept the project for longer than <see cref="LockWait"/>.</exception>
public async Task<WorkspaceSession> OpenAsync(Guid projectId, CancellationToken cancellationToken = default)
{
var project = Existing(projectId);
ProjectBundle bundle;
WorkspaceMedia? media;
using (await project.LockAsync(LockWait, cancellationToken).ConfigureAwait(false))
(bundle, media) = await Task.Run(() => (project.ReadBundle(), project.ReadMedia()), cancellationToken).ConfigureAwait(false);
return await Task.Run(() =>
{
var working = bundle.Project;
if (media is not null && !working.IsNewerFormat)
working = ProjectSettings.FromJson(ProjectContent.Place(working.ToJson(), media.Photos), allowNewer: true);
return new WorkspaceSession(project, bundle, working, media, media is null ? null : SequenceOf(project, bundle, media));
}, cancellationToken).ConfigureAwait(false);
}
/// <summary>Stores the working state (the editor's settings, positions on the media's frames); no version is made.</summary>
/// <exception cref="InvalidOperationException">The project is read-only (a newer Chronos's settings) or not in the workspace.</exception>
public async Task SaveAsync(Guid projectId, ProjectSettings working, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(working);
var project = Existing(projectId);
using (await project.LockAsync(LockWait, cancellationToken).ConfigureAwait(false))
{
var bundle = project.ReadBundle();
if (bundle.IsReadOnly || working.IsNewerFormat) throw ReadOnly(projectId);
project.WriteBundle(bundle with { Project = working, Saved = DateTimeOffset.UtcNow });
}
}
/// <summary>
/// Saves the working state as a new local version (parent = the last one), unless it is the last one's already: null
/// then. The version waits here until pushed. After a restore it is a "restore" of the version brought back.
/// </summary>
/// <exception cref="InvalidOperationException">The project is read-only, has a conflict to settle, or is not in the workspace.</exception>
public async Task<ProjectVersion?> CommitAsync(Guid projectId, string? note = null, string? author = null, CancellationToken cancellationToken = default)
{
var project = Existing(projectId);
using (await project.LockAsync(LockWait, cancellationToken).ConfigureAwait(false))
{
if (project.ReadSync().Conflict is not null) throw new InvalidOperationException($"Project {projectId} has a conflict to settle first.");
return CommitLocked(project, project.ReadBundle(), note, author);
}
}
/// <summary>
/// Pushes the local versions in order. When the home moved meanwhile, its versions are pulled and the local ones
/// replayed on top of them first; a real conflict stops the push (<see cref="WorkspaceState.Conflict"/>). A project the
/// home no longer has reads as <see cref="WorkspaceState.Gone"/>, its versions kept here. A push that ended well prunes
/// <c>versions/.replaced</c> (<see cref="WorkspaceProject.PruneReplaced"/>).
/// </summary>
/// <exception cref="HomeUnavailableException">The home cannot be reached; nothing was lost, push again later.</exception>
/// <exception cref="WorkspaceSyncException">The home refused a version for good.</exception>
public async Task<WorkspaceSyncResult> PushAsync(Guid projectId, IProjectHome home, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(home);
var project = Existing(projectId);
using (await project.LockAsync(LockWait, cancellationToken).ConfigureAwait(false))
{
var pushed = new List<ProjectVersion>();
var pulled = new List<ProjectVersion>();
// Ended well: every local version sent, no conflict met, no attempt left over.
var sent = false;
try
{
await SettleAsync(project, home, cancellationToken).ConfigureAwait(false);
for (var attempt = 0; attempt < PushAttempts; attempt++)
{
var bundle = project.ReadBundle();
if (project.ReadSync().Conflict is not null) break;
var diverged = false;
var locals = Unpushed(project, bundle);
EnsureChained(project, bundle, locals);
foreach (var local in locals)
{
var result = await home.PushAsync(projectId, local, cancellationToken).ConfigureAwait(false);
switch (result.Outcome)
{
case PushOutcome.Stored or PushOutcome.AlreadyStored when result.Version is { } stored:
Verify(stored);
// The base moves first: cut after it, the local file still reads as unsent and the next sync
// adopts it; cut before it, the home answers "already stored" next time. Marked sent first, it
// would leave the chain from the old base and take the versions after it along.
bundle = WithBase(bundle, stored);
project.WriteBundle(bundle);
Step("push.base");
Adopt(project, local, stored);
pushed.Add(stored);
continue;
case PushOutcome.Diverged:
diverged = true;
break;
default:
throw Refused(result.Outcome, local, $"project {projectId}");
}
break;
}
if (!diverged)
{
sent = true;
break;
}
var (_, conflict, taken) = await ReplayAsync(project, bundle, home, null, cancellationToken).ConfigureAwait(false);
pulled.AddRange(taken);
if (conflict is not null) break;
}
}
catch (ProjectException ex) when (NotFoundAtHome(ex))
{
// Deleted at the home: no failure to retry but the state the project is in (Gone); nothing here moves.
if (!await GoneAsync(project, home, ex, cancellationToken).ConfigureAwait(false)) throw;
return new WorkspaceSyncResult(await StatusLockedAsync(project, home, cancellationToken).ConfigureAwait(false), pushed, pulled);
}
// A push stopped at a conflict, or by a home that kept moving, leaves versions/.replaced as it is.
if (sent && project.ReadSync().Conflict is null) PruneReplaced(project);
return new WorkspaceSyncResult(await StatusLockedAsync(project, home, cancellationToken).ConfigureAwait(false), pushed, pulled);
}
}
/// <summary>
/// Takes the home's versions newer than the base: the working state (and any local versions, replayed) moves onto the
/// home's head; a real conflict stops it (<see cref="WorkspaceState.Conflict"/>). A conflict recorded before is worked
/// out again (the home may have moved on, this side may have been edited since), so what is asked is what differs now;
/// one that no longer conflicts is settled by the merge. A project the home no longer has reads as
/// <see cref="WorkspaceState.Gone"/>, its copy here left as it is.
/// </summary>
/// <exception cref="HomeUnavailableException">The home cannot be reached.</exception>
public async Task<WorkspaceSyncResult> PullAsync(Guid projectId, IProjectHome home, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(home);
var project = Existing(projectId);
using (await project.LockAsync(LockWait, cancellationToken).ConfigureAwait(false))
{
IReadOnlyList<ProjectVersion> pulled = [];
try
{
await SettleAsync(project, home, cancellationToken).ConfigureAwait(false);
var bundle = project.ReadBundle();
EnsureChained(project, bundle, Unpushed(project, bundle));
if (project.ReadSync().Conflict is not null)
{
await EnsureAtHomeAsync(project, home, cancellationToken).ConfigureAwait(false);
(_, var conflict, pulled) = await ReplayAsync(project, bundle, home, null, cancellationToken).ConfigureAwait(false);
if (conflict is null) project.WriteSync(project.ReadSync() with { Conflict = null });
}
else
{
var head = await home.HeadAsync(projectId, cancellationToken).ConfigureAwait(false);
if (head.Id != bundle.Workspace!.Base?.VersionId) (_, _, pulled) = await ReplayAsync(project, bundle, home, null, cancellationToken).ConfigureAwait(false);
}
}
catch (ProjectException ex) when (NotFoundAtHome(ex))
{
if (!await GoneAsync(project, home, ex, cancellationToken).ConfigureAwait(false)) throw;
return new WorkspaceSyncResult(await StatusLockedAsync(project, home, cancellationToken).ConfigureAwait(false), [], pulled);
}
if (project.ReadSync().Conflict is null) PruneReplaced(project);
return new WorkspaceSyncResult(await StatusLockedAsync(project, home, cancellationToken).ConfigureAwait(false), [], pulled);
}
}
/// <summary>
/// Settles the conflict a pull or push stopped at, as the user was shown it (<paramref name="shown"/>, else the one
/// recorded). <see cref="ConflictChoice.KeepMine"/> keeps this workspace's values (but for the members in
/// <paramref name="takeServer"/>), <see cref="ConflictChoice.TakeServer"/> takes the server's on the settings listed;
/// either way the local versions are replayed on the home's head and wait to be pushed (one the choice leaves changing
/// nothing is not: its file is kept in <c>versions/.replaced</c>). A choice applies only to the conflicts listed: when
/// what differs now is not that (the home moved on, or this side was edited meanwhile), nothing is settled, and the
/// conflict found now is recorded and returned to be asked again; so is one the home holds no versions for now (nothing
/// to take them from). A choice cut short past its commit point was applied: the conflict is over, and a later choice
/// finds nothing to settle. Taking the server's keeps the working state it replaces as a version file in
/// <c>versions/.replaced</c>, outside the chain: nothing is committed, replayed or sent for it, so a take that leaves
/// only the server's values sends nothing. After a Save both cut short, either choice first sends the rest of this
/// workspace's versions to the copy it made (one somebody else pushed to since is left as it is).
/// <see cref="ConflictChoice.SaveBoth"/> saves the working state as a version, sends this workspace's versions to a new
/// project at the home (<paramref name="saveBothName"/>, else this project's name with a number) made from the base,
/// and moves this project onto the home's head; a project deleted at the home meanwhile keeps its versions here, the
/// copy holding them too (<see cref="WorkspaceState.Gone"/>). Keep mine and Take the server's on a project deleted at the
/// home change nothing here (it reads as Gone): nothing is merged onto the last version it had there.
/// </summary>
/// <exception cref="HomeUnavailableException">The home cannot be reached; the conflict stays.</exception>
public async Task<WorkspaceSyncResult> ResolveAsync(Guid projectId, IProjectHome home, ConflictChoice choice, IReadOnlySet<string>? takeServer = null,
string? saveBothName = null, WorkspaceConflict? shown = null, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(home);
var project = Existing(projectId);
using (await project.LockAsync(LockWait, cancellationToken).ConfigureAwait(false))
{
try
{
await SettleAsync(project, home, cancellationToken).ConfigureAwait(false);
var bundle = project.ReadBundle();
var sync = project.ReadSync();
if (sync.Conflict is not { } recorded) return new WorkspaceSyncResult(await StatusLockedAsync(project, home, cancellationToken).ConfigureAwait(false), [], []);
EnsureChained(project, bundle, Unpushed(project, bundle));
if (choice == ConflictChoice.SaveBoth)
return await SaveBothAsync(project, bundle, home, saveBothName, cancellationToken).ConfigureAwait(false);
// Never merged onto the last version of a project deleted at the home: it reads as Gone, everything here kept.
await EnsureAtHomeAsync(project, home, cancellationToken).ConfigureAwait(false);
// Nothing is committed first: a version of the working state would be replayed into one that only repeats the
// head, and sent. What the take replaces of the working state is kept aside by the replay instead.
IReadOnlySet<string> takeTheirs = choice == ConflictChoice.TakeServer ? AllPaths : takeServer ?? new HashSet<string>();
ProjectSettings? kept = null;
var (_, conflict, pulled) = await ReplayAsync(project, bundle, home, takeTheirs, cancellationToken, shown ?? recorded, k => kept = k).ConfigureAwait(false);
if (conflict is null) project.WriteSync(project.ReadSync() with { Conflict = null });
return new WorkspaceSyncResult(await StatusLockedAsync(project, home, cancellationToken).ConfigureAwait(false), [], pulled)
{
Kept = conflict is null ? kept : null,
};
}
catch (ProjectException ex) when (NotFoundAtHome(ex))
{
if (!await GoneAsync(project, home, ex, cancellationToken).ConfigureAwait(false)) throw;
return new WorkspaceSyncResult(await StatusLockedAsync(project, home, cancellationToken).ConfigureAwait(false), [], []);
}
}
}
/// <summary>
/// Brings version <paramref name="versionId"/> back as the working state (from this workspace, else from
/// <paramref name="home"/>); the next version saved is a "restore" of it while the working state still holds it.
/// Uncommitted changes are saved as a version first, also while a conflict waits (settling it replays that version
/// too), so a restore loses nothing.
/// </summary>
/// <exception cref="ProjectException">Neither this workspace nor the home has the version.</exception>
public async Task<ProjectSettings> RestoreAsync(Guid projectId, Guid versionId, IProjectHome? home = null, CancellationToken cancellationToken = default)
{
var project = Existing(projectId);
using (await project.LockAsync(LockWait, cancellationToken).ConfigureAwait(false))
{
var version = project.Versions().FirstOrDefault(v => v.Id == versionId);
if (version is null && home is not null && await home.GetVersionAsync(projectId, versionId, cancellationToken).ConfigureAwait(false) is { } fetched)
{
Verify(fetched);
project.WriteVersion(fetched, replace: true);
version = fetched;
}
if (version is null) throw new ProjectException(ProjectError.NotFound, $"Version {versionId} of project {projectId} is not known here.");
var bundle = project.ReadBundle();
if (bundle.IsReadOnly) throw ReadOnly(projectId);
CommitLocked(project, bundle, null, null);
var media = project.ReadMedia();
var photos = media?.Photos ?? NoFrames;
var working = ProjectSettings.FromJson(ProjectContent.Place(version.SettingsJson, photos), allowNewer: true);
bundle = project.ReadBundle();
project.WriteBundle(bundle with { Project = working, Saved = DateTimeOffset.UtcNow });
// Restoring what the project already is changes nothing: the next version is no restore.
if (Tip(project, bundle, Unpushed(project, bundle)) is not { } tip || !ProjectContent.SameChoices(tip.SettingsJson, version.SettingsJson, photos))
project.WriteSync(project.ReadSync() with { RestoredFrom = versionId });
return working;
}
}
/// <summary>
/// Where the project stands; with a <paramref name="home"/>, against the home's head too (offline when it cannot be
/// reached), the media naming the photos as the home does now (written when the project's lease is free).
/// </summary>
public async Task<WorkspaceStatus> StatusAsync(Guid projectId, IProjectHome? home = null, CancellationToken cancellationToken = default)
{
var project = Existing(projectId);
return await StatusLockedAsync(project, home, cancellationToken, leased: false).ConfigureAwait(false);
}
/// <summary>The versions this workspace saved and has not pushed yet, oldest first.</summary>
public IReadOnlyList<ProjectVersion> Unpushed(Guid projectId)
{
var project = Existing(projectId);
return Unpushed(project, project.ReadBundle());
}
private static readonly IReadOnlySet<string> AllPaths = new EveryPath();
/// <summary>
/// Pulls the home's versions after the base and replays the local ones and the working state on its head. With
/// <paramref name="takeTheirs"/> null a conflict is recorded and nothing else moves (the versions pulled are kept as
/// files; the base only moves onto local commits the home already holds); with a choice the conflicts are settled by it,
/// only when they are those <paramref name="shown"/> lists (others are recorded and asked about first, as without a
/// choice). Taking the server's everywhere tells <paramref name="kept"/> the working state as keeping this side's values
/// would have left it (placed), and the working state it replaces is kept aside (<c>versions/.replaced</c>), never in the
/// chain. A conflict lists this side's value as the working state holds it, the base value it went back to included; where
/// the working state holds the home's value already there is nothing to ask, and the replayed versions take that value
/// too. A conflict recorded before stays when the home holds no version after the base (it lost them: nothing to
/// settle it against).
/// Written in steps a cut can stop at any point (<see cref="WorkspaceReplay"/>): the journal, the replayed versions, the
/// base with the merged working state (the commit point), then the old versions set aside and the held ones adopted.
/// Before the commit point the rest of a Save both cut short goes to its copy (<see cref="FinishCopyAsync"/>).
/// </summary>
private async Task<(ProjectBundle Bundle, WorkspaceConflict? Conflict, IReadOnlyList<ProjectVersion> Pulled)> ReplayAsync(WorkspaceProject project,
ProjectBundle bundle, IProjectHome home, IReadOnlySet<string>? takeTheirs, CancellationToken cancellationToken, WorkspaceConflict? shown = null,
Action<ProjectSettings>? kept = null)
{
var basis = await BaseVersionAsync(project, bundle, home, cancellationToken).ConfigureAwait(false);
// A local commit the home stored although its answer was lost comes back among the home's versions, and the commits
// after it were made on it. Its file keeps reading as unsent until the base has moved onto it or past it: written
// over now, it would leave the chain from the base, and the commits after it with it.
var locals = Unpushed(project, bundle);
var chain = locals;
var unsent = project.Versions().Where(v => v.StoredAt is null).Select(v => v.Id).ToHashSet();
var theirs = new List<ProjectVersion>();
for (var after = basis?.Number ?? 0; ;)
{
var page = await home.VersionsAfterAsync(project.ProjectId, after, PullPage, cancellationToken).ConfigureAwait(false);
foreach (var version in page)
{
Verify(version);
if (!unsent.Contains(version.Id)) project.WriteVersion(version, replace: true);
}
theirs.AddRange(page);
if (page.Count < PullPage) break;
after = page[^1].Number;
}
if (theirs.Count == 0) return (bundle, StillRecorded(project, bundle), theirs);
var held = theirs.Where(v => unsent.Contains(v.Id)).ToList();
var head = theirs[^1];
var atHome = theirs.Select(v => v.Id).ToHashSet();
var ours = locals.TakeWhile(v => atHome.Contains(v.Id)).ToList();
var tipVersion = locals.Count > 0 ? locals[^1] : basis;
if (ours.Count > 0) basis = theirs.First(v => v.Id == ours[^1].Id);
locals = [.. locals.Skip(ours.Count)];
// The media first learn how the home names its photos now (hashed or converted there since the copy).
var media = await MediaNamedAsAtHomeAsync(project, home, cancellationToken).ConfigureAwait(false);
var photos = media?.Photos ?? NoFrames;
if (IsNewer(head) || IsNewer(basis))
return await NewerAtHomeAsync(project, bundle, home, media, tipVersion, chain, locals, basis, ours.Count > 0, held, head, theirs, takeTheirs,
cancellationToken).ConfigureAwait(false);
// Each side names a photo as the frames held it when it was saved: named as the media hold it now (a conversion or a
// hash since), the same photo merges as one.
var basisSettings = basis is null ? new ProjectSettings() : ProjectMerge.Read(ProjectContent.Rebind(basis.SettingsJson, photos));
var headSettings = ProjectMerge.Read(ProjectContent.Rebind(head.SettingsJson, photos));
var tip = tipVersion is null ? basisSettings : ProjectMerge.Read(ProjectContent.Rebind(tipVersion.SettingsJson, photos));
var working = ProjectMerge.Read(Portable(bundle.Project, media) ?? throw MediaMissing(project.ProjectId));
// "Take the server's" everywhere: the paths are those the merge finds, so it runs once to find them.
var choice = takeTheirs is EveryPath ? null : takeTheirs;
List<ProjectSettings> mine = [.. locals.Select(v => ProjectMerge.Read(ProjectContent.Rebind(v.SettingsJson, photos)))];
var replay = ProjectMerge.Replay(basisSettings, mine, headSettings, choice);
// Where the working state holds the home's value already, the user has nothing to choose: the home's value is taken
// there, in the replayed versions too, so they and the working state agree (never asked, never listed).
var workingSide = WorkingConflicts(tip, working, basisSettings, headSettings);
var silent = replay.Conflicts.Concat(ProjectMerge.Conflicts(tip, working, replay.Commits.Count > 0 ? replay.Commits[^1] : headSettings))
.Select(d => d.Path).Where(p => !workingSide.Keys.Any(q => Overlaps(p, q))).ToHashSet(StringComparer.Ordinal);
if (silent.Count > 0)
{
choice = choice is null ? silent : new HashSet<string>(choice.Concat(silent), StringComparer.Ordinal);
replay = ProjectMerge.Replay(basisSettings, mine, headSettings, choice);
}
// This side's values wherever both changed: what taking the server's everywhere replaces (the editor's undo of it).
var keptTip = replay.Commits.Count > 0 ? replay.Commits[^1] : headSettings;
if (takeTheirs is EveryPath) replay = ProjectMerge.Replay(basisSettings, mine, headSettings, Paths(replay.Conflicts));
var newTip = replay.Commits.Count > 0 ? replay.Commits[^1] : headSettings;
var merge = ProjectMerge.MergeWithConflicts(tip, working, newTip, choice);
var keptMine = takeTheirs is EveryPath ? ProjectMerge.MergeWithConflicts(tip, working, keptTip).Merged : merge.Merged;
if (takeTheirs is EveryPath) merge = ProjectMerge.MergeWithConflicts(tip, working, newTip, Paths(merge.Conflicts));
var differences = AsInTheWorkingState(replay.Conflicts.Concat(merge.Conflicts).DistinctBy(d => d.Path).Where(d => !silent.Contains(d.Path)), workingSide);
if (differences.Count > 0 && (takeTheirs is null || (shown is not null && !SameDifferences(shown.Differences, differences))))
{
var conflict = new WorkspaceConflict(head.Id, head.Number, basis?.Id ?? Guid.Empty, [.. locals.Select(v => v.Id)], differences, DateTimeOffset.UtcNow);
if (ours.Count > 0) bundle = await MoveOntoHeldAsync(project, bundle, basis!, held, home, cancellationToken).ConfigureAwait(false);
project.WriteSync(project.ReadSync() with { Conflict = conflict, Replay = null });
return (bundle, conflict, theirs);
}
var rebased = new List<ProjectVersion>(locals.Count);
ProjectVersion parent = head;
// The last replayed version that carries the merge: its own commit met a setting the conflict listed.
var merged = -1;
var listed = Paths(differences);
for (var i = 0; i < locals.Count; i++)
{
var local = locals[i];
var portable = ProjectContent.Canonical(replay.Commits[i].ToJson());
// A commit the replay leaves changing nothing (the server's values taken over its own, or the same change made
// there) is not replayed into an empty version to send: its file goes to versions/.replaced with the others.
if (ProjectContent.SameChoices(parent.SettingsJson, portable, photos))
{
Log.Info("workspace", $"Version {local.Number} of project {project.ProjectId} changes nothing once replayed on version {head.Number}: it is not sent");
continue;
}
if (listed.Count > 0 && ProjectMerge.Conflicts(basisSettings, mine[i], headSettings).Any(d => listed.Contains(d.Path))) merged = rebased.Count;
parent = local with
{
Id = Guid.CreateVersion7(),
ParentId = parent.Id,
Number = parent.Number + 1,
SettingsJson = portable,
ContentHash = ProjectContent.Hash(portable),
SettingsFormat = ProjectContent.FormatOf(portable),
StoredAt = null,
};
rebased.Add(parent);
}
// Marked on the version that holds the choice made, never on an earlier one a skipped version leaves last.
if (merged >= 0) rebased[merged] = rebased[merged] with { Kind = ProjectVersionKind.Merge };
var placed = ProjectSettings.FromJson(ProjectContent.Place(ProjectContent.Canonical(merge.Merged.ToJson()), photos), allowNewer: true);
var moved = WithBase(bundle, head) with { Project = placed, Saved = DateTimeOffset.UtcNow };
await FinishCopyAsync(project, home, chain, cancellationToken).ConfigureAwait(false);
// Taking the server's values over changes no version holds yet: the working state is kept aside first, never in the chain.
if (takeTheirs is EveryPath && differences.Count > 0 && Portable(bundle.Project, media) is { } unsaved
&& (tipVersion is null || !ProjectContent.SameChoices(tipVersion.SettingsJson, unsaved, photos)))
project.WriteAside(VersionOf(project, unsaved, tipVersion, ProjectVersionKind.Commit, null, null, null, media?.FramesVersion ?? bundle.Workspace!.Media.FramesVersion));
// Past its commit point the replay has settled the conflict recorded beside it, the choice made applied (SettleAsync).
var journal = new WorkspaceReplay(head.Id, [.. locals.Select(v => v.Id)], [.. rebased.Select(v => v.Id)], [.. held.Select(v => v.Id)], Settles: true);
project.WriteSync(project.ReadSync() with { Replay = journal });
Step("replay.journal");
foreach (var version in rebased)
{
project.WriteVersion(version);
Step("replay.version");
}
// The commit point: before it the old chain hangs from the old base, after it the replayed one from the head.
project.WriteBundle(moved);
Step("replay.base");
await FinishReplayAsync(project, journal, held, home, cancellationToken).ConfigureAwait(false);
// A save-both cut short whose copy got the rest above is over with the conflict it was for.
project.WriteSync(project.ReadSync() with { Replay = null, SaveBoth = null });
if (takeTheirs is EveryPath && kept is not null)
kept(ProjectSettings.FromJson(ProjectContent.Place(ProjectContent.Canonical(keptMine.ToJson()), photos), allowNewer: true));
return (moved, null, theirs);
}
/// <summary>
/// A conflict met past local commits the home holds (their answer lost): the base moves onto the last of them,
/// <paramref name="onto"/> (their files adopted once it has), so the commits after them stay in the chain from the base
/// and the conflict is settled against it. Journaled like a replay.
/// </summary>
private async Task<ProjectBundle> MoveOntoHeldAsync(WorkspaceProject project, ProjectBundle bundle, ProjectVersion onto, IReadOnlyList<ProjectVersion> held,
IProjectHome home, CancellationToken cancellationToken)
{
var journal = new WorkspaceReplay(onto.Id, [], [], [.. held.Select(v => v.Id)]);
var sync = project.ReadSync();
// A save-both cut short goes on from the new base: the move is not its commit point (SettleAsync).
project.WriteSync(sync with { Replay = journal, SaveBoth = sync.SaveBoth is { } both ? both with { FromBase = onto.Id } : null });
Step("replay.journal");
bundle = WithBase(bundle, onto);
project.WriteBundle(bundle);
Step("replay.base");
await FinishReplayAsync(project, journal, held, home, cancellationToken).ConfigureAwait(false);
return bundle;
}
/// <summary>
/// The conflict recorded, while the home holds no version after the base: it lost them (an index taken back from an
/// older backup, a home answering late), so there is nothing to settle it against and it stays. One the base has reached
/// since (a replay that settled it, cut before it said so) is over: null.
/// </summary>
private static WorkspaceConflict? StillRecorded(WorkspaceProject project, ProjectBundle bundle)
{
if (project.ReadSync().Conflict is not { } recorded) return null;
var basis = bundle.Workspace!.Base;
if (basis is not null && (basis.VersionId == recorded.TheirsVersionId || basis.Number >= recorded.TheirsNumber)) return null;
Log.Info("workspace", $"The home holds no version of project {project.ProjectId} after the base: its conflict stays");
return recorded;
}
/// <summary>
/// The workspace's media with their photos named as the home names them now (<see cref="WorkspaceMedia.NamedAsAt"/>),
/// written back when the home taught them anything, so the versions, the working state and every comparison after this
/// one name each photo alike. Null while the workspace holds no media.
/// </summary>
private static async Task<WorkspaceMedia?> MediaNamedAsAtHomeAsync(WorkspaceProject project, IProjectHome home, CancellationToken cancellationToken)
{
if (project.ReadMedia() is not { } media) return null;
var named = media.NamedAsAt(await home.FramesAsync(cancellationToken).ConfigureAwait(false));
if (!ReferenceEquals(named, media)) project.WriteMedia(named);
return named;
}
private static bool IsNewer(ProjectVersion? version) => version is not null && ProjectContent.FormatOf(version.SettingsJson) > ProjectSettings.CurrentVersion;
/// <summary>
/// The home's head, or the base, holds settings of a newer Chronos: nothing is merged into or out of them, as their known
/// members may mean something else there. With nothing of this workspace's own (no version to send, the working state the
/// last version's), the project moves onto the head as it is, read-only when that is the newer one. Otherwise the local
/// versions and the working state stay as they are and a conflict of format waits (<see cref="WorkspaceConflict.NewerFormat"/>):
/// Save both sends them to a new project made from the base, in this Chronos's format. A choice to merge them
/// (<paramref name="takeTheirs"/> set: keep mine, take the server's) is refused, the conflict left recorded. As for any
/// conflict, one met past local commits the home holds (<paramref name="pastHeld"/>: <paramref name="basis"/> is the last
/// of them) first moves the base onto them, so the conflict, the chain and the base name one version.
/// </summary>
/// <exception cref="WorkspaceSyncException">A choice to merge was made.</exception>
private async Task<(ProjectBundle Bundle, WorkspaceConflict? Conflict, IReadOnlyList<ProjectVersion> Pulled)> NewerAtHomeAsync(WorkspaceProject project,
ProjectBundle bundle, IProjectHome home, WorkspaceMedia? media, ProjectVersion? tipVersion, IReadOnlyList<ProjectVersion> chain,
IReadOnlyList<ProjectVersion> locals, ProjectVersion? basis, bool pastHeld, IReadOnlyList<ProjectVersion> held, ProjectVersion head,
IReadOnlyList<ProjectVersion> theirs, IReadOnlySet<string>? takeTheirs, CancellationToken cancellationToken)
{
var photos = media?.Photos ?? NoFrames;
var tip = tipVersion?.SettingsJson ?? ProjectContent.ToPortable(new ProjectSettings(), photos);
if (locals.Count == 0 && (bundle.IsReadOnly
|| ProjectContent.SameChoices(tip, Portable(bundle.Project, media) ?? throw MediaMissing(project.ProjectId), photos)))
{
await FinishCopyAsync(project, home, chain, cancellationToken).ConfigureAwait(false);
var placed = ProjectSettings.FromJson(ProjectContent.Place(head.SettingsJson, photos), allowNewer: true);
var moved = WithBase(bundle, head) with { Project = placed, Saved = DateTimeOffset.UtcNow };
project.WriteBundle(moved);
// Moved onto the head: the local files of versions the home held take its copies, as after a replay.
if (held.Count > 0)
await FinishReplayAsync(project, new WorkspaceReplay(head.Id, [], [], [.. held.Select(v => v.Id)]), held, home, cancellationToken).ConfigureAwait(false);
if (project.ReadSync().SaveBoth is not null) project.WriteSync(project.ReadSync() with { SaveBoth = null });
return (moved, null, theirs);
}
if (pastHeld) bundle = await MoveOntoHeldAsync(project, bundle, basis!, held, home, cancellationToken).ConfigureAwait(false);
var newer = Math.Max(ProjectContent.FormatOf(head.SettingsJson), basis is null ? 0 : ProjectContent.FormatOf(basis.SettingsJson));
var format = new SettingsDifference(ProjectMerge.FormatPath, ProjectContent.FormatOf(tip).ToString(CultureInfo.InvariantCulture),
newer.ToString(CultureInfo.InvariantCulture));
var conflict = new WorkspaceConflict(head.Id, head.Number, basis?.Id ?? Guid.Empty, [.. locals.Select(v => v.Id)], [format], DateTimeOffset.UtcNow);
project.WriteSync(project.ReadSync() with { Conflict = conflict, Replay = null });
if (takeTheirs is not null)
throw new WorkspaceSyncException(PushOutcome.NewerFormat,
$"Version {head.Number} of project {project.ProjectId} holds settings of a newer Chronos: nothing of this workspace's is merged into it (save both, or update Chronos).");
return (bundle, conflict, theirs);
}
/// <summary>
/// Finishes a replay whose base moved (<see cref="WorkspaceReplay.OntoBase"/>): each local version the home holds takes
/// its copy (from <paramref name="known"/>, a file of the workspace, else the home), the replaced ones are set aside.
/// </summary>
private static async Task FinishReplayAsync(WorkspaceProject project, WorkspaceReplay replay, IReadOnlyList<ProjectVersion> known, IProjectHome home,
CancellationToken cancellationToken)
{
foreach (var id in replay.Adopted)
{
var locals = project.Versions().Where(v => v.Id == id && v.StoredAt is null).ToList();
if (locals.Count == 0) continue;
if ((known.FirstOrDefault(v => v.Id == id) ?? await StoredCopyAsync(project, id, home, cancellationToken).ConfigureAwait(false)) is not { } stored) continue;
foreach (var local in locals) Adopt(project, local, stored);
}
foreach (var id in replay.Replaced) SetAsideUnsent(project, id);
}
/// <summary>
/// Finishes or undoes what a cut left (see the class remarks), before anything reads the chain: a replay's journal is
/// finished once the base is its target, else its replayed versions are set aside (the old chain still hangs from the
/// base); a save-both whose project moved onto the home's head sets its sent versions aside and clears its conflict; a
/// local file of the version the base names (a push cut after it moved the base) takes the home's copy. A version file
/// that cannot be read takes the home's copy when the home holds that version (the damaged file is kept aside, in
/// <c>versions/.replaced</c>); one the home never had stays, a gap every sync then stops at (<see cref="EnsureChained"/>).
/// </summary>
private static async Task SettleAsync(WorkspaceProject project, IProjectHome home, CancellationToken cancellationToken)
{
foreach (var unreadable in project.UnreadableVersions())
{
if (await home.GetVersionAsync(project.ProjectId, unreadable.Id, cancellationToken).ConfigureAwait(false) is not { StoredAt: not null } copy
|| copy.Id != unreadable.Id)
continue;
Verify(copy);
project.SetAside(unreadable.Path);
// A local version the home stored (its answer lost) with versions saved after it stays in their chain, unsent,
// until the next push hears "already stored"; any other is the home's version it is.
var parentOfUnsent = project.Versions().Any(v => v.StoredAt is null && v.ParentId == copy.Id);
project.WriteVersion(parentOfUnsent ? copy with { StoredAt = null } : copy, replace: true);
Log.Warning("workspace", $"Version file {unreadable.Path} could not be read ({unreadable.Reason}); it took the home's copy and was kept aside");
}
var sync = project.ReadSync();
var basis = project.ReadBundle().Workspace!.Base;
if (sync.Replay is { } replay)
{
var finished = basis?.VersionId == replay.OntoBase;
if (finished) await FinishReplayAsync(project, replay, [], home, cancellationToken).ConfigureAwait(false);
else foreach (var id in replay.Replayed) SetAsideUnsent(project, id);
// Past its commit point a replay that settles a conflict has applied the choice made: the conflict is over. Cut
// before it, nothing moved and the conflict stays to be asked again.
sync = sync with { Replay = null, Conflict = finished && replay.Settles ? null : sync.Conflict };
project.WriteSync(sync);
Log.Info("workspace", $"A sync of project {project.ProjectId} cut short was {(finished ? "finished" : "undone")}");
}
if (sync.SaveBoth is { } both && basis?.VersionId != both.FromBase)
{
foreach (var move in both.Moves) SetAsideUnsent(project, move.LocalId);
var settled = sync.Conflict is { } conflict && conflict.BaseVersionId == (both.FromBase ?? Guid.Empty);
sync = sync with { SaveBoth = null, Conflict = settled ? null : sync.Conflict };
project.WriteSync(sync);
Log.Info("workspace", $"A save-both of project {project.ProjectId} cut short was finished");
}
if (basis is not null && project.Versions().Where(v => v.Id == basis.VersionId && v.StoredAt is null).ToList() is { Count: > 0 } lingering
&& await StoredCopyAsync(project, basis.VersionId, home, cancellationToken).ConfigureAwait(false) is { } stored)
{
foreach (var local in lingering) Adopt(project, local, stored);
}
}
/// <summary>The home's copy of version <paramref name="id"/>: a file of the workspace marked sent, else asked from the home (kept as a file); null when the home has none.</summary>
private static async Task<ProjectVersion?> StoredCopyAsync(WorkspaceProject project, Guid id, IProjectHome home, CancellationToken cancellationToken)
{
if (project.Versions().FirstOrDefault(v => v.Id == id && v.StoredAt is not null) is { } file) return file;
if (await home.GetVersionAsync(project.ProjectId, id, cancellationToken).ConfigureAwait(false) is not { StoredAt: not null } fetched) return null;
Verify(fetched);
return fetched;
}
/// <summary>
/// The local file of a version the home holds takes the home's copy (sent, maybe under another number). Called only once
/// the base names the version or one after it: before, the copy would take it out of the chain of unsent versions.
/// </summary>
private static void Adopt(WorkspaceProject project, ProjectVersion local, ProjectVersion stored)
{
project.WriteVersion(stored, replace: true);
if (stored.Number != local.Number) project.SetAside(local);
}
/// <summary>Sets aside the unsent local files of version <paramref name="id"/>.</summary>
private static void SetAsideUnsent(WorkspaceProject project, Guid id)
{
foreach (var version in project.Versions().Where(v => v.Id == id && v.StoredAt is null)) project.SetAside(version);
}
/// <summary>
/// Both sides kept: this workspace's versions to a new project at the home, this project onto the home's head. The
/// copy's name and the version it is made from are journaled before the home is asked for it, then the copy and the ids
/// its versions are sent under, before the first is sent (<see cref="WorkspaceSaveBoth"/>): a retry after a cut finds the
/// copy the home made by its name (<see cref="WorkspaceCopyIntent"/>) and sends it what it lacks (the home answers
/// "already stored" for the rest), whatever name it is asked for, and the base moving onto the home's head is the
/// commit point. A copy deleted at the home since, or one somebody else sent versions to (it is theirs now, left as it
/// is), gives way to a new copy that takes them all. Every local version after the base is sent: one the chain does not
/// reach stops it first (<see cref="EnsureChained"/>). A project the home deleted meanwhile keeps its versions and its
/// working state here as they are, the copy holding them too (<see cref="WorkspaceState.Gone"/>); a retry sends nothing new.
/// </summary>
private async Task<WorkspaceSyncResult> SaveBothAsync(WorkspaceProject project, ProjectBundle bundle, IProjectHome home, string? name, CancellationToken cancellationToken)
{
var workspace = bundle.Workspace!;
if (!bundle.IsReadOnly) CommitLocked(project, bundle, null, null);
bundle = project.ReadBundle();
var locals = Unpushed(project, bundle);
EnsureChained(project, bundle, locals);
var pending = project.ReadSync().SaveBoth;
SequenceProject? copy = null;
if (locals.Count > 0 && pending is not null && await OurCopyAsync(project, home, pending, cancellationToken).ConfigureAwait(false) is { } ours)
(copy, pending) = ours;
else
pending = null;
var pushed = new List<ProjectVersion>();
for (var attempt = 1; locals.Count > 0; attempt++)
{
var journaled = pending is not null;
if (pending is null)
{
var basis = await BaseVersionAsync(project, bundle, home, cancellationToken).ConfigureAwait(false);
(copy, pending) = await CreateCopyAsync(project, home, name, workspace, basis, locals, cancellationToken).ConfigureAwait(false);
}
var known = pending.Moves.Select(m => m.LocalId).ToHashSet();
if (!journaled || locals.Any(l => !known.Contains(l.Id)))
{
pending = pending with { Moves = [.. pending.Moves, .. locals.Where(l => !known.Contains(l.Id)).Select(l => new WorkspaceMove(l.Id, Guid.CreateVersion7()))] };
project.WriteSync(project.ReadSync() with { SaveBoth = pending });
}
Step("saveboth.journal");
try
{
pushed = await SendToCopyAsync(project, home, copy!, pending, locals, cancellationToken).ConfigureAwait(false);
break;
}
catch (CopyMovedException)
{
if (attempt >= 2)
throw new WorkspaceSyncException(PushOutcome.Diverged, $"Project '{copy!.Name}' at the home kept moving while project {project.ProjectId} was saved into it.");
// Somebody sent a version to the copy between our check and our push: it is theirs; a new one takes ours.
Log.Info("workspace", $"Save both of project {project.ProjectId}: project '{copy!.Name}' at the home moved meanwhile; a new copy is made");
pending = null;
}
}
// This project takes the home's head: the local versions now live in the copy (and their files in versions/.replaced).
ProjectVersion head;
try
{
head = await home.HeadAsync(project.ProjectId, cancellationToken).ConfigureAwait(false);
}
catch (ProjectException ex) when (NotFoundAtHome(ex))
{
// Deleted at the home meanwhile: nothing to move onto. The versions are in the copy, and stay here too, unsent,
// with the journal, so a retry finds that copy and sends nothing again.
ReportGone(project, ex);
return new WorkspaceSyncResult(await StatusLockedAsync(project, home, cancellationToken).ConfigureAwait(false), pushed, [], copy);
}
Verify(head);
project.WriteVersion(head, replace: true);
var media = project.ReadMedia();
var working = ProjectSettings.FromJson(ProjectContent.Place(head.SettingsJson, media?.Photos ?? NoFrames), allowNewer: true);
project.WriteBundle(WithBase(bundle, head) with { Project = working, Saved = DateTimeOffset.UtcNow });
Step("saveboth.base");
foreach (var local in locals) project.SetAside(local);
foreach (var move in pending?.Moves ?? []) SetAsideUnsent(project, move.LocalId);
// Settled and finished; a restore recorded stays, the next version saved tells whether it still restores.
project.WriteSync(project.ReadSync() with { Conflict = null, Replay = null, SaveBoth = null });
return new WorkspaceSyncResult(await StatusLockedAsync(project, home, cancellationToken).ConfigureAwait(false), pushed, [head], copy);
}
/// <summary>
/// The copy a save-both cut short made (<paramref name="pending"/>) while it is still this workspace's to send to: at the
/// home, its head the version it was made with or one this workspace sent it; with the journal as it now names it. One
/// only asked for (<see cref="WorkspaceSaveBoth.Intended"/>: a cut came before its answer) is found by the name it was
/// asked under: a project of that name made since, whose only version is the one it was made from (its ids journaled
/// then). Null when it was deleted since, never made, or somebody else sent it a version (it is theirs: never sent to,
/// never deleted by this workspace).
/// </summary>
private static async Task<(SequenceProject Copy, WorkspaceSaveBoth Pending)?> OurCopyAsync(WorkspaceProject project, IProjectHome home,
WorkspaceSaveBoth pending, CancellationToken cancellationToken)
{
var listed = await home.ListAsync(cancellationToken).ConfigureAwait(false);
if (pending.Intended)
{
if (pending.Intent is not { } intent
|| listed.FirstOrDefault(p => string.Equals(p.Name, intent.Name.Trim(), StringComparison.OrdinalIgnoreCase)) is not { } named
|| named.CreatedAt < intent.Asked - IntentSkew)
{
Log.Info("workspace", $"The copy a save-both of project {project.ProjectId} asked for was not made at the home");
return null;
}
var first = await home.HeadAsync(named.Id, cancellationToken).ConfigureAwait(false);
if (first.Number != 1 || (intent.FromHash is { } hash && !string.Equals(first.ContentHash, hash, StringComparison.OrdinalIgnoreCase)))
{
Log.Info("workspace", $"Project '{named.Name}' at the home is not the copy a save-both of project {project.ProjectId} asked for: it is left as it is");
return null;
}
var found = pending with { CopyId = named.Id, CopyBaseId = first.Id, CopyBaseNumber = first.Number };
project.WriteSync(project.ReadSync() with { SaveBoth = found });
Log.Info("workspace", $"Project '{named.Name}' at the home is the copy a save-both of project {project.ProjectId} cut short asked for: it is used");
return (named, found);
}
if (listed.FirstOrDefault(p => p.Id == pending.CopyId) is not { } copy)
{
Log.Info("workspace", $"The copy a save-both of project {project.ProjectId} made is no longer at the home");
return null;
}
var head = await home.HeadAsync(copy.Id, cancellationToken).ConfigureAwait(false);
if (head.Id == pending.CopyBaseId || pending.Moves.Any(m => m.CopyVersionId == head.Id)) return (copy, pending);
Log.Info("workspace", $"Project '{copy.Name}', the copy a save-both of project {project.ProjectId} made, took another's version since: it is left as it is");
return null;
}
/// <summary>How much earlier than asked a home's clock may date the copy a save-both asked for (<see cref="WorkspaceCopyIntent.Asked"/>).</summary>
private static readonly TimeSpan IntentSkew = TimeSpan.FromMinutes(5);
/// <summary>
/// A new project at the home for a save-both, named <paramref name="name"/>, else "<paramref name="workspace"/>'s name
/// (2)", "(3)"… whichever is free, made from <paramref name="basis"/>. Each name is journaled before the home is asked
/// (<see cref="WorkspaceCopyIntent"/>), then the copy with the ids <paramref name="locals"/> are sent under. A name the
/// home finds taken leaves no journal when it was asked for (the caller asks again without one).
/// </summary>
/// <exception cref="ProjectException">The name asked for is taken or invalid.</exception>
private async Task<(SequenceProject Copy, WorkspaceSaveBoth Pending)> CreateCopyAsync(WorkspaceProject project, IProjectHome home, string? name,
BundleWorkspace workspace, ProjectVersion? basis, IReadOnlyList<ProjectVersion> locals, CancellationToken cancellationToken)
{
var moves = locals.Select(l => new WorkspaceMove(l.Id, Guid.CreateVersion7())).ToList();
var named = !string.IsNullOrWhiteSpace(name);
HashSet<string> taken = named ? [] : (await home.ListAsync(cancellationToken).ConfigureAwait(false)).Select(p => p.Name).ToHashSet(StringComparer.OrdinalIgnoreCase);
for (var n = 2; ; n++)
{
var candidate = named ? name!.Trim() : Numbered(workspace.Name, n);
if (!named && n < 100 && taken.Contains(candidate)) continue;
var intended = new WorkspaceSaveBoth(Guid.Empty, Guid.Empty, 0, workspace.Base?.VersionId, moves,
new WorkspaceCopyIntent(candidate, basis?.Id, basis?.ContentHash, DateTimeOffset.UtcNow));
project.WriteSync(project.ReadSync() with { SaveBoth = intended });
Step("saveboth.intent");
SequenceProject copy;
try
{
copy = await home.CreateAsync(candidate, basis?.Id, null, cancellationToken).ConfigureAwait(false);
}
catch (ProjectException ex) when (ex.Reason == ProjectError.NameTaken && !named && n < 100)
{
continue;
}
catch (ProjectException ex) when (ex.Reason is ProjectError.NameTaken or ProjectError.InvalidName)
{
// Not made: no copy of that name is this workspace's.
project.WriteSync(project.ReadSync() with { SaveBoth = null });
throw;
}
Step("saveboth.created");
var first = await home.HeadAsync(copy.Id, cancellationToken).ConfigureAwait(false);
var pending = intended with { CopyId = copy.Id, CopyBaseId = first.Id, CopyBaseNumber = first.Number };
project.WriteSync(project.ReadSync() with { SaveBoth = pending });
return (copy, pending);
}
static string Numbered(string source, int n)
{
var candidate = $"{source} ({n})";
return candidate.Length > ProjectService.MaxNameLength ? $"{source[..(ProjectService.MaxNameLength - 6)]} ({n})" : candidate;
}
}
/// <summary>
/// Sends <paramref name="locals"/> to the save-both's copy under the ids its journal gives them, in order on the version
/// it was made with (the home answers "already stored" for those a cut sent before).
/// </summary>
/// <exception cref="CopyMovedException">The copy's head is not the parent sent: somebody else sent it a version.</exception>
/// <exception cref="WorkspaceSyncException">The home refused a version for good.</exception>
private async Task<List<ProjectVersion>> SendToCopyAsync(WorkspaceProject project, IProjectHome home, SequenceProject copy, WorkspaceSaveBoth pending,
IReadOnlyList<ProjectVersion> locals, CancellationToken cancellationToken)
{
var ids = pending.Moves.ToDictionary(m => m.LocalId, m => m.CopyVersionId);
var (parentId, parentNumber) = (pending.CopyBaseId, pending.CopyBaseNumber);
var pushed = new List<ProjectVersion>(locals.Count);
foreach (var local in locals)
{
var moved = local with { Id = ids[local.Id], ProjectId = copy.Id, ParentId = parentId, Number = parentNumber + 1, StoredAt = null };
var result = await home.PushAsync(copy.Id, moved, cancellationToken).ConfigureAwait(false);
if (result.Outcome == PushOutcome.Diverged) throw new CopyMovedException();
if (result is not { Outcome: PushOutcome.Stored or PushOutcome.AlreadyStored, Version: { } stored }) throw Refused(result.Outcome, local, $"project '{copy.Name}'");
Verify(stored);
pushed.Add(stored);
(parentId, parentNumber) = (stored.Id, stored.Number);
Step("saveboth.pushed");
}
Log.Info("workspace", $"{pushed.Count} version(s) of project {project.ProjectId} are in its copy '{copy.Name}' at the home");
return pushed;
}
/// <summary>
/// Keep mine or take the server's after a Save both cut short (its journal still there): the rest of this workspace's
/// versions the journal lists go to the copy it made (or asked for, found by its name), so the home never keeps half of
/// one; a copy deleted, never made or taken over since is left as it is. The journal itself goes once the project moved
/// (the caller's commit point).
/// </summary>
private async Task FinishCopyAsync(WorkspaceProject project, IProjectHome home, IReadOnlyList<ProjectVersion> chain, CancellationToken cancellationToken)
{
if (project.ReadSync().SaveBoth is not { } pending) return;
var listed = pending.Moves.Select(m => m.LocalId).ToHashSet();
var sent = chain.TakeWhile(v => listed.Contains(v.Id)).ToList();
if (sent.Count == 0 || await OurCopyAsync(project, home, pending, cancellationToken).ConfigureAwait(false) is not { } ours) return;
var (copy, known) = ours;
try
{
await SendToCopyAsync(project, home, copy, known, sent, cancellationToken).ConfigureAwait(false);
}
catch (CopyMovedException)
{
Log.Info("workspace", $"Project '{copy.Name}' took another's version while a save-both of project {project.ProjectId} was finished: it is left as it is");
}
}
/// <summary>The home answered a save-both's copy moved under it (<see cref="PushOutcome.Diverged"/>).</summary>
private sealed class CopyMovedException() : Exception("The copy moved at the home.");
/// <summary>A push the home refused for good: too large for a version, or refused otherwise (<see cref="WorkspaceSyncException.Outcome"/> says why).</summary>
private static WorkspaceSyncException Refused(PushOutcome outcome, ProjectVersion local, string target) => outcome switch
{
PushOutcome.TooLarge => new WorkspaceSyncException(outcome,
$"Version {local.Number} of {target} is larger than the {ProjectSidecars.MaxVersionBytes >> 20} MB a version may be: the home did not store it."),
_ => new WorkspaceSyncException(outcome, $"The home refused version {local.Number} of {target}: {outcome}."),
};
private ProjectVersion? CommitLocked(WorkspaceProject project, ProjectBundle bundle, string? note, string? author)
{
if (bundle.IsReadOnly) throw ReadOnly(project.ProjectId);
var workspace = bundle.Workspace!;
var media = project.ReadMedia();
var portable = Portable(bundle.Project, media) ?? throw MediaMissing(project.ProjectId);
var hash = ProjectContent.Hash(portable);
var versions = project.Versions();
var locals = Unpushed(versions, bundle);
// Saved after a gap, the new version would hang from the versions before it and the ones after it would be left out.
EnsureChained(project, bundle, locals);
var (tipId, tipNumber, tipHash) = locals.Count > 0
? (locals[^1].Id, locals[^1].Number, locals[^1].ContentHash)
: (workspace.Base?.VersionId, workspace.Base?.Number ?? 0, workspace.Base?.ContentHash);
if (hash == tipHash || SameAsTip(versions, bundle, locals, portable, media)) return null;
var sync = project.ReadSync();
// A restore taken back since (Undo, Ctrl+Z) no longer shows in the working state: that version is no restore.
var restoredFrom = sync.RestoredFrom is { } restored && StillRestores(project, restored, Tip(project, bundle, locals), portable, media?.Photos ?? NoFrames)
? sync.RestoredFrom
: null;
var version = VersionOf(project, portable, (tipId, tipNumber), restoredFrom is null ? ProjectVersionKind.Commit : ProjectVersionKind.Restore, note, author,
restoredFrom, media?.FramesVersion ?? workspace.Media.FramesVersion);
project.WriteVersion(version);
if (sync.RestoredFrom is not null) project.WriteSync(sync with { RestoredFrom = null });
return version;
}
/// <summary>A version of this workspace holding <paramref name="portable"/>, the child of <paramref name="parent"/> (null: the first).</summary>
private ProjectVersion VersionOf(WorkspaceProject project, string portable, ProjectVersion? parent, ProjectVersionKind kind, string? note, string? author,
Guid? restoredFrom, int framesVersion) =>
VersionOf(project, portable, (parent?.Id, parent?.Number ?? 0), kind, note, author, restoredFrom, framesVersion);
private ProjectVersion VersionOf(WorkspaceProject project, string portable, (Guid? Id, int Number) parent, ProjectVersionKind kind, string? note, string? author,
Guid? restoredFrom, int framesVersion) => new()
{
Id = Guid.CreateVersion7(),
ProjectId = project.ProjectId,
ParentId = parent.Id,
Number = parent.Number + 1,
Kind = kind,
Author = author,
ClientName = clientName,
Note = string.IsNullOrWhiteSpace(note) ? null : note.Trim(),
SettingsJson = portable,
ContentHash = ProjectContent.Hash(portable),
SettingsFormat = ProjectContent.FormatOf(portable),
FramesVersion = framesVersion,
RestoredFrom = restoredFrom,
CreatedAt = DateTimeOffset.UtcNow,
};
/// <summary>
/// Where the project stands. With a <paramref name="home"/>, the media first learn how it names its photos now (a photo
/// hashed or converted there since the copy, <see cref="WorkspaceMedia.NamedAsAt"/>), so a version the home made since
/// and the working state name each photo alike: written at once while the project's lease is held
/// (<paramref name="leased"/>), else when it can be taken now or after a moment (<see cref="WorkspaceStatus.MediaOutdated"/>
/// when it cannot). The home's frames are read again only when its stamp of them moved (<see cref="IProjectHome.FramesStampAsync"/>).
/// Without the lease, a version file that reads as damaged is read again under it before the state says so: another
/// operation between two of its writes is no damage; while that one holds the project, it is not reported (that
/// operation stops at it itself). A home that answers it no longer has the project reads as <see cref="WorkspaceState.Gone"/>,
/// with no conflict to answer (<see cref="WorkspaceStatus.Conflict"/> null; the one recorded stays in <c>sync.json</c>).
/// </summary>
private async Task<WorkspaceStatus> StatusLockedAsync(WorkspaceProject project, IProjectHome? home, CancellationToken cancellationToken, bool leased = true)
{
var local = ReadLocal(project);
if (local.Damaged > 0 && !leased) local = await ConfirmDamagedAsync(project, local, cancellationToken).ConfigureAwait(false);
var (bundle, media, sync, versions, locals, damaged) = local;
var workspace = bundle.Workspace!;
// A take of the server's values (or any replay that settles) cut past its commit point applied the choice: the
// conflict is over, as the next sync writes.
var conflict = WorkspaceProject.Unsettled(sync, bundle);
ProjectVersion? head = null;
HomeFrames? frames = null;
var offline = false;
var gone = false;
if (home is not null)
{
try
{
head = await home.HeadAsync(project.ProjectId, cancellationToken).ConfigureAwait(false);
if (media is not null) frames = await FramesOfAsync(home, workspace.Home, cancellationToken).ConfigureAwait(false);
}
catch (HomeUnavailableException)
{
offline = true;
}
catch (ProjectException ex) when (ex.Reason is ProjectError.NotFound or ProjectError.SequenceNotFound)
{
ReportGone(project, ex);
(gone, offline) = (true, true);
}
}
var outdated = false;
if (media is not null && frames is not null)
{
var named = media.NamedAsAt(frames);
if (!ReferenceEquals(named, media)) outdated = !await LearnNamesAsync(project, frames, leased, cancellationToken).ConfigureAwait(false);
outdated |= frames.FramesVersion != media.FramesVersion;
media = named;
}
var tipHash = locals.Count > 0 ? locals[^1].ContentHash : workspace.Base?.ContentHash;
var modified = !bundle.IsReadOnly && (Portable(bundle.Project, media) is not { } portable
|| (ProjectContent.Hash(portable) != tipHash && !SameAsTip(versions, bundle, locals, portable, media)));
var behind = head is not null && head.Id != workspace.Base?.VersionId ? Math.Max(1, head.Number - (workspace.Base?.Number ?? 0)) : 0;
// A local commit the home stored although its answer was lost reads as behind by the commits it holds: they are ours.
if (head is not null && locals.Any(v => v.Id == head.Id)) behind = 0;
var state = gone ? WorkspaceState.Gone
: media is null ? WorkspaceState.MediaRemoved
: damaged > 0 ? WorkspaceState.Damaged
: conflict is not null ? WorkspaceState.Conflict
: offline ? WorkspaceState.Offline
: behind > 0 ? WorkspaceState.Behind
: locals.Count > 0 ? WorkspaceState.Ahead
: modified ? WorkspaceState.Modified
: WorkspaceState.InSync;
// A project the home no longer has holds no conflict to answer (nothing can be merged with it): the one recorded stays
// in sync.json, for the record and for a project the home holds again.
return new WorkspaceStatus(state, modified, locals.Count, behind, offline, media is null, bundle.IsReadOnly, gone ? null : conflict, workspace.Base, head)
{
MediaOutdated = outdated,
Damaged = damaged,
};
}
/// <summary>What the workspace holds of a project, read once: its working state, media, sync state and versions.</summary>
private sealed record LocalState(ProjectBundle Bundle, WorkspaceMedia? Media, WorkspaceSyncState Sync, IReadOnlyList<ProjectVersion> Versions,
List<ProjectVersion> Locals, int Damaged);
private static LocalState ReadLocal(WorkspaceProject project)
{
var bundle = project.ReadBundle();
WorkspaceMedia? media;
try
{
media = project.ReadMedia();
}
catch (InvalidDataException)
{
media = null;
}
var sync = project.ReadSync();
var (versions, unreadable) = project.ReadVersionFiles();
var locals = Unpushed(versions, bundle);
return new LocalState(bundle, media, sync, versions, locals, Gaps(project, bundle, locals, sync, versions, unreadable).Count);
}
/// <summary>
/// A state read without the lease that found damaged versions, read again under it (taken without waiting): what that
/// read finds. While another operation holds the project its writes may be half done, so nothing is reported damaged.
/// </summary>
private static async Task<LocalState> ConfirmDamagedAsync(WorkspaceProject project, LocalState local, CancellationToken cancellationToken)
{
IDisposable lease;
try
{
lease = await project.LockAsync(TimeSpan.Zero, cancellationToken).ConfigureAwait(false);
}
catch (LeaseUnavailableException)
{
Log.Debug("workspace", $"Project {project.ProjectId} reads as damaged while another operation holds it: not reported yet");
return local with { Damaged = 0 };
}
using (lease) return ReadLocal(project);
}
/// <summary>
/// The home's frames: read again only when the home's stamp of them moved since they were last read for that home
/// (<see cref="IProjectHome.FramesStampAsync"/>), else those read then; always read from a home that keeps no stamp.
/// </summary>
private async Task<HomeFrames> FramesOfAsync(IProjectHome home, BundleHome key, CancellationToken cancellationToken)
{
var stamp = await home.FramesStampAsync(cancellationToken).ConfigureAwait(false);
if (stamp is not null && _frames.TryGetValue(key, out var known) && known.Stamp == stamp) return known.Frames;
var frames = await home.FramesAsync(cancellationToken).ConfigureAwait(false);
// Read after the stamp: these frames are at least as new as it, so a stamp moved meanwhile reads them again next time.
if (stamp is not null) _frames[key] = (stamp, frames);
return frames;
}
/// <summary>The homes' frames last read by a status, by home, with the stamp they were read at.</summary>
private readonly System.Collections.Concurrent.ConcurrentDictionary<BundleHome, (string Stamp, HomeFrames Frames)> _frames = new();
/// <summary>The projects whose home answered it no longer has them, each logged once.</summary>
private readonly System.Collections.Concurrent.ConcurrentDictionary<Guid, byte> _gone = new();
private void ReportGone(WorkspaceProject project, ProjectException ex)
{
if (_gone.TryAdd(project.ProjectId, 0))
Log.Info("workspace", $"The home no longer has project {project.ProjectId}: its copy here is kept as it is, with its versions ({ex.Message})");
}
/// <summary>
/// After <paramref name="ex"/> (a project or version the home does not know) stopped a sync: whether the home no longer
/// has the project itself (deleted there, or its sequence gone), which is no failure to retry but the state the project is
/// in. False when the home still has it (the version alone was not found) or cannot be asked now.
/// </summary>
private async Task<bool> GoneAsync(WorkspaceProject project, IProjectHome home, ProjectException ex, CancellationToken cancellationToken)
{
try
{
await home.HeadAsync(project.ProjectId, cancellationToken).ConfigureAwait(false);
return false;
}
catch (ProjectException gone) when (gone.Reason is ProjectError.NotFound or ProjectError.SequenceNotFound)
{
ReportGone(project, ex);
return true;
}
catch (HomeUnavailableException)
{
return false;
}
}
/// <summary>
/// <c>versions/.replaced</c> pruned after a sync that ended well (<see cref="WorkspaceProject.PruneReplaced"/>): never a
/// version the sync state still names (a journal, a conflict, a restore). A file that cannot go waits for the next one.
/// </summary>
private static void PruneReplaced(WorkspaceProject project)
{
var sync = project.ReadSync();
var referenced = new HashSet<Guid>();
if (sync.Replay is { } replay) referenced.UnionWith([.. replay.Replaced, .. replay.Replayed, .. replay.Adopted, replay.OntoBase]);
if (sync.SaveBoth is { } both) referenced.UnionWith([.. both.Moves.Select(m => m.LocalId), .. both.Moves.Select(m => m.CopyVersionId)]);
if (sync.Conflict is { } conflict) referenced.UnionWith([.. conflict.Mine, conflict.TheirsVersionId, conflict.BaseVersionId]);
if (sync.RestoredFrom is { } restored) referenced.Add(restored);
var removed = project.PruneReplaced(referenced, DateTime.UtcNow);
if (removed > 0) Log.Info("workspace", $"{removed} old file(s) of project {project.ProjectId} left versions/.replaced");
}
/// <summary>
/// The home still has the project: asked before a choice or a recorded conflict is replayed, so nothing is merged onto
/// the last version of a project deleted there (whose history the home may still read back).
/// </summary>
/// <exception cref="ProjectException">The home no longer has the project or its sequence (<see cref="GoneAsync"/>).</exception>
private static async Task EnsureAtHomeAsync(WorkspaceProject project, IProjectHome home, CancellationToken cancellationToken) =>
await home.HeadAsync(project.ProjectId, cancellationToken).ConfigureAwait(false);
/// <summary>Whether a sync stopped because the home does not know a project or a version (<see cref="GoneAsync"/> tells which).</summary>
private static bool NotFoundAtHome(ProjectException ex) => ex.Reason is ProjectError.NotFound or ProjectError.SequenceNotFound;
/// <summary>
/// Writes the media with the photos named as <paramref name="frames"/> name them (read again under the lease: a copy
/// may have written new media meanwhile); without <paramref name="leased"/>, only when the lease can be taken now or,
/// tried once more, after a moment (<see cref="LearnWait"/>). Whether the media know the home's names.
/// </summary>
private async Task<bool> LearnNamesAsync(WorkspaceProject project, HomeFrames frames, bool leased, CancellationToken cancellationToken)
{
IDisposable? lease = null;
try
{
if (!leased)
{
try
{
lease = await project.LockAsync(TimeSpan.Zero, cancellationToken).ConfigureAwait(false);
}
catch (LeaseUnavailableException)
{
// Two reads of the state at once (the picker and the menu), or a copy's commit: held only for a moment.
lease = await project.LockAsync(LearnWait, cancellationToken).ConfigureAwait(false);
}
}
if (project.ReadMedia() is not { } media) return false;
var named = media.NamedAsAt(frames);
if (!ReferenceEquals(named, media)) project.WriteMedia(named);
return true;
}
catch (Exception ex) when (ex is LeaseUnavailableException or InvalidDataException)
{
return false;
}
finally
{
lease?.Dispose();
}
}
/// <summary>How long a status read waits, the second time, for the lease to teach the media the home's names.</summary>
private static readonly TimeSpan LearnWait = TimeSpan.FromMilliseconds(500);
/// <summary>
/// The version files after the base the chain of versions to send (<paramref name="chain"/>) does not reach: files that
/// cannot be read, and unsent versions that do not follow from the base. Those a replay or a save-both cut short left
/// (their journals, which the next sync settles) are not counted. The files are read once.
/// </summary>
private static List<string> Gaps(WorkspaceProject project, ProjectBundle bundle, IReadOnlyList<ProjectVersion> chain, WorkspaceSyncState sync)
{
var (versions, unreadable) = project.ReadVersionFiles();
return Gaps(project, bundle, chain, sync, versions, unreadable);
}
private static List<string> Gaps(WorkspaceProject project, ProjectBundle bundle, IReadOnlyList<ProjectVersion> chain, WorkspaceSyncState sync,
IReadOnlyList<ProjectVersion> versions, IReadOnlyList<UnreadableVersion> unreadable)
{
var basis = bundle.Workspace!.Base;
var after = basis?.Number ?? 0;
var reached = chain.Select(v => v.Id).ToHashSet();
if (sync.Replay is { } replay) reached.UnionWith([.. replay.Replayed, .. replay.Replaced]);
if (sync.SaveBoth is { } both) reached.UnionWith(both.Moves.Select(m => m.LocalId));
var gaps = unreadable.Where(u => u.Number > after && !reached.Contains(u.Id)).Select(u => u.Path).ToList();
gaps.AddRange(versions.Where(v => v.StoredAt is null && v.Number > after && v.Id != basis?.VersionId && !reached.Contains(v.Id))
.Select(project.VersionPath));
return gaps;
}
/// <summary>
/// Refuses to save, send or take anything while a version saved here after the base cannot be read or does not follow
/// from it (<see cref="WorkspaceState.Damaged"/>): the chain would go on without it and the versions after it.
/// </summary>
/// <exception cref="InvalidDataException">There is such a version; the log names its file.</exception>
private static void EnsureChained(WorkspaceProject project, ProjectBundle bundle, IReadOnlyList<ProjectVersion> chain)
{
var gaps = Gaps(project, bundle, chain, project.ReadSync());
if (gaps.Count == 0) return;
Log.Warning("workspace", $"Project {project.ProjectId} has versions saved here the chain from its base does not reach: {string.Join(", ", gaps)}");
throw new InvalidDataException($"Project {project.ProjectId} has {gaps.Count} version file(s) after its base that cannot be read or do not follow from it: {gaps[0]}.");
}
/// <summary>
/// Whether the working state's portable form holds what the last version does (the last local one, else the base),
/// once each names its photos as the media hold them now: a version saved before the media were copied again after a
/// conversion or a hash at the home parks the same photos under their former names.
/// </summary>
private static bool SameAsTip(IReadOnlyList<ProjectVersion> versions, ProjectBundle bundle, List<ProjectVersion> locals, string portable, WorkspaceMedia? media)
{
if (media is null) return false;
return Tip(versions, bundle, locals) is { } tip && ProjectContent.SameChoices(tip.SettingsJson, portable, media.Photos);
}
/// <summary>The last version: the newest local one, else the base (null when this workspace keeps neither).</summary>
private static ProjectVersion? Tip(WorkspaceProject project, ProjectBundle bundle, List<ProjectVersion> locals) =>
locals.Count > 0 ? locals[^1] : Tip(project.Versions(), bundle, locals);
/// <summary>The last version among <paramref name="versions"/> (the workspace's, read once).</summary>
private static ProjectVersion? Tip(IReadOnlyList<ProjectVersion> versions, ProjectBundle bundle, List<ProjectVersion> locals) =>
locals.Count > 0 ? locals[^1] : bundle.Workspace!.Base is { } basis ? versions.FirstOrDefault(v => v.Id == basis.VersionId) : null;
/// <summary>
/// Whether the working state <paramref name="portable"/>, saved as a version on <paramref name="parent"/>, still restores
/// version <paramref name="restoredId"/>: that version differs from the parent, and the working state holds its values
/// wherever it does. A restore taken back since (the toast's Undo, Ctrl+Z) is no restore; one whose version this
/// workspace no longer keeps counts as recorded.
/// </summary>
private static bool StillRestores(WorkspaceProject project, Guid restoredId, ProjectVersion? parent, string portable, FramePhotos photos)
{
if (project.Versions().FirstOrDefault(v => v.Id == restoredId) is not { } restored) return true;
var then = ProjectContent.Rebind(restored.SettingsJson, photos);
var before = parent is null ? ProjectContent.ToPortable(new ProjectSettings(), photos) : ProjectContent.Rebind(parent.SettingsJson, photos);
if (then == before) return false;
var now = ProjectContent.Rebind(portable, photos);
var merge = ProjectMerge.MergeWithConflicts(ProjectMerge.Read(before), ProjectMerge.Read(now), ProjectMerge.Read(then));
return merge.Conflicts.Count == 0 && ProjectContent.Canonical(merge.Merged.ToJson()) == ProjectContent.Canonical(now);
}
/// <summary>
/// The conflicts found as the user sees them: where the working state holds a value of its own against the home's
/// (<paramref name="working"/>, <see cref="WorkingConflicts"/>), that value is this side's, not the version's.
/// Keeping mine keeps it, and taking the server's replaces it, so one answer settles what was listed.
/// </summary>
private static List<SettingsDifference> AsInTheWorkingState(IEnumerable<SettingsDifference> found, IReadOnlyDictionary<string, SettingsDifference> working) =>
[.. found.Select(d => working.TryGetValue(d.Path, out var held) ? held : d)];
/// <summary>
/// The settings where the working state holds a value of its own against the home's head, by path: against the last
/// local version (<paramref name="tip"/>: edited since it, the base value it went back to included), else against the
/// base (the version's value, unchanged since). A conflict at a path neither lists has the home's value in the working
/// state already.
/// </summary>
private static Dictionary<string, SettingsDifference> WorkingConflicts(ProjectSettings tip, ProjectSettings working, ProjectSettings basis, ProjectSettings head) =>
ProjectMerge.Conflicts(tip, working, head).Concat(ProjectMerge.Conflicts(basis, working, head)).DistinctBy(d => d.Path)
.ToDictionary(d => d.Path, StringComparer.Ordinal);
/// <summary>Two merge paths name the same setting, or one lies inside the other ("camera" and "camera.aspect").</summary>
private static bool Overlaps(string a, string b) =>
a == b || a.StartsWith(b + ".", StringComparison.Ordinal) || b.StartsWith(a + ".", StringComparison.Ordinal);
/// <summary>Whether two conflicts list the same settings with the same values on each side (in any order).</summary>
private static bool SameDifferences(IReadOnlyList<SettingsDifference> shown, IReadOnlyList<SettingsDifference> found) =>
shown.Count == found.Count && shown.ToHashSet().SetEquals(found);
/// <summary>The local versions after the base, oldest first, each the child of the one before (the base's for the first).</summary>
private static List<ProjectVersion> Unpushed(WorkspaceProject project, ProjectBundle bundle) => Unpushed(project.Versions(), bundle);
/// <summary>The local versions after the base among <paramref name="versions"/> (the workspace's, read once).</summary>
private static List<ProjectVersion> Unpushed(IReadOnlyList<ProjectVersion> versions, ProjectBundle bundle)
{
var basis = bundle.Workspace!.Base;
var candidates = versions.Where(v => v.StoredAt is null && v.Number > (basis?.Number ?? 0)).ToList();
var chain = new List<ProjectVersion>();
var parent = basis?.VersionId;
while (candidates.FirstOrDefault(v => v.ParentId == parent) is { } next)
{
chain.Add(next);
candidates.Remove(next);
parent = next.Id;
}
return chain;
}
/// <summary>The base version's content: this workspace's file, else the home's (kept as a file).</summary>
private static async Task<ProjectVersion?> BaseVersionAsync(WorkspaceProject project, ProjectBundle bundle, IProjectHome home, CancellationToken cancellationToken)
{
if (bundle.Workspace!.Base is not { } basis) return null;
if (project.Versions().FirstOrDefault(v => v.Id == basis.VersionId) is { } local) return local;
var fetched = await home.GetVersionAsync(project.ProjectId, basis.VersionId, cancellationToken).ConfigureAwait(false)
?? throw new ProjectException(ProjectError.NotFound, $"The home no longer has version {basis.Number} of project {project.ProjectId}.");
Verify(fetched);
project.WriteVersion(fetched, replace: true);
return fetched;
}
private static ProjectBundle WithBase(ProjectBundle bundle, ProjectVersion version) =>
bundle with { Workspace = bundle.Workspace! with { Base = Base(version) } };
private static BundleBase Base(ProjectVersion version) => new(version.Id, version.Number, version.ContentHash);
/// <summary>A version as a home handed it: its settings match its hash.</summary>
private static void Verify(ProjectVersion version)
{
if (!ProjectContent.Verifies(version.SettingsJson, version.ContentHash))
throw new InvalidDataException($"Version {version.Number} of project {version.ProjectId} does not match its content hash.");
}
private static HashSet<string> Paths(IEnumerable<SettingsDifference> differences) => [.. differences.Select(d => d.Path)];
/// <summary>
/// The portable form of a working state, whose positions name the frames of <paramref name="media"/>; null when the
/// workspace holds no media and the working state names frames by position (which frames, only the media say).
/// </summary>
private static string? Portable(ProjectSettings working, WorkspaceMedia? media)
{
if (media is not null) return ProjectContent.ToPortable(working, media.Photos);
var positions = working.ExcludedFrames.Count > 0 || working.HeroFrame is not null || working.Camera.Keyframes.Count > 0;
return positions ? null : ProjectContent.ToPortable(working, NoFrames);
}
private static InvalidOperationException MediaMissing(Guid projectId) =>
new($"Project {projectId} names frames the workspace holds no media of: copy the media again first.");
private WorkspaceProject Existing(Guid projectId) =>
store.Find(projectId) ?? throw new InvalidOperationException($"Project {projectId} is not in the workspace.");
private static InvalidOperationException ReadOnly(Guid projectId) =>
new($"Project {projectId} holds settings of a newer Chronos: it opens read-only.");
/// <summary>The sequence of a workspace project over its media, for the editor.</summary>
private static WorkspaceSequence SequenceOf(WorkspaceProject project, ProjectBundle bundle, WorkspaceMedia media)
{
var frames = media.ToFrames();
var files = media.Frames.Select(f => f.File is { } file && File.Exists(project.MediaFile(file)) ? project.MediaFile(file) : null).ToList();
var times = frames.Where(f => f.CapturedAt is not null).Select(f => f.CapturedAt!.Value).Order().ToList();
var sequence = new Sequence
{
Id = bundle.Workspace!.Home.SequenceId,
ArchiveId = bundle.Archive.Id,
Title = bundle.Sequence.Title,
Folder = bundle.Sequence.Folder,
FrameCount = frames.Count,
FirstCapture = times.Count > 0 ? times[0] : null,
LastCapture = times.Count > 0 ? times[^1] : null,
FramesVersion = media.FramesVersion,
SettingsJson = bundle.Project.ToJson(),
Origin = bundle.Workspace.Home.IsArchive ? SequenceOrigin.Local : SequenceOrigin.Server,
};
return new WorkspaceSequence(sequence, frames, files.Select((f, i) => frames[i].Flags.HasFlag(FrameFlags.Missing) ? null : f).ToList());
}
/// <summary>The choice "take the server's wherever both changed": its paths are only known once the merge ran.</summary>
private sealed class EveryPath : HashSet<string>;
}