feat(tools): screenshot diff against committed baselines, reproducible captures

BRIEF §20.2 asks for visual regression of the --capture screenshots with a
pixel-diff tolerance; CI only published them. `Titano.Tools screenshots diff
<dir>` compares every PNG with tests/visual/baselines: a pixel differs when a
channel moves by more than --tolerance (8 of 255), and a screen fails when more
than --max-diff of its pixels differ (0.05 %), when its size changed or when a
baseline lost its screen; a screen without a baseline only warns. It writes a
diff image per screen with differing pixels (the baseline faded, the differences
magenta) and a Markdown summary, also to the GitHub job summary.
`screenshots accept <dir>` copies screenshots over the baselines. The baselines
folder starts with its README; the pictures come later from ci-screenshots.

The captures were not reproducible: the sample's jobs, renders, transfers,
cards, trash and remote sequences were dated from the clock, the Library
timeline and the Archive heatmap laid out around today, the Uploader captured
in the computer's language and an indeterminate bar animated. The sample now
dates everything from SampleData.Now, --capture freezes the new AppClock there,
formats are English, and the new Motion attached property (off in the captured
windows) keeps the indeterminate sweep still in the middle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012iLMBNwBE2GnphAsu4cjaC
This commit is contained in:
Claude committed 2026-10-12 10:05:49 +00:00
1 parent 2f978d1c80
commit e583b6ed74
15 files changed
+603 -39

No files matched your search

+4 -3
View File
@@ -1,3 +1,4 @@
using Titano.App.DesignTime;
using Titano.Core.Archiving;
using Titano.Data.Archiving;
using Titano.Data.Repositories;
@@ -34,18 +35,18 @@ public static class SampleReports
/// <summary>The trash dialog's rows: the first two sample sequences, deleted an hour and a day ago (nothing moves on disk).</summary>
public static IReadOnlyList<TrashEntry> Trash(Guid archiveId, IReadOnlyList<SequenceLocation> sequences)
{
var now = DateTimeOffset.UtcNow;
var now = SampleData.Now;
return
[
.. sequences.Take(2).Select((s, i) => new TrashEntry(archiveId, s.Id.ToString("N"), s.Id, s.Title ?? s.Folder, s.Folder, 1_200 * (i + 1), 1_206 * (i + 1),
31_457_280L * 1_200 * (i + 1), now.AddHours(-1 - (23 * i)), Environment.UserName)),
31_457_280L * 1_200 * (i + 1), now.AddHours(-1 - (23 * i)), "alberto")),
];
}
/// <summary>What the rebuild dialog counts: a sidecar in every sequence folder but the last two, read by metadata instead.</summary>
public static IReadOnlyList<SequenceFolder> Folders(IReadOnlyList<SequenceLocation> sequences)
{
var stamp = new FileStamp(4_096, DateTime.UtcNow);
var stamp = new FileStamp(4_096, SampleData.Now.UtcDateTime);
return [.. sequences.Select((s, i) => new SequenceFolder(s.Folder, i < sequences.Count - 2 ? stamp : null, stamp))];
}
}
+35 -20
View File
@@ -39,6 +39,13 @@ public sealed class SampleData
{
public const string ServerFingerprint = "9A:3C:71:E0:5B:12:C4:88:0F:6D:2A:91:E7:44:B3:5C:19:D2:6F:80:A1:3E:57:C9:0B:64:F8:2D:93:A6:71:CE";
/// <summary>
/// The sample's "now": every time it stores (jobs, transfers, renders, cards, the trash) is set from it, never from
/// the computer's clock, and <c>--capture</c> freezes <see cref="AppClock"/> at it, so the screenshots are the same
/// on any day (the running render's log reads 10:14 to 10:15).
/// </summary>
public static readonly DateTimeOffset Now = new(2026, 10, 2, 10, 18, 0, TimeSpan.Zero);
private const string ServerAddress = "https://nas.local:8443";
private readonly AppServices _services;
@@ -84,7 +91,7 @@ public sealed class SampleData
var settings = _services.Settings;
var root = Path.Combine(_services.Environment.DataDirectory, "archive");
Directory.CreateDirectory(root);
var archive = new Archive { Id = Guid.CreateVersion7(), Name = "TitanoArchive", RootPath = root, Created = DateTimeOffset.UtcNow.AddYears(-2) };
var archive = new Archive { Id = Guid.CreateVersion7(), Name = "TitanoArchive", RootPath = root, Created = Now.AddYears(-2) };
await db.Archives.UpsertAsync(archive);
var paths = new ArchivePaths(root);
Directory.CreateDirectory(paths.MetaDirectory);
@@ -131,7 +138,7 @@ public sealed class SampleData
{
var db = _db ?? throw new InvalidOperationException("FillAsync first.");
async Task<Sequence> LocalAsync(string text) => (await db.Sequences.ListAsync(new Data.Repositories.SequenceQuery { Text = text, Limit = 1 })).Items[0];
var now = DateTimeOffset.UtcNow;
var now = Now;
Job Transfer(JobKind kind, Guid target, string label, string parameters, JobState state, long done, long total) => new()
{
Id = Guid.CreateVersion7(),
@@ -168,8 +175,16 @@ public sealed class SampleData
{
var db = _db ?? throw new InvalidOperationException("FillAsync first.");
var service = _services.Sync.Service ?? throw new InvalidOperationException("The index is not open.");
// The history is kept whatever the date: the sample's finished jobs are dated from Now, not from today.
await using var engine = new JobEngine(db, service.CreateJobHandlers(_services.Sync),
new JobEngineOptions { Workers = 1, WorkerId = "sample-proxies", Kinds = [JobKind.Download], PollInterval = TimeSpan.FromMilliseconds(50) });
new JobEngineOptions
{
Workers = 1,
WorkerId = "sample-proxies",
Kinds = [JobKind.Download],
PollInterval = TimeSpan.FromMilliseconds(50),
HistoryRetention = TimeSpan.FromDays(36_500),
});
await engine.StartAsync();
var job = await service.EnqueueDownloadAsync(engine, ServerAddress, _sunrise, "Sunrise over Superga", SyncDownloadMode.Proxies, null);
for (var i = 0; i < 200 && await db.Jobs.GetAsync(job.Id) is { State: JobState.Queued or JobState.Running }; i++) await Task.Delay(50);
@@ -247,14 +262,14 @@ public sealed class SampleData
Folder = "2026/2026-07-02 Sony ILCE-7M4",
CameraMake = "SONY",
CameraModel = "ILCE-7M4",
FirstCapture = DateTime.Today.AddDays(-96).AddHours(5.2),
LastCapture = DateTime.Today.AddDays(-96).AddHours(6.8),
FirstCapture = new DateTime(2026, 7, 2).AddHours(5.2),
LastCapture = new DateTime(2026, 7, 2).AddHours(6.8),
MedianInterval = 2,
FrameCount = 2_880,
TotalBytes = 2_880 * 21_000_000L,
Status = SequenceStatus.ProxiesBuilt | SequenceStatus.ThumbsBuilt,
RenderedCount = 1,
UpdatedAt = DateTimeOffset.UtcNow,
UpdatedAt = Now,
}), null, 3);
_lakeComo = _remote.Add(SampleSyncRemote.Dto(new Sequence
{
@@ -264,13 +279,13 @@ public sealed class SampleData
Folder = "2026/2026-08-21 Canon EOS R5",
CameraMake = "Canon",
CameraModel = "Canon EOS R5",
FirstCapture = DateTime.Today.AddDays(-46).AddHours(16.5),
LastCapture = DateTime.Today.AddDays(-46).AddHours(17.3),
FirstCapture = new DateTime(2026, 8, 21).AddHours(16.5),
LastCapture = new DateTime(2026, 8, 21).AddHours(17.3),
MedianInterval = 2,
FrameCount = 1_450,
TotalBytes = 1_450 * 45_000_000L,
Status = SequenceStatus.ThumbsBuilt,
UpdatedAt = DateTimeOffset.UtcNow,
UpdatedAt = Now,
}), null, 3);
static string DeflickerWindow(int frames) =>
@@ -333,7 +348,7 @@ public sealed class SampleData
Fps = 24,
DurationSeconds = 5,
Bytes = bytes,
Created = DateTimeOffset.UtcNow.AddDays(-daysAgo),
Created = Now.AddDays(-daysAgo),
ProducedBy = "studio-pc",
});
}
@@ -344,7 +359,7 @@ public sealed class SampleData
public async Task MarkAnalysingAsync(Guid sequenceId)
{
var db = _db ?? throw new InvalidOperationException("OpenEmptyAsync first.");
var now = DateTimeOffset.UtcNow;
var now = Now;
await db.Jobs.EnqueueAsync(new Job
{
Id = Guid.CreateVersion7(),
@@ -356,7 +371,7 @@ public sealed class SampleData
ProgressDone = 40,
ProgressTotal = 96,
ItemsPerSecond = 12,
NodeId = Environment.MachineName,
NodeId = _services.WorkerId,
Attempts = 1,
Created = now.AddSeconds(-5),
Started = now.AddSeconds(-4),
@@ -477,7 +492,7 @@ public sealed class SampleData
WarningCount = warnings,
Origin = origin,
SyncState = sync,
UpdatedAt = DateTimeOffset.UtcNow,
UpdatedAt = Now,
}, colours, raw);
}
@@ -598,7 +613,7 @@ public sealed class SampleData
private IEnumerable<Job> Jobs(IReadOnlyList<Sequence> s, Guid archiveId)
{
var now = DateTimeOffset.UtcNow;
var now = Now;
var here = _services.WorkerId;
Job Make(JobKind kind, Sequence? target, string label, JobState state, long done, long total, double? rate = null, TimeSpan? eta = null,
TimeSpan? startedAgo = null, TimeSpan? runFor = null, string? node = null, string? error = null, string? log = null,
@@ -652,7 +667,7 @@ public sealed class SampleData
private sealed class SampleServerApi(SampleData sample, Uri baseAddress, IReadOnlyCollection<string> trusted) : IServerApi
{
private UserDto User => new(Guid.Empty, "alberto", "Alberto", "editor", (long)sample._permissions, 5_000_000_000_000, 2_100_000_000_000, false, false,
new DateTimeOffset(2025, 1, 1, 0, 0, 0, TimeSpan.Zero), DateTimeOffset.UtcNow);
new DateTimeOffset(2025, 1, 1, 0, 0, 0, TimeSpan.Zero), Now);
public Uri BaseAddress { get; } = baseAddress;
@@ -668,7 +683,7 @@ public sealed class SampleData
public Task<LoginResponse> LoginAsync(string username, string password, CancellationToken ct)
{
Token = "sample-token";
return Task.FromResult(new LoginResponse(Token, User, false, DateTimeOffset.UtcNow.AddDays(30)));
return Task.FromResult(new LoginResponse(Token, User, false, Now.AddDays(30)));
}
public Task<UserDto> MeAsync(CancellationToken ct) =>
@@ -744,7 +759,7 @@ public sealed class SampleData
lock (_sequences)
{
var (dto, settings, frames) = _sequences[id];
var updated = dto with { SettingsVersion = dto.SettingsVersion + 1, UpdatedAt = DateTimeOffset.UtcNow };
var updated = dto with { SettingsVersion = dto.SettingsVersion + 1, UpdatedAt = Now };
_sequences[id] = (updated, request.SettingsJson ?? settings, frames);
return Task.FromResult(updated);
}
@@ -779,7 +794,7 @@ public sealed class SampleData
public Task<JobDto> CreateJobAsync(CreateJobRequest request, CancellationToken cancellationToken)
{
var job = new JobDto(Guid.CreateVersion7(), request.Kind, request.Target, null, "queued", "user", 0, 0, null, null, null, null, 0, null,
DateTimeOffset.UtcNow, null, null);
Now, null, null);
lock (_jobs) _jobs[job.Id] = job;
return Task.FromResult(job);
}
@@ -787,7 +802,7 @@ public sealed class SampleData
/// <summary>A job of <paramref name="kind"/> running for <paramref name="sequenceId"/>, <paramref name="done"/> of <paramref name="total"/> frames.</summary>
public void Run(Guid sequenceId, string kind, long done, long total)
{
var now = DateTimeOffset.UtcNow;
var now = Now;
var job = new JobDto(Guid.CreateVersion7(), kind, [sequenceId], null, "running", "user", done, total, 11.3, (total - done) / 11.3, null, "studio-pc", 1,
null, now.AddMinutes(-2), now.AddMinutes(-2), null);
lock (_jobs) _jobs[job.Id] = job;
@@ -809,7 +824,7 @@ public sealed class SampleData
}
public Task<JobDto> CancelJobAsync(Guid id, CancellationToken cancellationToken) =>
Task.FromResult(Update(id, job => job with { State = "cancelled", Finished = DateTimeOffset.UtcNow }));
Task.FromResult(Update(id, job => job with { State = "cancelled", Finished = Now }));
public Task<JobDto> RetryJobAsync(Guid id, CancellationToken cancellationToken) =>
Task.FromResult(Update(id, job => job with { State = "queued", Error = null, Finished = null }));
+2 -1
View File
@@ -1,4 +1,5 @@
using System.IO;
using Titano.App.DesignTime;
using Titano.Core.Archiving;
using Titano.Core.Codecs.Jpeg;
using Titano.Core.Imaging;
@@ -61,7 +62,7 @@ public static class SampleCards
CameraModel = "Canon EOS R5",
CameraSerial = "0431",
FirstSeen = LastImport.AddMonths(-7),
LastSeen = DateTimeOffset.UtcNow,
LastSeen = SampleData.Now,
};
await db.MediaCards.UpsertAsync(known);
await db.MediaCards.RecordImportsAsync([new ImportRecord(known.Id, new ImportKey("DCIM/100CANON/IMG_9001.CR2", 24_000_000, LastImport), null, null, null,
+17
View File
@@ -0,0 +1,17 @@
namespace Titano.App.Infrastructure;
/// <summary>
/// Today's date for screens that lay themselves out around it (the Library timeline's years, the months of the
/// Archive heatmap). <c>--capture</c> freezes it at the sample's moment, so the screenshots draw the same pixels on
/// any day and the pixel diff against their baselines stays meaningful.
/// </summary>
public static class AppClock
{
private static DateTimeOffset? _frozen;
/// <summary>The local date: the frozen moment's while frozen, the computer's otherwise.</summary>
public static DateTime Today => (_frozen?.ToLocalTime().DateTime ?? DateTime.Now).Date;
/// <summary>Stops the clock at <paramref name="moment"/> for the rest of the process (screenshots only).</summary>
public static void Freeze(DateTimeOffset moment) => _frozen = moment;
}
+1 -1
View File
@@ -460,7 +460,7 @@ public sealed class ArchiveViewModel : SectionViewModel
{
_withoutProxies = withoutProxies;
Tiles = ArchiveFigures.Tiles(stats);
MonthRows = [.. ArchiveFigures.PhotosPerMonth(stats, DateTime.Today).Select(r => new ChartRow(r.Label, r.Values, r.ToolTips))];
MonthRows = [.. ArchiveFigures.PhotosPerMonth(stats, AppClock.Today).Select(r => new ChartRow(r.Label, r.Values, r.ToolTips))];
StorageBars = Bars(ArchiveFigures.StorageByYear(stats));
CameraBars = Bars(ArchiveFigures.ByCamera(stats));
FormatBars = Bars(ArchiveFigures.ByFormat(stats));
+1 -1
View File
@@ -729,7 +729,7 @@ public sealed class LibraryViewModel : SectionViewModel
try
{
var stats = LibraryFacets.Of(local, _remote, _origin.Value);
_timeline = LibraryTimeline.From(stats, DateTime.Today);
_timeline = LibraryTimeline.From(stats, AppClock.Today);
var year = _year.Value;
var camera = _camera.Value;
Years = WithChosen([AnyYear(), .. stats.ByYear.OrderByDescending(y => y.Year).Select(y => new FilterOption<int?>(y.Year, YearText(y.Year), YearText(y.Year)))],
+10 -2
View File
@@ -1,3 +1,4 @@
using System.Globalization;
using System.IO;
using System.Windows;
using System.Windows.Media;
@@ -12,6 +13,7 @@ using Titano.App.Library;
using Titano.App.Server;
using Titano.App.Settings;
using Titano.Core.Diagnostics;
using Titano.Ui.Controls;
using Titano.Ui.Theming;
namespace Titano.App.Shell;
@@ -21,8 +23,10 @@ namespace Titano.App.Shell;
/// <c>&lt;screen&gt;-&lt;theme&gt;.png</c> from <see cref="SampleData"/> (first run on an empty index, then a populated
/// one, then the sync table with transfers, a conflict and its confirmation, then the sequence editor on an archive
/// sequence and on a server sequence with proxies, its server job and a read-only account), plus the Library, the
/// editor's Exposure tab and the import plan in Arabic (<c>*-rtl-dark.png</c>, right to left), for visual review and
/// phase reports (BRIEF §20.2), then exits.
/// editor's Exposure tab and the import plan in Arabic (<c>*-rtl-dark.png</c>, right to left), for visual review,
/// the pixel diff against <c>tests/visual/baselines</c> and phase reports (BRIEF §20.2), then exits. The pictures are
/// the same on any day and machine: the clock stands at <see cref="SampleData.Now"/>, formats are English and nothing
/// animates.
/// </summary>
public static class ScreenCapture
{
@@ -34,6 +38,9 @@ public static class ScreenCapture
try
{
Directory.CreateDirectory(directory);
AppClock.Freeze(SampleData.Now);
CultureInfo.CurrentCulture = CultureInfo.DefaultThreadCurrentCulture = CultureInfo.GetCultureInfo("en");
Motion.SetIsEnabled(window, false);
window.WindowStartupLocation = WindowStartupLocation.Manual;
window.Left = -20000;
window.Top = -20000;
@@ -195,6 +202,7 @@ public static class ScreenCapture
dialog.Top = -20000;
dialog.ShowActivated = false;
dialog.ShowInTaskbar = false;
Motion.SetIsEnabled(dialog, false);
dialog.Show();
try
{
@@ -0,0 +1,294 @@
using System.Globalization;
using System.Text;
using Titano.Core.Codecs.Png;
namespace Titano.Tools.Commands;
/// <summary>
/// Visual regression for the <c>--capture</c> screenshots (BRIEF §20.2). <c>screenshots diff &lt;dir&gt;</c> compares every
/// PNG with the baseline of the same name (<c>tests/visual/baselines</c> unless <c>--baseline</c>): a pixel differs when
/// one of its channels moves by more than <c>--tolerance</c> (of 255), and a screen fails when more than <c>--max-diff</c>
/// of its pixels differ (a fraction), when its size changed, or when a baseline has no screenshot any more. A screen
/// without a baseline only warns. Every screen with differing pixels gets <c>&lt;name&gt;-diff.png</c> in <c>--out</c> (the
/// baseline faded to grey, the differing pixels magenta) and the run a <c>summary.md</c> there, copied to the job summary
/// in GitHub Actions. <c>screenshots accept &lt;dir&gt;</c> copies the screenshots over the baselines.
/// </summary>
internal sealed class ScreenshotsCommand : ICommand
{
/// <summary>Absorbs the last-bit noise of anti-aliased edges, not a changed colour.</summary>
public const int DefaultTolerance = 8;
/// <summary>
/// 0.05 %: about 650 pixels of a 1440×900 screen. A few digits of a measured value (latency, the last comparison's
/// time) stay below it; a moved or resized control, a new colour or a missing picture do not.
/// </summary>
public const double DefaultMaxDiff = 0.0005;
public string Name => "screenshots";
public string Summary => "Pixel diff of --capture screenshots against tests/visual/baselines (diff <dir> [--baseline, --out, --tolerance, --max-diff]; accept <dir>)";
public Task<int> RunAsync(string[] args) => Task.FromResult(Run(args, Environment.GetEnvironmentVariable, Console.Out));
/// <summary>The command with its environment passed in: tests run it without GitHub annotations or job summary.</summary>
internal static int Run(string[] args, Func<string, string?> environment, TextWriter output)
{
var a = new Args(args);
if (a.Positional is not [var verb, var directory]) throw new ToolException("Usage: screenshots diff <dir> [--baseline <dir>] [--out <dir>] [--tolerance 0-255] [--max-diff fraction] | screenshots accept <dir> [--baseline <dir>]");
if (!Directory.Exists(directory)) throw new ToolException($"Screenshot folder not found: {directory}");
var baseline = a.Get("baseline") ?? Path.Combine(Args.RepoRoot(), "tests", "visual", "baselines");
switch (verb)
{
case "diff":
var options = new DiffOptions(
directory,
baseline,
a.Get("out") ?? Path.TrimEndingDirectorySeparator(Path.GetFullPath(directory)) + "-diff",
ParseTolerance(a.Get("tolerance")),
ParseMaxDiff(a.Get("max-diff")));
var actions = environment("GITHUB_ACTIONS") == "true";
var summary = actions ? environment("GITHUB_STEP_SUMMARY") : null;
return Diff(options, output, annotate: actions, stepSummary: summary).Fails ? 1 : 0;
case "accept":
Directory.CreateDirectory(baseline);
var accepted = 0;
foreach (var file in Directory.EnumerateFiles(directory, "*.png").Order(StringComparer.Ordinal))
{
File.Copy(file, Path.Combine(baseline, Path.GetFileName(file)), overwrite: true);
accepted++;
}
output.WriteLine($"{accepted} screenshot(s) copied to {baseline}");
return 0;
default:
throw new ToolException($"Unknown screenshots verb '{verb}' (use diff or accept).");
}
}
internal sealed record DiffOptions(string Directory, string Baseline, string Output, int Tolerance, double MaxDiff);
internal enum Verdict
{
/// <summary>No pixel beyond the tolerance.</summary>
Same,
/// <summary>Some pixels beyond the tolerance, no more than the allowed fraction.</summary>
Within,
Changed,
Resized,
Unreadable,
/// <summary>A screenshot without a baseline: warns only.</summary>
New,
/// <summary>A baseline whose screen was not captured.</summary>
Missing,
}
internal sealed record ScreenResult(string Name, Verdict Verdict, long Differing, long Pixels, string? Detail = null)
{
public double Fraction => Pixels == 0 ? 0 : (double)Differing / Pixels;
public bool Fails => Verdict is Verdict.Changed or Verdict.Resized or Verdict.Unreadable or Verdict.Missing;
}
internal sealed record DiffReport(IReadOnlyList<ScreenResult> Screens)
{
public bool Fails => Screens.Any(s => s.Fails);
}
/// <summary>RGBA, 8 bits per channel, top-down.</summary>
internal sealed record Picture(int Width, int Height, byte[] Pixels);
/// <summary>Compares the folders, writes the diff images and <c>summary.md</c>, and reports every screen.</summary>
internal static DiffReport Diff(DiffOptions options, TextWriter output, bool annotate, string? stepSummary)
{
var captured = PngNames(options.Directory);
var baselines = Directory.Exists(options.Baseline) ? PngNames(options.Baseline) : new SortedSet<string>(StringComparer.OrdinalIgnoreCase);
// Diffs of an earlier run must not pass for this run's.
if (Directory.Exists(options.Output))
{
foreach (var old in Directory.EnumerateFiles(options.Output, "*-diff.png")) File.Delete(old);
}
Directory.CreateDirectory(options.Output);
var results = new List<ScreenResult>();
foreach (var name in captured)
{
if (!baselines.Contains(name))
{
results.Add(new ScreenResult(name, Verdict.New, 0, 0));
continue;
}
results.Add(CompareFiles(name, Path.Combine(options.Baseline, name), Path.Combine(options.Directory, name), options));
}
results.AddRange(baselines.Where(b => !captured.Contains(b)).Select(b => new ScreenResult(b, Verdict.Missing, 0, 0)));
results.Sort((x, y) => StringComparer.OrdinalIgnoreCase.Compare(x.Name, y.Name));
foreach (var r in results) output.WriteLine($"{Label(r.Verdict),-11} {r.Name}{Describe(r, options)}");
var report = new DiffReport(results);
var markdown = Markdown(report, options);
File.WriteAllText(Path.Combine(options.Output, "summary.md"), markdown);
if (stepSummary is not null) File.AppendAllText(stepSummary, markdown);
output.WriteLine(Totals(report));
if (annotate)
{
foreach (var r in results.Where(r => r.Fails))
output.WriteLine($"::error title={Escape("Screenshot " + Label(r.Verdict), property: true)}::{Escape(r.Name + Describe(r, options), property: false)}");
// One warning for them all: before the first baselines every screen is new, and GitHub shows only ten.
var unbased = results.Where(r => r.Verdict == Verdict.New).Select(r => r.Name).ToList();
if (unbased.Count > 0)
{
var names = string.Join(", ", unbased.Take(5)) + (unbased.Count > 5 ? ", …" : string.Empty);
output.WriteLine($"::warning title={Escape("Screenshots without baseline", property: true)}::" +
Escape($"{unbased.Count} screenshot(s) have no baseline ({names}). tests/visual/baselines/README.md says how to add them.", property: false));
}
}
return report;
}
private static ScreenResult CompareFiles(string name, string baselinePath, string capturedPath, DiffOptions options)
{
Picture expected, actual;
try
{
expected = Read(baselinePath);
actual = Read(capturedPath);
}
catch (InvalidDataException ex)
{
return new ScreenResult(name, Verdict.Unreadable, 0, 0, ex.Message);
}
if (expected.Width != actual.Width || expected.Height != actual.Height)
return new ScreenResult(name, Verdict.Resized, 0, 0, $"{actual.Width}×{actual.Height}, baseline {expected.Width}×{expected.Height}");
var (differing, diff) = Compare(expected, actual, options.Tolerance);
var pixels = (long)actual.Width * actual.Height;
if (differing == 0) return new ScreenResult(name, Verdict.Same, 0, pixels);
File.WriteAllBytes(Path.Combine(options.Output, Path.GetFileNameWithoutExtension(name) + "-diff.png"),
PngEncoder.Encode(diff, actual.Width, actual.Height, PngColorType.Rgb, level: System.IO.Compression.CompressionLevel.Fastest));
var verdict = (double)differing / pixels > options.MaxDiff ? Verdict.Changed : Verdict.Within;
return new ScreenResult(name, verdict, differing, pixels);
}
/// <summary>
/// Counts the pixels where a channel differs by more than <paramref name="tolerance"/> and draws the diff image: the
/// baseline as faded grey, differing pixels magenta.
/// </summary>
internal static (long Differing, byte[] Diff) Compare(Picture expected, Picture actual, int tolerance)
{
if (expected.Width != actual.Width || expected.Height != actual.Height) throw new ArgumentException("The pictures differ in size.");
var count = expected.Width * expected.Height;
var diff = new byte[count * 3];
long differing = 0;
var e = expected.Pixels;
var a = actual.Pixels;
for (var i = 0; i < count; i++)
{
var p = i * 4;
var delta = Math.Max(Math.Max(Math.Abs(e[p] - a[p]), Math.Abs(e[p + 1] - a[p + 1])), Math.Max(Math.Abs(e[p + 2] - a[p + 2]), Math.Abs(e[p + 3] - a[p + 3])));
var q = i * 3;
if (delta > tolerance)
{
differing++;
diff[q] = 255;
diff[q + 1] = 0;
diff[q + 2] = 255;
continue;
}
// Rec.709 luma of the baseline, over its alpha, squeezed into 112..207 so magenta stands out in light and dark themes.
var luma = (e[p] * 54 + e[p + 1] * 183 + e[p + 2] * 19) >> 8;
var grey = (byte)(112 + luma * e[p + 3] / 255 * 3 / 8);
diff[q] = diff[q + 1] = diff[q + 2] = grey;
}
return (differing, diff);
}
/// <summary>Decodes a PNG of any colour type and depth to <see cref="Picture"/> (16-bit samples keep their high byte).</summary>
internal static Picture Read(string path)
{
var png = PngDecoder.Decode(File.ReadAllBytes(path));
var count = png.Width * png.Height;
var rgba = new byte[count * 4];
var channels = png.Channels;
for (var i = 0; i < count; i++)
{
byte Sample(int c) => png.Pixels8 is { } p8 ? p8[i * channels + c] : (byte)(png.Pixels16![i * channels + c] >> 8);
var (r, g, b, alpha) = channels switch
{
1 => (Sample(0), Sample(0), Sample(0), (byte)255),
2 => (Sample(0), Sample(0), Sample(0), Sample(1)),
3 => (Sample(0), Sample(1), Sample(2), (byte)255),
_ => (Sample(0), Sample(1), Sample(2), Sample(3)),
};
rgba[i * 4] = r;
rgba[i * 4 + 1] = g;
rgba[i * 4 + 2] = b;
rgba[i * 4 + 3] = alpha;
}
return new Picture(png.Width, png.Height, rgba);
}
private static SortedSet<string> PngNames(string directory) =>
new(Directory.EnumerateFiles(directory, "*.png").Select(f => Path.GetFileName(f)), StringComparer.OrdinalIgnoreCase);
private static string Label(Verdict verdict) => verdict switch
{
Verdict.Same => "same",
Verdict.Within => "tolerated",
Verdict.Changed => "changed",
Verdict.Resized => "resized",
Verdict.Unreadable => "unreadable",
Verdict.New => "new",
_ => "missing",
};
private static string Describe(ScreenResult r, DiffOptions options) => r.Verdict switch
{
Verdict.Same => string.Empty,
Verdict.Within or Verdict.Changed => $": {Percent(r.Fraction)} of the pixels differ ({r.Differing.ToString("N0", CultureInfo.InvariantCulture)}; limit {Percent(options.MaxDiff)})",
Verdict.Resized or Verdict.Unreadable => $": {r.Detail}",
Verdict.New => ": no baseline",
_ => ": a baseline without a screenshot",
};
private static string Percent(double fraction) => (fraction * 100).ToString("0.###", CultureInfo.InvariantCulture) + " %";
private static string Totals(DiffReport report)
{
int Count(params Verdict[] verdicts) => report.Screens.Count(s => verdicts.Contains(s.Verdict));
return $"{Count(Verdict.Same, Verdict.Within, Verdict.Changed, Verdict.Resized, Verdict.Unreadable)} compared: {Count(Verdict.Same, Verdict.Within)} within tolerance, " +
$"{Count(Verdict.Changed, Verdict.Resized, Verdict.Unreadable)} changed; {Count(Verdict.New)} without a baseline; {Count(Verdict.Missing)} missing.";
}
private static string Markdown(DiffReport report, DiffOptions options)
{
var md = new StringBuilder();
md.AppendLine("### Screenshot diff").AppendLine();
md.AppendLine(CultureInfo.InvariantCulture, $"Tolerance {options.Tolerance} of 255 per channel, at most {Percent(options.MaxDiff)} of the pixels.").AppendLine();
var listed = report.Screens.Where(s => s.Verdict != Verdict.Same).ToList();
if (listed.Count > 0)
{
md.AppendLine("| Screen | Result | Detail |").AppendLine("|---|---|---|");
foreach (var r in listed) md.AppendLine(CultureInfo.InvariantCulture, $"| {r.Name} | {Label(r.Verdict)} | {Describe(r, options).TrimStart(':', ' ')} |");
md.AppendLine();
}
md.AppendLine(Totals(report)).AppendLine();
return md.ToString();
}
private static int ParseTolerance(string? value) =>
value is null ? DefaultTolerance
: int.TryParse(value, NumberStyles.None, CultureInfo.InvariantCulture, out var t) && t is >= 0 and <= 255 ? t
: throw new ToolException($"--tolerance takes 0 to 255, not '{value}'.");
private static double ParseMaxDiff(string? value) =>
value is null ? DefaultMaxDiff
: double.TryParse(value, NumberStyles.Float, CultureInfo.InvariantCulture, out var f) && f is >= 0 and <= 1 ? f
: throw new ToolException($"--max-diff takes a fraction from 0 to 1, not '{value}'.");
/// <summary>Workflow-command escaping (properties also escape ':' and ',').</summary>
private static string Escape(string value, bool property)
{
var escaped = value.Replace("%", "%25", StringComparison.Ordinal).Replace("\r", "%0D", StringComparison.Ordinal).Replace("\n", "%0A", StringComparison.Ordinal);
return property ? escaped.Replace(":", "%3A", StringComparison.Ordinal).Replace(",", "%2C", StringComparison.Ordinal) : escaped;
}
}
+1
View File
@@ -12,6 +12,7 @@ var commands = new ICommand[]
new SelfTestCommand(),
new DiagnoseCommand(),
new FixturesCommand(),
new ScreenshotsCommand(),
};
if (args.Length == 0 || args[0] is "-h" or "--help" or "help")
+18
View File
@@ -0,0 +1,18 @@
using System.Windows;
namespace Titano.Ui.Controls;
/// <summary>
/// Whether controls animate, inherited down the visual tree from where it is set. The screenshots (<c>--capture</c>)
/// turn it off on their windows: an indeterminate bar then stands still, so a screen draws the same pixels every run
/// and the pixel diff against its baseline compares like with like.
/// </summary>
public static class Motion
{
public static readonly DependencyProperty IsEnabledProperty = DependencyProperty.RegisterAttached(
"IsEnabled", typeof(bool), typeof(Motion), new FrameworkPropertyMetadata(true, FrameworkPropertyMetadataOptions.Inherits));
public static bool GetIsEnabled(DependencyObject d) => (bool)d.GetValue(IsEnabledProperty);
public static void SetIsEnabled(DependencyObject d, bool value) => d.SetValue(IsEnabledProperty, value);
}
+13 -6
View File
@@ -611,7 +611,8 @@
<Rectangle.Fill>
<LinearGradientBrush StartPoint="0,0" EndPoint="1,0">
<LinearGradientBrush.RelativeTransform>
<TranslateTransform x:Name="SweepShift" X="-1" />
<!-- In the middle while it stands still (Motion off); the sweep animation drives it otherwise. -->
<TranslateTransform x:Name="SweepShift" X="0" />
</LinearGradientBrush.RelativeTransform>
<GradientStop Color="Transparent" Offset="0.3" />
<GradientStop Color="{DynamicResource Color.Accent}" Offset="0.5" />
@@ -625,17 +626,23 @@
<Trigger Property="IsIndeterminate" Value="True">
<Setter TargetName="PART_Indicator" Property="Visibility" Value="Collapsed" />
<Setter TargetName="Sweep" Property="Visibility" Value="Visible" />
<Trigger.EnterActions>
</Trigger>
<MultiTrigger>
<MultiTrigger.Conditions>
<Condition Property="IsIndeterminate" Value="True" />
<Condition Property="ui:Motion.IsEnabled" Value="True" />
</MultiTrigger.Conditions>
<MultiTrigger.EnterActions>
<BeginStoryboard x:Name="SweepAnimation">
<Storyboard RepeatBehavior="Forever">
<DoubleAnimation Storyboard.TargetName="SweepShift" Storyboard.TargetProperty="X" From="-0.7" To="0.7" Duration="0:0:1.6" />
</Storyboard>
</BeginStoryboard>
</Trigger.EnterActions>
<Trigger.ExitActions>
</MultiTrigger.EnterActions>
<MultiTrigger.ExitActions>
<StopStoryboard BeginStoryboardName="SweepAnimation" />
</Trigger.ExitActions>
</Trigger>
</MultiTrigger.ExitActions>
</MultiTrigger>
</ControlTemplate.Triggers>
</ControlTemplate>
</Setter.Value>
+4 -1
View File
@@ -20,6 +20,9 @@ public sealed class SampleData
private const int FileCount = 4_812;
private const long FileSize = 20_000_000;
/// <summary>The sample's "now" (when the failed file failed): fixed, so the screenshots are the same on any day.</summary>
private static readonly DateTime Now = new(2026, 9, 28, 19, 2, 0, DateTimeKind.Utc);
private readonly SampleSession _session = new();
private readonly UploadQueue _queue = new();
private MainViewModel? _vm;
@@ -88,7 +91,7 @@ public sealed class SampleData
_queue.Items[2_201].State = UploadItemState.Uploading;
_queue.Items[2_201].BytesSent = FileSize * 46 / 100;
_queue.Items[2_202].State = UploadItemState.Hashing;
_queue.Fail(_queue.Items[2_203], UploadFailure.FileUnreadable, null, DateTime.UtcNow);
_queue.Fail(_queue.Items[2_203], UploadFailure.FileUnreadable, null, Now);
}
private sealed class SampleSession : IServerSession
+9 -4
View File
@@ -1,9 +1,11 @@
using System.Globalization;
using System.IO;
using System.Windows;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Threading;
using Titano.Core.Diagnostics;
using Titano.Ui.Controls;
using Titano.Ui.Theming;
using Titano.Uploader.DesignTime;
@@ -11,8 +13,9 @@ namespace Titano.Uploader.Shell;
/// <summary>
/// <c>Titano.Uploader.exe --capture &lt;dir&gt;</c>: renders the window at its default 760×700 — first run (connect),
/// queue running, settings panel — from <see cref="SampleData"/> in the dark and light themes, then once more in
/// Arabic (right to left, dark), and exits.
/// queue running, settings panel — from <see cref="SampleData"/> in the dark and light themes, in English whatever the
/// computer's language, then once more in Arabic (right to left, dark), and exits. Nothing animates, so the pictures
/// compare with their baselines pixel for pixel.
/// </summary>
public static class ScreenCapture
{
@@ -20,7 +23,7 @@ public static class ScreenCapture
private const double Height = 700;
/// <param name="directory">Where the PNG files go.</param>
/// <param name="applyLanguage">Switches the UI language (a culture name; null = the OS language).</param>
/// <param name="applyLanguage">Switches the UI language (a culture name).</param>
public static void Run(string directory, Action<string?> applyLanguage)
{
Directory.CreateDirectory(directory);
@@ -28,7 +31,8 @@ public static class ScreenCapture
{
try
{
applyLanguage(null);
CultureInfo.CurrentCulture = CultureInfo.DefaultThreadCurrentCulture = CultureInfo.GetCultureInfo("en");
applyLanguage("en");
CaptureScreens(directory, string.Empty, [AppTheme.Dark, AppTheme.Light]);
// Mirrored layout with paths, addresses and file names kept left to right (BRIEF §16).
applyLanguage("ar");
@@ -62,6 +66,7 @@ public static class ScreenCapture
ShowActivated = false,
ShowInTaskbar = false,
};
Motion.SetIsEnabled(window, false);
window.Show();
try
{
@@ -0,0 +1,152 @@
using Titano.Core.Codecs.Png;
using Titano.Testing;
using Titano.Tools.Commands;
using static Titano.Tools.Commands.ScreenshotsCommand;
namespace Titano.Tools.Tests;
/// <summary>The screenshot diff fails on changed screens, tolerates noise and only warns about screens without a baseline (BRIEF §20.2).</summary>
public sealed class ScreenshotsCommandTests(TestContext context)
{
private const int Size = 100;
/// <summary>A 100×100 RGB screen: a dark background with a light panel, then <paramref name="change"/> applied.</summary>
private static void WriteScreen(string folder, string name, Action<byte[]>? change = null, int width = Size, int height = Size)
{
var pixels = new byte[width * height * 3];
for (var y = 0; y < height; y++)
{
for (var x = 0; x < width; x++)
{
var v = (byte)(x is >= 20 and < 80 && y is >= 30 and < 60 ? 230 : 32);
pixels[(y * width + x) * 3] = pixels[(y * width + x) * 3 + 1] = pixels[(y * width + x) * 3 + 2] = v;
}
}
change?.Invoke(pixels);
File.WriteAllBytes(Path.Combine(folder, name), PngEncoder.Encode(pixels, width, height, PngColorType.Rgb));
}
/// <summary>Adds <paramref name="delta"/> to the red channel of the first <paramref name="count"/> pixels.</summary>
private static Action<byte[]> Shift(int count, int delta) => pixels =>
{
for (var i = 0; i < count; i++) pixels[i * 3] = (byte)(pixels[i * 3] + delta);
};
private (string Captured, string Baseline, string Output) Folders()
{
var root = context.CreateTempDirectory();
var folders = (Path.Combine(root, "screenshots"), Path.Combine(root, "baselines"), Path.Combine(root, "diff"));
Directory.CreateDirectory(folders.Item1);
Directory.CreateDirectory(folders.Item2);
return folders;
}
private static DiffReport Run(string captured, string baseline, string output, StringWriter log, int tolerance = DefaultTolerance, double maxDiff = DefaultMaxDiff) =>
ScreenshotsCommand.Diff(new DiffOptions(captured, baseline, output, tolerance, maxDiff), log, annotate: true, stepSummary: null);
[Test]
public void ChangedScreensFailAndNoiseWithinTheToleranceDoesNot()
{
var (captured, baseline, output) = Folders();
foreach (var name in new[] { "library-dark.png", "jobs-dark.png", "about-dark.png", "server-dark.png" }) WriteScreen(baseline, name);
WriteScreen(captured, "library-dark.png");
// Every pixel a little off: anti-aliasing noise below the per-channel tolerance.
WriteScreen(captured, "jobs-dark.png", Shift(Size * Size, DefaultTolerance));
// Four pixels clearly off: 0.04 % of the screen, under the 0.05 % allowed.
WriteScreen(captured, "about-dark.png", Shift(4, 100));
// A whole row moved: 1 % of the screen.
WriteScreen(captured, "server-dark.png", Shift(Size, 100));
using var log = new StringWriter();
var report = Run(captured, baseline, output, log);
Assert.True(report.Fails);
Assert.SequenceEqual(new[] { Verdict.Within, Verdict.Same, Verdict.Same, Verdict.Changed }, report.Screens.Select(s => s.Verdict));
Assert.SequenceEqual(new[] { "about-dark.png", "jobs-dark.png", "library-dark.png", "server-dark.png" }, report.Screens.Select(s => s.Name));
Assert.Equal(Size, report.Screens[3].Differing);
Assert.FileExists(Path.Combine(output, "server-dark-diff.png"));
Assert.FileExists(Path.Combine(output, "about-dark-diff.png"));
Assert.FileMissing(Path.Combine(output, "library-dark-diff.png"));
Assert.Contains("::error title=Screenshot changed::server-dark.png: 1 %25 of the pixels differ (100; limit 0.05 %25)", log.ToString());
Assert.Contains("| server-dark.png | changed | 1 % of the pixels differ (100; limit 0.05 %) |", File.ReadAllText(Path.Combine(output, "summary.md")));
}
[Test]
public void TheDiffImageShowsTheDifferingPixelsInMagentaOverTheFadedBaseline()
{
var (captured, baseline, output) = Folders();
WriteScreen(baseline, "a.png");
WriteScreen(captured, "a.png", Shift(Size, 100));
using var log = new StringWriter();
Run(captured, baseline, output, log);
var diff = ScreenshotsCommand.Read(Path.Combine(output, "a-diff.png"));
Assert.Equal(Size, diff.Width);
Assert.SequenceEqual(new byte[] { 255, 0, 255, 255 }, diff.Pixels[..4], "a differing pixel");
var panel = ((40 * Size) + 50) * 4;
var background = ((90 * Size) + 50) * 4;
Assert.Greater(diff.Pixels[panel], diff.Pixels[background], "the baseline's light panel stays lighter than its background");
Assert.InRange(diff.Pixels[background], 112, 207);
}
[Test]
public void AScreenWithoutABaselineOnlyWarns()
{
var (captured, baseline, output) = Folders();
WriteScreen(captured, "welcome-dark.png");
using var log = new StringWriter();
var report = Run(captured, Path.Combine(baseline, "not-there"), output, log);
Assert.False(report.Fails);
Assert.Equal(Verdict.New, report.Screens.Single().Verdict);
Assert.Contains("::warning title=Screenshots without baseline::1 screenshot(s) have no baseline (welcome-dark.png).", log.ToString());
Assert.Contains("0 compared", log.ToString());
}
[Test]
public void AResizedScreenAndABaselineWithoutItsScreenFail()
{
var (captured, baseline, output) = Folders();
WriteScreen(baseline, "settings-dark.png");
WriteScreen(baseline, "setup-failed-dark.png");
WriteScreen(captured, "settings-dark.png", width: Size + 1);
using var log = new StringWriter();
var report = Run(captured, baseline, output, log);
Assert.SequenceEqual(new[] { Verdict.Resized, Verdict.Missing }, report.Screens.Select(s => s.Verdict));
Assert.True(report.Screens.All(s => s.Fails));
Assert.Contains("settings-dark.png: 101×100, baseline 100×100", log.ToString());
}
[Test]
public void ThePerChannelToleranceIsInclusive()
{
var expected = new Picture(2, 1, [10, 10, 10, 255, 10, 10, 10, 255]);
var actual = new Picture(2, 1, [18, 10, 10, 255, 10, 10, 10, 247]);
Assert.Equal(0L, ScreenshotsCommand.Compare(expected, actual, tolerance: 8).Differing);
Assert.Equal(2L, ScreenshotsCommand.Compare(expected, actual, tolerance: 7).Differing, "alpha counts like a colour");
}
[Test]
public void TheCommandExitsWithOneOnAChangeAndAcceptCopiesTheBaselines()
{
var (captured, baseline, output) = Folders();
WriteScreen(baseline, "a.png");
WriteScreen(captured, "a.png", Shift(Size * 2, 120));
WriteScreen(captured, "b.png");
using var log = new StringWriter();
int Run(params string[] args) => ScreenshotsCommand.Run(args, _ => null, log);
Assert.Equal(1, Run("diff", captured, "--baseline", baseline, "--out", output, "--tolerance", "8", "--max-diff", "0.001"));
Assert.Equal(0, Run("diff", captured, "--baseline", baseline, "--out", output, "--max-diff", "0.05"));
Assert.FileExists(Path.Combine(output, "a-diff.png"));
Assert.Equal(0, Run("accept", captured, "--baseline", baseline));
Assert.FileExists(Path.Combine(baseline, "b.png"));
Assert.Equal(0, Run("diff", captured, "--baseline", baseline, "--out", output));
// The diff of the earlier run is removed with the change it showed.
Assert.FileMissing(Path.Combine(output, "a-diff.png"));
Assert.False(log.ToString().Contains("::", StringComparison.Ordinal), "no workflow commands outside GitHub Actions");
Assert.Throws<ToolException>(() => Run("diff", captured, "--baseline", baseline, "--tolerance", "300"));
Assert.Throws<ToolException>(() => Run("compare", captured));
}
}
+42
View File
@@ -0,0 +1,42 @@
# Screenshot baselines
The approved look of every screen, one PNG per `--capture` screenshot (`<screen>-<theme>.png`, as `Titano.exe`,
`Titano.Uploader.exe` and `Titano.Setup.exe --capture <dir>` write them). CI compares each new screenshot with the
file of the same name here (BRIEF §20.2, visual regression with a pixel-diff tolerance):
```
dotnet run --project src/Titano.Tools -- screenshots diff out/screenshots --out out/screenshot-diff
```
- A pixel differs when one of its channels moves by more than `--tolerance` (default 8 of 255); a screen fails when
more than `--max-diff` of its pixels differ (default 0.0005, that is 0.05 %), when its size changed, or when a
baseline here has no screenshot any more.
- A screenshot without a baseline only warns.
- `out/screenshot-diff` gets `<screen>-diff.png` for every screen with differing pixels (the baseline faded to grey, the
differences magenta) and `summary.md`; CI uploads them as the `screenshot-diff` artifact and adds the summary to the
job page.
## The pictures are reproducible
The capture runs on sample data whose clock stands at `SampleData.Now` (2026-10-02 10:18 UTC), in English with English
formats and with animations off, so the same code draws the same pixels on any day. Two things still come from the
machine:
- the time zone: times are shown in local time, and the baselines are made on the CI runner (`windows-latest`, UTC);
- a few measured values: the server's response time and the time and duration of the last sync comparison. They change
a few dozen pixels, well inside the allowed fraction.
## Adding or updating baselines
Take them from CI, never from a local run (fonts, scaling and the time zone differ). Every same-repository pull request
pushes its screenshots to the `ci-screenshots` branch (one commit, the PNGs at its root). After checking them:
```
git fetch origin ci-screenshots
git worktree add ../ci-screenshots origin/ci-screenshots
dotnet run --project src/Titano.Tools -- screenshots accept ../ci-screenshots
git worktree remove ../ci-screenshots
```
`screenshots accept <dir>` copies every PNG of `<dir>` here. Commit only the screens whose change is intended, and delete
the baseline of a screen that no longer exists.