fix(tools): screenshot diff fails a screen without a baseline once any is committed

A screenshot without a baseline only warned, so with no baseline committed every
screen was "new" and the CI step could not fail, and a screen added after the
first baselines could stay ungated for good. Now the baseline folder is armed as
soon as it holds a PNG: from then on a screen without its own baseline fails
(Verdict.Unbased, an error annotation). Before the first baselines the run says
plainly, in its warning and in bold in the job summary, that it compared nothing
and that no visual change can fail it.

The baselines README explains why the existing ci-screenshots pictures cannot
serve (captured before the clock was fixed: times and dates follow the capture
moment) and how the first set is accepted from the first green windows-latest
run of the reproducible capture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012iLMBNwBE2GnphAsu4cjaC
This commit is contained in:
Claude committed 2026-10-12 10:34:25 +00:00
1 parent 8a5f4ab512
commit 2c643fba96
4 files changed
+86 -23

No files matched your search

+3 -2
View File
@@ -118,8 +118,9 @@ jobs:
done
exit $failed
# BRIEF §20.2 visual regression against tests/visual/baselines: a screen beyond the tolerance fails, one without
# a baseline only warns (the baselines README says how to add them).
# BRIEF §20.2 visual regression against tests/visual/baselines: a screen beyond the tolerance fails, and so does one
# without a baseline once any baseline is committed; before the first ones the step warns that it compared nothing
# (the baselines README says how to add them).
- name: Screenshot diff
if: runner.os == 'Windows' && !cancelled() && hashFiles('out/screenshots/*.png') != ''
run: dotnet "$TOOLS" screenshots diff out/screenshots --baseline tests/visual/baselines --out out/screenshot-diff