fix(tools): screenshot diff fails a screen without a baseline once any is committed
A screenshot without a baseline only warned, so with no baseline committed every screen was "new" and the CI step could not fail, and a screen added after the first baselines could stay ungated for good. Now the baseline folder is armed as soon as it holds a PNG: from then on a screen without its own baseline fails (Verdict.Unbased, an error annotation). Before the first baselines the run says plainly, in its warning and in bold in the job summary, that it compared nothing and that no visual change can fail it. The baselines README explains why the existing ci-screenshots pictures cannot serve (captured before the clock was fixed: times and dates follow the capture moment) and how the first set is accepted from the first green windows-latest run of the reproducible capture. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012iLMBNwBE2GnphAsu4cjaC
This commit is contained in:
4 files changed
+86
-23
No files matched your search
@@ -118,8 +118,9 @@ jobs:
|
||||
done
|
||||
exit $failed
|
||||
|
||||
# BRIEF §20.2 visual regression against tests/visual/baselines: a screen beyond the tolerance fails, one without
|
||||
# a baseline only warns (the baselines README says how to add them).
|
||||
# BRIEF §20.2 visual regression against tests/visual/baselines: a screen beyond the tolerance fails, and so does one
|
||||
# without a baseline once any baseline is committed; before the first ones the step warns that it compared nothing
|
||||
# (the baselines README says how to add them).
|
||||
- name: Screenshot diff
|
||||
if: runner.os == 'Windows' && !cancelled() && hashFiles('out/screenshots/*.png') != ''
|
||||
run: dotnet "$TOOLS" screenshots diff out/screenshots --baseline tests/visual/baselines --out out/screenshot-diff
|
||||
|
||||
Reference in new issue
Block a user