Make sure that perceptuallyCompare
fails if Metal is not available
#702
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
Metal is currently unavailable on Github Actions. The
perceptuallyCompare
diffing algorithm relies on Metal render to obtain the level of correspondence to compare it with the given threshold. Snapshot assertions will falsely succeed, even if a given image does not match the stored reference image.Why
The snapshot assertion should fail if it hasn't got the requirements to execute the tests. Currently, the only indication that something's wrong is hidden inside the logs, stating
I'm aware that there are other initiatives, such as #666, which would be perfect, but until this is solved I think it's important to make people aware that the assertions might be wrong when relying on virtual machines without Metal support.
any real issue, any real inconsistency will be undetected when using
perceptuallyCompare
and running on a virtual machine without Metal support.How
Added a pre-condition check to
perceptuallyCompare
which will fail when Metal is unavailable.Testing
Any image assertion using
perceptuallyCompare
running on an environment without Metal support. See also https://github.com/mschuetz-viz/DemoSnapshotTest