Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2E tests: screenshot threshold set to 0.2 by default #1630

Closed
scurker opened this issue Aug 13, 2024 · 0 comments · Fixed by #1701
Closed

E2E tests: screenshot threshold set to 0.2 by default #1630

scurker opened this issue Aug 13, 2024 · 0 comments · Fixed by #1701
Labels
bug Something isn't working
Milestone

Comments

@scurker
Copy link
Member

scurker commented Aug 13, 2024

This came up in a recent E2E test where the colors of an existing screenshot comparison "passed" even though the colors had changed. This appears to be due to a default threshold value set via playwright:

https://playwright.dev/docs/api/class-pageassertions#page-assertions-to-have-screenshot-1

An acceptable perceived color difference in the YIQ color space between the same pixel in compared images, between zero (strict) and one (lax), default is configurable with TestConfig.expect. Defaults to 0.2.

Since we're doing component snapshots, we want screenshots to be an exact match so we can adequately determine when a component has changed irrespective of the threshold.

We will need to update the default config to set the threshold values: https://playwright.dev/docs/api/class-testconfig#test-config-expect

@scurker scurker added the bug Something isn't working label Aug 13, 2024
@scurker scurker added this to the Q3 2024 milestone Aug 13, 2024
@scurker scurker modified the milestones: Q3 2024, Q4 2024 Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant