Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable running the snapshot tests for both color modes #4926

Merged
merged 7 commits into from
Sep 19, 2024

Conversation

obulat
Copy link
Contributor

@obulat obulat commented Sep 13, 2024

Fixes

Extracted from #4915
Related to #4305 by @zackkrida

Description

This is one idea of how we can implement testing both dark and light theme for visual regressions without running each test twice. It relies on the expectSnapshot function, which now runs 2 snapshot comparisons instead of one.

First, the default snapshot is taken. Then, the theme switcher is used to change the color mode, and the second snapshot is taken. To make sure that we always run both snapshot checks, even if the first one fails, we use the expect.soft that runs the test to the end, but fails if one of the expects in the test fails.

We use the theme switcher to change the color mode. To do this, we add a themeSwitcher to all stories using a decorator, and use the footer switcher for the tests that use the Nuxt app.

expectSnapshot function is extracted from the breakpoints.ts file. It needs the page argument from Playwright test to allow locating the theme switcher. The optional (nullable) parameters were combined to a separate options object, which allows passing only the necessary options instead of having to pass {} blank object for the unused one.

This PR also adds the theme switcher to the Storybook toolbar, so now you can view both color modes. The Storybook switcher's state is synced with the state of our own theme switcher in the story.

The image cell snapshots had to be updated. I made the width smaller to match the real size of images in the grid, and to make the snapshot files smaller, and to fit all of them into a single screen height. View updated image cell story

Testing Instructions

Go to Storybook and see check that the color theme switcher is working. Also, check the follow-up PRs that add the dark snapshots.

Checklist

  • My pull request has a descriptive title (not a vague title likeUpdate index.md).
  • My pull request targets the default branch of the repository (main) or a parent feature branch.
  • My commit messages follow best practices.
  • My code follows the established code style of the repository.
  • I added or updated tests for the changes I made (if applicable).
  • I added or updated documentation (if applicable).
  • I tried running the project locally and verified that there are no visible errors.
  • I ran the DAG documentation generator (ov just catalog/generate-docs for catalog
    PRs) or the media properties generator (ov just catalog/generate-docs media-props
    for the catalog or ov just api/generate-docs for the API) where applicable.

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

@openverse-bot openverse-bot added 🧱 stack: documentation Related to Sphinx documentation 🧱 stack: frontend Related to the Nuxt frontend 🚦 status: awaiting triage Has not been triaged & therefore, not ready for work labels Sep 13, 2024
@obulat obulat added 🟧 priority: high Stalls work on the project or its dependents 💻 aspect: code Concerns the software code in the repository 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🧱 stack: frontend Related to the Nuxt frontend and removed 🚦 status: awaiting triage Has not been triaged & therefore, not ready for work 🧱 stack: frontend Related to the Nuxt frontend 🧱 stack: documentation Related to Sphinx documentation labels Sep 13, 2024
@obulat obulat changed the base branch from main to add/color-mode-VR-tests September 13, 2024 16:00
@openverse-bot openverse-bot added the 🧱 stack: documentation Related to Sphinx documentation label Sep 13, 2024
@obulat obulat force-pushed the add/color-mode-VR-tests branch 2 times, most recently from 5a985d6 to 6a486ad Compare September 13, 2024 16:57
@obulat obulat force-pushed the enable-color-theme-snapshots branch 2 times, most recently from 72b29c5 to 7d44ee8 Compare September 13, 2024 18:00
@obulat obulat force-pushed the enable-color-theme-snapshots branch 2 times, most recently from 8d08737 to 3becb32 Compare September 13, 2024 19:33
Copy link

Full-stack documentation: https://docs.openverse.org/_preview/4926

Please note that GitHub pages takes a little time to deploy newly pushed code, if the links above don't work or you see old versions, wait 5 minutes and try again.

You can check the GitHub pages deployment action list to see the current status of the deployments.

Base automatically changed from add/color-mode-VR-tests to main September 14, 2024 05:30
@obulat obulat force-pushed the enable-color-theme-snapshots branch 3 times, most recently from 82e3c7a to 6bcc141 Compare September 15, 2024 10:54
@obulat obulat marked this pull request as ready for review September 16, 2024 11:00
@obulat obulat requested a review from a team as a code owner September 16, 2024 11:00
@obulat obulat force-pushed the enable-color-theme-snapshots branch 3 times, most recently from bd11e2e to effb9d2 Compare September 18, 2024 06:36
@obulat obulat self-assigned this Sep 18, 2024
Copy link
Collaborator

@sarayourfriend sarayourfriend left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 from me! This is great, the refactors are nice and all make sense to me in addition to the new expect.soft usage. Maybe there is a way to get rid of the innerExpectSnapshot and multiple getSnapshotName layers, but I'm not too worried about it and didn't have any immediate ideas, that didn't just involve passing through more parameters and having a bigger final getSnapshotName with more conditions, so it's all good to me!

Nice work 🚀 I left some questions and one whitespace nitpick but nothing blocking from my perspective.

onMounted(() => {
document.body.classList.add("bg-default")

watch(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Just curiousity) What is the reason to have these watch calls inside onMounted? Does this effectively delay starting the watching until after the component is ready?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just used the same structure as the withRtl decorator. After your comment, I went back and refactored this. Moved the watch calls out of onMounted. I also found a way of using useGlobals (it didn't work when I try running it in the onMounted, but worked when I moved it out of it), so now the global theme is updated correctly when the Openverse theme switcher value is changed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll make an issue to also refactor the withRtl decorator so that when dir is changed using Openverse footer, the global languageDirection is updated, similar to how with-theme works now.

Comment on lines 20 to 21
}
export type ExpectSnapshot = <T extends Locator | Page>(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Whitespace to help break up the file's components

Suggested change
}
export type ExpectSnapshot = <T extends Locator | Page>(
}
export type ExpectSnapshot = <T extends Locator | Page>(

@@ -102,6 +102,7 @@ export const project: TSESLint.Linter.ConfigType = {
"expectEventPayloadToMatch",
// Shared assertion for visual regression tests
"expectSnapshot",
"expectScreenshotAreaSnapshot",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find this used anywhere. Is it left over from a previous version, and do we still need to include it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is used in storybook visual regression tests, e.g. frontend/test/storybook/visual-regression/v-checkbox.spec.ts. It makes testing the screenshot area (instead of the full page) easier.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah! A diff must have been collapsed when I "ctrl + f" searched for it. Sorry!

@obulat obulat merged commit 50e401f into main Sep 19, 2024
44 checks passed
@obulat obulat deleted the enable-color-theme-snapshots branch September 19, 2024 08:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
💻 aspect: code Concerns the software code in the repository 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🟧 priority: high Stalls work on the project or its dependents 🧱 stack: documentation Related to Sphinx documentation 🧱 stack: frontend Related to the Nuxt frontend
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants