Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add base for GUI tests #2476

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

GuillaumeGomez
Copy link
Member

As discussed with @ehuss, this PR adds the basics for adding more GUI tests. Just add more .goml files in the tests/gui folder.

This is the same GUI test framework used in rustdoc and docs.rs.

If you need any explanation on how it works, or if you have any question, please don't hesitate to ask.

@rustbot rustbot added the S-waiting-on-review Status: waiting on a review label Nov 8, 2024
@GuillaumeGomez GuillaumeGomez force-pushed the gui-tests branch 2 times, most recently from a3f8cd0 to 1a2991f Compare November 8, 2024 14:39
@GuillaumeGomez GuillaumeGomez reopened this Nov 8, 2024
@GuillaumeGomez GuillaumeGomez force-pushed the gui-tests branch 5 times, most recently from ef31501 to 4b48db3 Compare November 8, 2024 16:46
@GuillaumeGomez
Copy link
Member Author

Finally fixed windows build. :')

@notriddle
Copy link
Contributor

notriddle commented Nov 19, 2024

I think we should not use the default test harness for this. We can lean on Cargo a lot more instead of using environment variables:

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 984e19222..1f9c84522 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -58,11 +58,9 @@ jobs:
     - name: Install browser-ui-test
       if: matrix.os != 'windows-latest'
       run: npm install browser-ui-test@"${BROWSER_UI_TEST_VERSION}"
-    - name: Build and run tests (+ GUI)
+    - name: Build and run GUI tests
       if: matrix.os != 'windows-latest'
-      env:
-        GUI_TESTS: 1
-      run: cargo test --locked --target ${{ matrix.target }}
+      run: cargo test --locked --target ${{ matrix.target }} --test gui
     - name: Build and run tests
       if: matrix.os == 'windows-latest'
       run: cargo test --locked --target ${{ matrix.target }}
diff --git a/Cargo.toml b/Cargo.toml
index 5d593b270..3d3704390 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -82,3 +82,10 @@ name = "remove-emphasis"
 path = "examples/remove-emphasis/test.rs"
 crate-type = ["lib"]
 test = true
+
+[[test]]
+harness = false
+test = false
+name = "gui"
+path = "tests/gui/runner.rs"
+crate-type = ["bin"]
diff --git a/tests/gui.rs b/tests/gui/runner.rs
similarity index 93%
rename from tests/gui.rs
rename to tests/gui/runner.rs
index 92d9df319..0ec7f5456 100644
--- a/tests/gui.rs
+++ b/tests/gui/runner.rs
@@ -38,12 +38,7 @@ fn expected_browser_ui_test_version() -> String {
     panic!("failed to retrieved `browser-ui-test` version");
 }
 
-#[test]
-fn gui() {
-    if !std::env::var("GUI_TESTS").is_ok_and(|value| value == "1") {
-        eprintln!("Skipping GUI tests. To enable them, add the GUI_TESTS=1 environment variable.");
-        return;
-    }
+fn main() {
     let browser_ui_test_version = expected_browser_ui_test_version();
     match get_available_browser_ui_test_version() {
         Some(version) => {
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index dcbb9dbb4..347f3a04f 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -141,7 +141,7 @@ If possible, do your best to avoid breaking older browser releases.
 GUI tests are checked with the GUI testsuite. To run it, you need to install `npm` first. Then run:
 
 ```
-GUI_TESTS=1 cargo test gui
+cargo test --test gui
 ```
 
 The first time, it'll fail and ask you to install the `browser-ui-test` package. Install it then re-run the tests.

@GuillaumeGomez
Copy link
Member Author

Great idea, thanks! Updated.

CONTRIBUTING.md Outdated Show resolved Hide resolved
tests/gui/runner.rs Outdated Show resolved Hide resolved
CONTRIBUTING.md Outdated
cargo test --test gui -- --disable-headless-test
```

You can find the `browser-ui-test` package documentation on [its repository](https://github.com/GuillaumeGomez/browser-UI-test/blob/master/goml-script.md).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is missing a lot of information on how those are written. Something like this?

The GUI tests are in the directory tests/gui in text files with the .goml extension. These tests are run using a node.js program called browser-ui-test. You can find documentation for this language on its repository.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, thanks! Hard to know what is missing when you're too involved in something. ^^'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: waiting on a review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants