Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

playwright-test: init at 1.31.1 #227071

Merged
merged 1 commit into from
May 4, 2023
Merged

playwright-test: init at 1.31.1 #227071

merged 1 commit into from
May 4, 2023

Conversation

teto
Copy link
Member

@teto teto commented Apr 19, 2023

Adds "playwright test" command.

Description of changes

Raw dump of a package working in an out of tree project, still needs some polish
Fixes #217693

I've tried buildNpmPackage but doesn't seem ready yet to handle this kind of package.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.05 Release Notes (or backporting 22.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Adds "playwright test" command.
@teto teto force-pushed the playwright-test branch from ed27884 to 9665f56 Compare April 27, 2023 15:36
@teto teto marked this pull request as ready for review April 27, 2023 19:36
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/running-playwright-tests/25655/9

@pbek
Copy link
Contributor

pbek commented Apr 28, 2023

I built the PR and run the resulting binary in a (unfortunately private) repository with Playwright tests written in Javascript:

/nix/store/8ck7islaxlgffm584kf91rq05ygigima-_at_playwright_slash_test-1.31.1/bin/playwright test

gave me errors like

Error: Playwright Test did not expect test() to be called here.
Most common reasons include:
- You are calling test() in a configuration file.
- You are calling test() in a file that is imported by the configuration file.
- You have two different versions of @playwright/test. This usually happens
  when one of the dependencies in your package.json depends on @playwright/test.

   at mono/mono.spec.js:7

   5 |     await page.goto('https://url.domain');
   6 |     await expect(page).toHaveURL(/auth\/realms/);
>  7 |     await page.locator('#username').fill(process.env.STUDENT_USERNAME ?? '');
     | ^
   8 |     await page.locator('#password').fill(process.env.STUDENT_PASSWORD ?? '');
   9 |     await page.locator('button[type="submit"]').click()
  10 |     await expect(page).toHaveURL(/mono/);

    at TestTypeImpl._currentSuite (/home/user/dev/e2e-tests-ng/node_modules/@playwright/test/lib/common/testType.js:70:13)
    at TestTypeImpl._createTest (/home/user/dev/e2e-tests-ng/node_modules/@playwright/test/lib/common/testType.js:76:24)
    at /home/user/dev/e2e-tests-ng/node_modules/@playwright/test/lib/common/transform.js:226:12
    at Object.<anonymous> (/home/user/dev/e2e-tests-ng/tests/mono/mono.spec.js:7:1)

Do I need to do something to my test project first?

@teto
Copy link
Member Author

teto commented Apr 28, 2023

You have two different versions of @playwright/test.

Ha I have seen that error at some point. I was told that javascript has a peculiar singleton system based on path that even symlinks could throw off. This package is a port of a package we use internally with success and that shouldn't suffer from the problem.
Why is it that the path is a local one ? /home/user/dev/e2e-tests-ng/node_modules/@playwright/test/lib/common/testType.js:76:24
it should come from the nix store

So maybe it's a plawyright error. Have you tried clearning up NODE_PATH in case it does load an alternative playwright ?

@pbek
Copy link
Contributor

pbek commented Apr 28, 2023

My NODE_PATH is NODE_PATH=/nix/store/njh6p007ymbmcgas5l7sqqxfmm394zx1-nodejs-18.15.0/lib/node_modules.
Setting it to an empty string didn't help. 🤔

Only when I was manually removing node_modules/@playwright/ (note that it will be installed again by npm install) I managed to use the test successfully in Chromium!

But not in Firefox, there I got:

browserType.launch: Executable doesn't exist at /nix/store/dkpc25ivrhx19d5v9px74qnsafn65hlr-playwright-browsers-chromium/firefox-1378/firefox/firefox

I don't know where it gets the path from, /nix/store/dkpc25ivrhx19d5v9px74qnsafn65hlr-playwright-browsers-chromium is the path for the chromium browser. Any chance to make it work in firefox too?

My shell.nix is:

{ pkgs ? import <nixpkgs> {} }:
  pkgs.mkShell {
    buildInputs = [
      pkgs.nodejs-18_x
      pkgs.playwright
    ];

    PLAYWRIGHT_BROWSERS_PATH="${pkgs.playwright-driver.browsers}";
  }

env said:
PLAYWRIGHT_BROWSERS_PATH=/nix/store/dkpc25ivrhx19d5v9px74qnsafn65hlr-playwright-browsers-chromium

And is there a chance to make it work inside VS-Code with the Playwright extension (where you are supposed to do the actual development of the tests)?

@aszenz
Copy link
Contributor

aszenz commented Apr 28, 2023

And is there a chance to make it work inside VS-Code with the Playwright extension (where you are supposed to do the actual development of the tests)?

Add this to your vscode settings

"playwright.env": {
    "PLAYWRIGHT_BROWSERS_PATH": "/nix/store/mlyamflq8icq0lwfircr03aiba0dbzg3-chromium-playwright",
    "PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS": 1
  },

Unfortunately u have to hard code the nix store path, maybe there's a way to read it from env var in vscode

@pbek
Copy link
Contributor

pbek commented Apr 28, 2023

Hm, I tried to do that, but vs-code doesn't even seem to detect my JS tests any more. It want to show python tests. 😅

pbek added a commit to pbek/playwright-example that referenced this pull request May 3, 2023
@pbek
Copy link
Contributor

pbek commented May 3, 2023

I created https://github.com/pbek/playwright-example with an example playwright test to test this PR as follow-up from #215450 (comment).

https://github.com/pbek/playwright-example/blob/main/shell.nix uses this PR.

@teto
Copy link
Member Author

teto commented May 4, 2023

awesome, maybe we can integrate your repository as part of the testsuite ?

Seems like it works for the only supported browser aka chrome:

Running 3 tests using 3 workers
  1) [webkit] › example.spec.js:4:1 › has title ────────────────────────────────────────────────────

    browserType.launch: Executable doesn't exist at /nix/store/fpibn255xp5iazcdnghbf06qgbacrqv3-playwright-browsers-chromium/webkit-1792/pw_run.sh
    ╔═════════════════════════════════════════════════════════════════════════╗
    ║ Looks like Playwright Test or Playwright was just installed or updated. ║
    ║ Please run the following command to download new browsers:              ║
    ║                                                                         ║
    ║     npx playwright install                                              ║
    ║                                                                         ║
    ║ <3 Playwright Team                                                      ║
    ╚═════════════════════════════════════════════════════════════════════════╝

  2) [firefox] › example.spec.js:4:1 › has title ───────────────────────────────────────────────────

    browserType.launch: Executable doesn't exist at /nix/store/fpibn255xp5iazcdnghbf06qgbacrqv3-playwright-browsers-chromium/firefox-1378/firefox/firefox
    ╔═════════════════════════════════════════════════════════════════════════╗
    ║ Looks like Playwright Test or Playwright was just installed or updated. ║
    ║ Please run the following command to download new browsers:              ║
    ║                                                                         ║
    ║     npx playwright install                                              ║
    ║                                                                         ║
    ║ <3 Playwright Team                                                      ║
    ╚═════════════════════════════════════════════════════════════════════════╝

  2 failed
    [firefox] › example.spec.js:4:1 › has title ────────────────────────────────────────────────────
    [webkit] › example.spec.js:4:1 › has title ─────────────────────────────────────────────────────
  1 passed (6.0s)

  Serving HTML report at http://localhost:9323. Press Ctrl+C to quit.
  

I am tempted to merge as is and discuss the test afterwards. There might even be a native testsuite that could be run

@pbek
Copy link
Contributor

pbek commented May 4, 2023

awesome, maybe we can integrate your repository as part of the testsuite ?

fine with me! 🚀

@teto
Copy link
Member Author

teto commented May 4, 2023

Let's go !

@teto teto merged commit 6a8cb45 into NixOS:master May 4, 2023
@teto teto deleted the playwright-test branch May 4, 2023 13:43
@pbek
Copy link
Contributor

pbek commented May 5, 2023

@teto, do you maybe know why there is no --ui parameter, as in playwright test --ui? (see https://playwright.dev/docs/test-cli)

@teto
Copy link
Member Author

teto commented May 5, 2023

nope sry no idea, documentation is not clear to me either.

@samhh
Copy link
Contributor

samhh commented May 9, 2023

@pbek That was added in 1.32.0.

@pbek
Copy link
Contributor

pbek commented May 9, 2023

Ah, I see, thank you :)
So we'll need to wait for @toto. 😅

@samhh
Copy link
Contributor

samhh commented May 9, 2023

@teto We're also keen on a newer version at work. What would be the process of updating? Do we need to think about the driver/browsers as well?

@toto
Copy link

toto commented May 9, 2023

I think you mean @teto not @toto ;)

@teto
Copy link
Member Author

teto commented May 9, 2023

I am sure you could sell @toto for a shitton of money, if only to https://en.wikipedia.org/wiki/Toto_Ltd.

@samhh Absolutely, the playwright-driver version must match the package version ! we could add a warning to check that
the playwright python package has this script to update the driver that I might have broken ?!
https://github.com/teto/nixpkgs/blob/3beded5aa4b3e20e510b2eea292c3f49fbe44341/pkgs/development/python-modules/playwright/update.sh#L24
I dont know if nix-update can replace that script (probably not) but I guess the last lines should be moved to pkgs/development/web/playwright/

I have spent my plawyright FOSS credits but I am happy to review PRs if you wanna update the package.

pkgs/development/web/playwright-test/wrapped.nix was written manually but depends on the driver version. Updating the driver and running pkgs/development/web/playwright-test $ node2nix -i package.json (from memory) should work

@pbek
Copy link
Contributor

pbek commented May 9, 2023

I think you mean @teto not @toto ;)

I'm so sorry! 😆

@kalekseev
Copy link
Contributor

@samhh pr with latest playwright version #235914

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Package request: playwright test
7 participants