[FEAT] - Updates and Refactoring for Node 20

[dgrebb]

Requirements

The project should be installed and run with Node 20. As gaps are identified, create issues for larger lifts, or add tasks below to upgrade specific dependencies one at a time.

Tasks

• install BackstopJS from master using Node 20
• update dependencies with ncu
• update Dockerfile to Node 20 ([CHORE]: Update Dockerfile and test with Node 20 (#34) #40)
• validate santiy-test-docker
• validate sanity-test
• validate ember-backstop
• validate remote
• validate test
• validate reference
• validate approve
• create dependency compatibility issues

Compatibility Tests

• test Node 14
• test Node 16
• test Node 18

[garris]

2 things I'd add...

• P0 – validate sanity-test-docker
• as a P1 – validate that basic tests pass in the ember-backstop project as well. There is some risk that changes to remote could cause a regression there.

[dgrebb]

@garris FYA — BackstopJS master works on Node 20 out of the box :)

I'll keep this open as the main issue for all dependency updates pending what we decide in #39.

[dgrebb]

@garris great progress! Please see below failing functional (manual), unit, and integration tests.

Here's a comparison between the feature branch and upstream master

Functional Tests Performed

• npm run init
• npm run reference
• npm run test
• npm run remote
• npm run stop
• npm run approve
• npm run openReport
• npm run report
• npm run unit-test - fails — details below in "Unit Tests Performed"
• npm run build-compare - fails — to be addressed with [BUG] - Update Compare application, dependencies, and webpack configuration #38
• npm run dev-compare - fails — to be addressed with [BUG] - Update Compare application, dependencies, and webpack configuration #38
• npm run genConfig - fails — should this be makeConfig, and exposed in runner.js?
• npm run echo - fails (should this be exposed in runner.js?)
• npm run integration-test - fails — genConfig is not exposed publicly
• npm run sanity-test
• npm run smoke-test - succeeds? mismatch errors found; report opens in browser
• npm run smoke-test-docker - succeeds? mismatch errors found; report opens in
• npm run smoke-test-playwright-docker - succeeds? mismatch errors found; report opens in browser
• npm run sanity-test-palywright - succeeds? mismatch errors found; report opens in browser
• npm run kill-zombies
• npm run copy-report-bundle
• build-and-copy-report-bundle - fails — to be addressed with [BUG] - Update Compare application, dependencies, and webpack configuration #38
• publish-npm
• npm run build-docker
• npm run docker-load
• publish-docker
• build-and-publish
• npm run init-docker-builder

Unit Tests Performed

All passing, except compare_spec.js — where are img1.png and img2.png?

  compare
    - should resolve if compare-hashes succeed
    - should resolve if compare-hashes fail, but compare-resemble succeeds
    - should reject if compare-hashes and compare-resemble fail

Docker Validations Performed

Note: a npm run docker-load script was added to load local buildx multi-platform image without publishing

• cd test/configs/ && node ../../cli/index.js test --docker
• npm run sanity-test-docker
• npm run smoke-test-docker - exits without indication of passing smoke tests; report opens in browser. expected?
• npm run sanity-test-playwright-docker

[garris]

@dgrebb Great! Did you want to book a time to review the different fail cases? Or are you still running these down?

[dgrebb]

@garris sounds good. I'm digging into Ember next. After that, we can review everything in one go. I'll reach out to coordinate a time when everything is ready. Thank you!

[dgrebb]

@garris Ember.js checks out! This is all running with Node 20. Here are:

• ember-playground, which uses the below ember-backstop fork
• ember-backstop: forked with a branch running [email protected]

EmberJS Test Dashboard

In Ember's test runner dashboard, before and after accepting the first backstop run, the report image displays, then disappears. Is this expected?

Backstop Remote running via ember backstop-remote

[garris]

Wow Amazing work!! This is ridiculous -- there will be a lot of backstop users who are super thankful for this!

[garris]

And yeah -- I'm not sure about the QuickStart tests disappearing behavior -- but I am not that concerned about it as long as the tests give consistent results.

[dgrebb]

I'll get a list of items for review and get back to you with some times this week. Thank you!

[dgrebb]

Node Testing

@garris one more update, regarding Node version testing:

Node 18

Works!

Node 16

Works!

Node 14

It appears later versions of puppeteer do not play nice with node@14. Node 14 also requires a downgrade to python <=3.10.x, so maybe it's time to drop support?

added 1022 packages from 768 contributors and audited 1024 packages in 23.222s

209 packages are looking for funding
  run `npm fund` for details

found 4 vulnerabilities (1 low, 2 high, 1 critical)
  run `npm audit fix` to fix them, or `npm audit` for details
❯ npm run sanity-test

> [email protected] sanity-test ~/Projects/BackstopJS
> cd test/configs/ && node ../../cli/index.js test

~/Projects/BackstopJS/node_modules/puppeteer-core/lib/cjs/puppeteer/util/disposable.js:19
Symbol.dispose ??= Symbol('dispose');
               ^^^

SyntaxError: Unexpected token '??='
    at wrapSafe (internal/modules/cjs/loader.js:1029:16)
    at Module._compile (internal/modules/cjs/loader.js:1078:27)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1143:10)
    at Module.load (internal/modules/cjs/loader.js:979:32)
    at Function.Module._load (internal/modules/cjs/loader.js:819:12)
    at Module.require (internal/modules/cjs/loader.js:1003:19)
    at require (internal/modules/cjs/helpers.js:107:18)
    at Object.<anonymous> (~/Projects/BackstopJS/node_modules/puppeteer-core/lib/cjs/puppeteer/common/EventEmitter.js:23:25)
    at Module._compile (internal/modules/cjs/loader.js:1114:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1143:10)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] sanity-test: `cd test/configs/ && node ../../cli/index.js test`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] sanity-test script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     ~/.npm/_logs/2023-11-07T02_17_49_762Z-debug.log
❯ node -v
v14.21.3

[garris]

Agree, let's drop support for 14. Let's please add at the top of the docs requirements section.

[dgrebb]

garris#1523