Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate from jest to vitest and stop using jsdom based tests #11084

Open
23 of 30 tasks
AbhiPrasad opened this issue Mar 13, 2024 · 5 comments
Open
23 of 30 tasks

Migrate from jest to vitest and stop using jsdom based tests #11084

AbhiPrasad opened this issue Mar 13, 2024 · 5 comments
Labels
Type: Tests

Comments

@AbhiPrasad
Copy link
Member

AbhiPrasad commented Mar 13, 2024 •
edited
Loading

After v8 gets merged in, let's look at migrating from jest to vitest. vitest is way faster, and doesn't fall into the same traps jest does in terms of esm compatibility. Example attempt with Vue SDK: #11071

In addition, we should stop using jsdom based tests, and instead move tests that rely on the browser to use playwright instead. Simulating jsdom in an node environment always has it's faults, easier to not attempt to do that.

First we should align on a common vitest testing standard. Right now we use set globals via vitest/globals, but this is not recommended by vitest themselves. We should instead use direct imports like import { describe, test, expect } from 'vitest';.

sentry-javascript/tsconfig.dev.json

Line 8 in 780875f

"types": ["node", "jest", "vitest/globals"]

There's probably some other discussion that needs to be done here too. We can validate our decisions by changing our vitest usage in the Astro and SvelteKit SDKs.

Update existing vitest usage
Beta Give feedback
  1. test(astro): Switch to explicit vitest imports #13093
    AbhiPrasad
  2. test(nestjs): Switch to explicit vitest imports #13214
    AbhiPrasad
  4. test(solid): Switch to explicit vitest imports #13028
    AbhiPrasad
  6. test(svelte): Switch to explicit vitest imports #13026
    AbhiPrasad
  7. test(sveltekit): Switch to explicit vitest imports #12960
    AbhiPrasad
  8. test(replay): Switch to explicit vitest imports #13027
    AbhiPrasad

We can then tackle the following list in whatever order we want!

Convert to using vitest
Beta Give feedback
  2. test(browser): Switch from jest to vitest #13092
    AbhiPrasad
  3. Switch to vitest in @sentry/core #13434
  7. test(node): Switch from Jest to vitest #13605
    timfish
  9. test(replay-canvas): Switch to using vitest #12964
    AbhiPrasad
  10. feat(replay): Use vitest instead of jest #11899
  11. test(replay): Switch to using vitest for replay worker #12961
    AbhiPrasad
  12. test(vercel-edge): Switch to using vitest #12958
    AbhiPrasad
  13. test(vue): Switch to using vitest #12955
    AbhiPrasad

We also have some packages that use mocha. We should try to drop that dependency and make them use vitest

Remove mocha
Beta Give feedback
  1. ref(eslint): Remove mocha #11296
    AbhiPrasad
  2. test: Port captureException and captureMessage tests from karma runner #11412
    AbhiPrasad
  3. test: Port breadcrumb tests from karma runner #11436
    AbhiPrasad
  4. test: Port ignoreErrors and denyUrls tests from karma runner #11449
    AbhiPrasad
  5. test: Port onerror tests to playwright #11666
    AbhiPrasad
  6. test: Port loader tests from karma to playwright #11688
    AbhiPrasad
  7. test(browser): Port builtin tests to playwright #11733
    AbhiPrasad
  8. test(browser): Port unhandledrejection tests to playwright #11758
    AbhiPrasad
@AbhiPrasad
Copy link
Member Author

ref #6040

@timfish timfish mentioned this issue Mar 13, 2024
Merged
@s1gr1d
Copy link
Member

s1gr1d commented Mar 22, 2024

vistest was already added to Angular here: #11091
I will check it on the list 👍🏻

@AbhiPrasad AbhiPrasad mentioned this issue Mar 26, 2024
Merged
AbhiPrasad added a commit that referenced this issue Mar 27, 2024
@AbhiPrasad
ref(eslint): Remove mocha (#11296)
dde1b3d 
ref #11084

Removes usage of `mocha` from eslint plugin
@AbhiPrasad AbhiPrasad mentioned this issue Apr 3, 2024
Merged
AbhiPrasad added a commit that referenced this issue Apr 4, 2024
@AbhiPrasad
test: Port captureException and captureMessage tests from karma runner (
e307608 
#11412)

I want to remove the karma/mocha based tests in the browser package. To
accomplish this, I'll be porting 1 test suite a day from the old
integration tests to playwright. Today is Day 1:
`packages/browser/test/integration/suites/api.js`

We have decent coverage around singular calls to these methods, so I
just added the tests that validate calling `captureException` and
`captureMessage` multiple times.

I also fixed a spelling mistake with
`dev-packages/browser-integration-tests/suites/public-api/captureException/linkedErrors/subject.js`,
`linkedErrrors` -> `linkedErrors`

ref #11084
@AbhiPrasad AbhiPrasad mentioned this issue Apr 4, 2024
Merged
AbhiPrasad added a commit that referenced this issue Apr 5, 2024
@AbhiPrasad
test: Port breadcrumb tests from karma runner (#11436)
658898c 
I want to remove the karma/mocha based tests in the browser package. To
accomplish this, I'll be porting 1 test suite a day from the old
integration tests to playwright. Today is Day 2:
`packages/browser/test/integration/suites/breadcrumbs.js`

This adds console and history breadcrumb tests (which didn't exist
before), and expands upon the dom and xhr/fetch tests, and cleans up
some code here and there as well.

ref #11084
day 1: #11412
@AbhiPrasad AbhiPrasad mentioned this issue Apr 5, 2024
Merged
AbhiPrasad added a commit that referenced this issue Apr 8, 2024
@AbhiPrasad
test: Port ignoreErrors and denyUrls tests from karma runner (#11449)
091d23b 
I want to remove the karma/mocha based tests in the browser package. To
accomplish this, I'll be porting 1 test suite a day from the old
integration tests to playwright. Today is Day 3:
`packages/browser/test/integration/suites/config.js`

I was surprised we never had `ignoreErrors` or `denyUrls` tests in
playwright, so it's good to get the confidence that everything works
here.

ref #11084
day 2: #11436
This was referenced Apr 18, 2024
Merged
Merged
AbhiPrasad added a commit that referenced this issue Apr 19, 2024
@AbhiPrasad
test: Port loader tests from karma to playwright (#11688)
fce8e8c 
This ports `packages/browser/test/integration/suites/loader.js` and
`packages/browser/test/integration/suites/loader-specific.js` to
playwright. Specifically it just adds tests for SDK source and
breadcrumbs, which were missing previously.

ref #11084
cadesalaberry pushed a commit to cadesalaberry/sentry-javascript that referenced this issue Apr 19, 2024
@AbhiPrasad @cadesalaberry
ref(eslint): Remove mocha (getsentry#11296)
c85f5c6 
ref getsentry#11084

Removes usage of `mocha` from eslint plugin
cadesalaberry pushed a commit to cadesalaberry/sentry-javascript that referenced this issue Apr 19, 2024
@AbhiPrasad @cadesalaberry
test: Port captureException and captureMessage tests from karma runner (
1f40194 
getsentry#11412)

I want to remove the karma/mocha based tests in the browser package. To
accomplish this, I'll be porting 1 test suite a day from the old
integration tests to playwright. Today is Day 1:
`packages/browser/test/integration/suites/api.js`

We have decent coverage around singular calls to these methods, so I
just added the tests that validate calling `captureException` and
`captureMessage` multiple times.

I also fixed a spelling mistake with
`dev-packages/browser-integration-tests/suites/public-api/captureException/linkedErrors/subject.js`,
`linkedErrrors` -> `linkedErrors`

ref getsentry#11084
cadesalaberry pushed a commit to cadesalaberry/sentry-javascript that referenced this issue Apr 19, 2024
@AbhiPrasad @cadesalaberry
test: Port breadcrumb tests from karma runner (getsentry#11436)
cfeb509 
I want to remove the karma/mocha based tests in the browser package. To
accomplish this, I'll be porting 1 test suite a day from the old
integration tests to playwright. Today is Day 2:
`packages/browser/test/integration/suites/breadcrumbs.js`

This adds console and history breadcrumb tests (which didn't exist
before), and expands upon the dom and xhr/fetch tests, and cleans up
some code here and there as well.

ref getsentry#11084
day 1: getsentry#11412
cadesalaberry pushed a commit to cadesalaberry/sentry-javascript that referenced this issue Apr 19, 2024
@AbhiPrasad @cadesalaberry
test: Port ignoreErrors and denyUrls tests from karma runner (getsent…
9671596 
…ry#11449)

I want to remove the karma/mocha based tests in the browser package. To
accomplish this, I'll be porting 1 test suite a day from the old
integration tests to playwright. Today is Day 3:
`packages/browser/test/integration/suites/config.js`

I was surprised we never had `ignoreErrors` or `denyUrls` tests in
playwright, so it's good to get the confidence that everything works
here.

ref getsentry#11084
day 2: getsentry#11436
This was referenced Apr 22, 2024
Merged
Merged
AbhiPrasad added a commit that referenced this issue Apr 24, 2024
@AbhiPrasad
test(browser): Port unhandledrejection tests to playwright (#11758)
3eea537 
ref #11084

This test ports
`packages/browser/test/integration/suites/onunhandledrejection.js`
playwright. Because of the same limitations as outlined with the on
error tests #11666, I
had to use calls to `window.onunhandledrejection` to simulate these
tests instead of just using `Promise.reject` to test the handler.

#11678 tracks being
able to fix this so we can avoid directly calling
`window.onunhandledrejection` to test.

As `onunhandledrejection.js` was the last suite to use the old
integration tests, I fully removed that code and the corresponding GH
action workflow. I also removed the monorepo deps on `karma`, `chai` and
`sinon`. Extremely satisfying.
This was referenced Jul 17, 2024
Merged
Merged
Closed
AbhiPrasad added a commit that referenced this issue Jul 17, 2024
@AbhiPrasad
test(vercel-edge): Switch to using vitest (#12958)
80d76c9 
Before: `Time: 2.621 s`

After: `Duration 638ms (transform 343ms, setup 0ms, collect 1.61s, tests
22ms, environment 0ms, prepare 278ms)`

ref #11084

Also removes `edge-runtime/jest-environment` which we weren't using
anyway, removes a lot of unnecessary stuff in our `yarn.lock` (-151
lines!)
This was referenced Jul 17, 2024
Merged
Merged
@billyvg
Copy link
Member

billyvg commented Jul 17, 2024 •
edited
Loading

@AbhiPrasad Replay was migrated here: #11899

edit though we are using globals and not importing everything from vitest

@AbhiPrasad AbhiPrasad mentioned this issue Jul 17, 2024
Merged
This was referenced Jul 23, 2024
Merged
Merged
Merged
Closed
andreiborza pushed a commit that referenced this issue Jul 24, 2024
@AbhiPrasad
test(solid): Switch to explicit vitest imports (#13028)
b2dded8 
As per https://vitest.dev/config/#globals

> By default, vitest does not provide global APIs for explicitness

I think we should follow vitest defaults here and explicitly import in
the APIs that we need. This refactors our Solid SDK tests to do so.

ref #11084

This change also removes `environment: 'jsdom'` from the vite config as
it seems nothing needs this for solid. This should means that our tests
are not polluted with jsdom globals, and that future writers have to
explicitly opt-in to the behaviour.
AbhiPrasad added a commit that referenced this issue Jul 24, 2024
@AbhiPrasad
test(svelte): Switch to explicit vitest imports (#13026)
3c7e220 
As per https://vitest.dev/config/#globals

> By default, vitest does not provide global APIs for explicitness

I think we should follow vitest defaults here and explicitly import in
the APIs that we need. This refactors our Svelte SDK tests to do so.

ref #11084

This change also removes `environment: 'jsdom'` from the vite config in
favour of explicitly adding jsdom environment via the
`@vitest-environment` pragma to the specific test file that needs it.
This should means that our server tests are not polluted with jsdom
globals, and that future writers have to explicitly opt-in to the
behaviour.

I was also getting some ts errors in the tests, so addressed those as
well.
This was referenced Jul 29, 2024
Merged
Merged
Merged
AbhiPrasad added a commit that referenced this issue Aug 6, 2024
@AbhiPrasad
test(nestjs): Switch to explicit vitest imports (#13214)
0666eb5 
As per https://vitest.dev/config/#globals

> By default, vitest does not provide global APIs for explicitness

I think we should follow vitest defaults here and explicitly import in
the APIs that we need. This refactors our Nestjs SDK tests to do so.

ref #11084
@timfish
Copy link
Collaborator

timfish commented Sep 6, 2024

Looks like the task list needs updating because Node v14 doesn't support vitest?

@AbhiPrasad
Copy link
Member Author

I updated it - looks like we are stuck with jest longer than I thought.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Type: Tests
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@billyvg @timfish @AbhiPrasad @s1gr1d