Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Refactor E2E test running #8497

Merged
merged 2 commits into from
Jul 25, 2023
Merged

test: Refactor E2E test running #8497

merged 2 commits into from
Jul 25, 2023

Conversation

mydea
Copy link
Member

@mydea mydea commented Jul 11, 2023

This refactors how we run E2E tests, to a more simple approach.

CI

In CI, we now have a matrix job for the E2E tests, where each test runs in parallel & isolation.

Example: https://github.com/getsentry/sentry-javascript/actions/runs/5541658717

image

Canary

For the canary tests, we also have to specify each canary job we want to run in a matrix.

Example: https://github.com/getsentry/sentry-javascript/actions/runs/5541662790

image

Locally

You can either run all tests:

yarn test:run

Or only run a single test suite:

yarn test:run node-express-app

@mydea mydea self-assigned this Jul 11, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Jul 11, 2023

size-limit report 📦

Path Size
@sentry/browser - ES5 CDN Bundle (gzipped + minified) 21.99 KB (-0.01% 🔽)
@sentry/browser - ES5 CDN Bundle (minified) 69.25 KB (0%)
@sentry/browser - ES6 CDN Bundle (gzipped + minified) 20.33 KB (-0.01% 🔽)
@sentry/browser - ES6 CDN Bundle (minified) 60.48 KB (0%)
@sentry/browser - Webpack (gzipped + minified) 21.95 KB (0%)
@sentry/browser - Webpack (minified) 71.63 KB (0%)
@sentry/react - Webpack (gzipped + minified) 21.96 KB (0%)
@sentry/nextjs Client - Webpack (gzipped + minified) 50.83 KB (0%)
@sentry/browser + @sentry/tracing - ES5 CDN Bundle (gzipped + minified) 30.37 KB (-0.01% 🔽)
@sentry/browser + @sentry/tracing - ES6 CDN Bundle (gzipped + minified) 28.24 KB (-0.01% 🔽)
@sentry/replay ES6 CDN Bundle (gzipped + minified) 49.56 KB (-0.01% 🔽)
@sentry/replay - Webpack (gzipped + minified) 43.33 KB (0%)
@sentry/browser + @sentry/tracing + @sentry/replay - ES6 CDN Bundle (gzipped + minified) 69.73 KB (-0.01% 🔽)
@sentry/browser + @sentry/replay - ES6 CDN Bundle (gzipped + minified) 62 KB (0%)

@mydea mydea force-pushed the fn/e2e-refactor branch 8 times, most recently from a1ff4e0 to 60a6618 Compare July 13, 2023 09:52
@mydea mydea changed the title WIP: Manually run E2E tests on CI? test: Refactor E2E test running Jul 13, 2023
@mydea mydea force-pushed the fn/e2e-refactor branch 2 times, most recently from 2041d6a to 4ec760a Compare July 13, 2023 10:53
@mydea mydea marked this pull request as ready for review July 17, 2023 07:54
@mydea mydea force-pushed the fn/e2e-refactor branch 3 times, most recently from eed11a1 to 338a5ae Compare July 18, 2023 08:26
Copy link
Member

@Lms24 Lms24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! Love the simplifications and that we now immediately see which test failed!

Copy link
Member

@AbhiPrasad AbhiPrasad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good changes!!!

Copy link
Member

@lforst lforst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that we don't run in parallel on the same machine anymore we could switch back to yarn. But LGTM

@mydea
Copy link
Member Author

mydea commented Jul 25, 2023

FYI I also updated the canary issue creation to include which test failed, so we get distinct issues per E2E test failure.

@mydea mydea merged commit 9cf76d8 into develop Jul 25, 2023
86 of 87 checks passed
@mydea mydea deleted the fn/e2e-refactor branch July 25, 2023 10:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants