Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Flaky CI]: Browser Integration Tests #8589

Closed
Lms24 opened this issue Jul 19, 2023 · 4 comments
Closed

[Flaky CI]: Browser Integration Tests #8589

Lms24 opened this issue Jul 19, 2023 · 4 comments
Labels
Type: Tests

Comments

@Lms24
Copy link
Member

Lms24 commented Jul 19, 2023

Flakiness Type

Other / Unknown

Name of Job

bundle_tracing_replay_es6

Name of Test

  1. [firefox] › suites/replay/dsc/test.ts:11:11 › should add replay_id to dsc of transactions when in session mode

Link to Test Run

https://github.com/getsentry/sentry-javascript/actions/runs/5600029443/jobs/10242097476#step:10:322

Details

Hmm maybe a firefox problem again?
@onurtemizkan
Copy link
Collaborator

@Lms24 - It seems Playwright is in trouble with Firefox support. We tried disabling specific tests before, but flakiness always came back somewhere else.

What would you guys feel about disabling Firefox tests entirely until microsoft/playwright#12182 is resolved properly?

@Lms24
Copy link
Member Author

Lms24 commented Jul 20, 2023

@onurtemizkan Probably worth considering. I'll bring it up with the team.

@mydea mydea mentioned this issue Jul 21, 2023
Merged
mydea added a commit that referenced this issue Jul 25, 2023
@mydea
test: Only run specified integration tests with firefox (#8603)
428598d 
As a possible fix for
#8589, this sets up
our integration tests to only run tests in firefox when the test
contains `@firefox`. This way, we manually tag tests we want to run in
firefox, as the tests are quite often flaky.

We may think about reverting this once
microsoft/playwright#12182 is fixed.
@AbhiPrasad
Copy link
Member

I'm gonna close this because from looking at recent CI and our CI dashboard in Sentry, CI seems to be smooth re: the firefox tests.

@AbhiPrasad
Copy link
Member

ref #8603 and #8662

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Type: Tests
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@onurtemizkan @Lms24 @AbhiPrasad