Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix flakey evaluation form system specs #295

Open
3 tasks
stepchud opened this issue Nov 26, 2024 · 2 comments · May be fixed by #449
Open
3 tasks

Fix flakey evaluation form system specs #295

stepchud opened this issue Nov 26, 2024 · 2 comments · May be fixed by #449
Assignees
@stepchud
Labels
task
Milestone
Sprint 03/11/25

Comments

@stepchud
Copy link
Contributor

stepchud commented Nov 26, 2024 •
edited
Loading

What/Why

As a developer, I want a stable CI build that doesn't give false negative results which slow development.
We have seen a few of the specs in spec/system/evaluation_form_spec.rb fail at random times on CircleCI and sometimes local dev.

Failing specs:

  • evaluation_form_spec.rb:262 # Evaluation Form update evaluation form page allows removing existing criteria
    • Occurences: **
  • evaluation_form_spec.rb:206 # Evaluation Form update evaluation form page allows editing of an existing form values
    • Occurences: *

These are from the same PR that was not updating eval forms:
Image
Image

Acceptance Criteria

  • the rspec step on CircleCI passes 5 times in a row when run from the start

NOTE: the "Re-run workflow from start" option has been observed to have more success than "Re-run workflow from failed" on CircleCI
@stepchud stepchud changed the title Fix flakey Evaluation Form system specs Fix flakey evaluation form system specs Nov 26, 2024
@stepchud stepchud added the task label Nov 26, 2024
@r-bartlett-gsa r-bartlett-gsa added this to the Sprint 12/03/24 milestone Nov 26, 2024
@stepchud stepchud self-assigned this Feb 13, 2025
@stepchud
Copy link
Contributor Author

stepchud commented Feb 19, 2025 •
edited
Loading

@r-bartlett-gsa After spending several hours investigating and troubleshooting this issue, I have found a few issues with our integration tooling that prevent this from being resolved completely. The good news is that the CI builds are passing over 90% of the time and re-running them allows the PRs to merge. I have submitted a support request to the Capybara team to take another look at the issue that has been reported by other users on their repo.
Since I created this ticket to ensure a stable CI build, I think we are not going to be able to resolve the above issue and recommend we push this to MMP 2. I have a WIP PR that has some improvements to enable the team to troubleshoot these issues more easily in the future.

@r-bartlett-gsa
Copy link
Member

@r-bartlett-gsa After spending several hours investigating and troubleshooting this issue, I have found a few issues with our integration tooling that prevent this from being resolved completely. The good news is that the CI builds are passing about 90% of the time and re-running them allows the PRs to merge. I have submitted a support request to the Capybara team to take another look at the issue that has been reported by other users on their repo. Since I created this ticket to ensure a stable CI build, I think we are not going to be able to resolve the above issue and recommend we push this to MMP 2. I have a WIP PR that has some improvements to enable the team to troubleshoot these issues more easily in the future.

@stepchud Let's discuss this at the next dev hour, so the O&M team knows how to handle this (@kkrug, @jdonis, @jairoanaya )

@r-bartlett-gsa r-bartlett-gsa linked a pull request Feb 27, 2025 that will close this issue
[295] Fix flappers #449
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@stepchud stepchud

Labels
task
Projects
None yet
Milestone
Sprint 03/11/25
Development

Successfully merging a pull request may close this issue.

[295] Fix flappers GSA/Challenge_platform
2 participants
@stepchud @r-bartlett-gsa