Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDCSRM-388 Fix flaky unit test timing #62

Merged
merged 2 commits into from
Apr 22, 2024
Merged

Conversation

AdamHawtin
Copy link
Contributor

Motivation and Context

This particular unit test was intermittently failing in the build pipeline when the result wasn't within the arbitrary time window. I spent some time looking at it but couldn't see any good reason this test would run slower than any others, so I'm concluding for now that it's not a particular issue with this function or test performance, and is more likely down to performance hiccups in the build environment, and we simply don't test this timing in many (if any) other places to see the issue. For now, I am fixing it by checking the timing differently, simply checking the timestamp is within the time window of the test.

What has changed

  • Fix flaky unit test timing

How to test?

Check the change. Nothing to test locally, the issue only seems to present in remote build environments.

Links

https://jira.ons.gov.uk/browse/SDCSRM-388

@AdamHawtin AdamHawtin added bug Something isn't working patch A non-feature change, e.g. bug or issue fix labels Apr 19, 2024
Copy link

codecov bot commented Apr 19, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.30%. Comparing base (16c9bbf) to head (a2b1836).

Additional details and impacted files
@@             Coverage Diff              @@
##               main      #62      +/-   ##
============================================
- Coverage     86.05%   82.30%   -3.75%     
+ Complexity      216      210       -6     
============================================
  Files            45       45              
  Lines           667      667              
  Branches         33       33              
============================================
- Hits            574      549      -25     
- Misses           81      102      +21     
- Partials         12       16       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@dbanks91 dbanks91 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your fix looks good.

Might be worth just raising a card so in future we can do performance tests in build environments to see why we might get performance hiccups?

@AdamHawtin
Copy link
Contributor Author

Might be worth just raising a card so in future we can do performance tests in build environments to see why we might get performance hiccups?

@dbanks91 I think it's something for us to keep an eye on, but most of the build jobs have been reliable despite these slow downs, so unless it causes us any proper issues I don't think it's worth any more of our time at the moment.

@AdamHawtin AdamHawtin merged commit 77d2b65 into main Apr 22, 2024
3 checks passed
@AdamHawtin AdamHawtin deleted the SDCSRM-388-fix-test-issues branch April 22, 2024 08:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working patch A non-feature change, e.g. bug or issue fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants