Skip to content

Commit

Permalink
[8.x] [Security Solution] Fix flaky test for multiLineStringDiffAlgor…
Browse files Browse the repository at this point in the history
…ithm (#205038) (#205083)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Security Solution] Fix flaky test for multiLineStringDiffAlgorithm
(#205038)](#205038)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Georgii
Gorbachev","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-12-23T11:36:28Z","message":"[Security
Solution] Fix flaky test for multiLineStringDiffAlgorithm
(#205038)\n\n**Fixes:
https://github.com/elastic/kibana/issues/205014**\r\n\r\n##
Summary\r\n\r\nThis test on CI runs at least 10x slower than locally,
and apparently\r\neven 1000ms timeout is not enough.\r\n\r\nBumping it
to 2000ms and hopefully that will be it. Not sure if it makes\r\nsense
to bump it even higher, because we need this threshold to
be\r\nreasonably low, and more than 2 seconds doesn't sound low for
local test\r\nruns.\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests
changed","sha":"8e0561a27b99a99d1b279e6f152b17622a685f1c","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","v9.0.0","Team:Detections
and Resp","Team: SecuritySolution","Team:Detection Rule
Management","Feature:Prebuilt Detection
Rules","backport:version","v8.18.0","v8.16.3","v8.17.1"],"title":"[Security
Solution] Fix flaky test for
multiLineStringDiffAlgorithm","number":205038,"url":"https://github.com/elastic/kibana/pull/205038","mergeCommit":{"message":"[Security
Solution] Fix flaky test for multiLineStringDiffAlgorithm
(#205038)\n\n**Fixes:
https://github.com/elastic/kibana/issues/205014**\r\n\r\n##
Summary\r\n\r\nThis test on CI runs at least 10x slower than locally,
and apparently\r\neven 1000ms timeout is not enough.\r\n\r\nBumping it
to 2000ms and hopefully that will be it. Not sure if it makes\r\nsense
to bump it even higher, because we need this threshold to
be\r\nreasonably low, and more than 2 seconds doesn't sound low for
local test\r\nruns.\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests
changed","sha":"8e0561a27b99a99d1b279e6f152b17622a685f1c"}},"sourceBranch":"main","suggestedTargetBranches":["8.x","8.16","8.17"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205038","number":205038,"mergeCommit":{"message":"[Security
Solution] Fix flaky test for multiLineStringDiffAlgorithm
(#205038)\n\n**Fixes:
https://github.com/elastic/kibana/issues/205014**\r\n\r\n##
Summary\r\n\r\nThis test on CI runs at least 10x slower than locally,
and apparently\r\neven 1000ms timeout is not enough.\r\n\r\nBumping it
to 2000ms and hopefully that will be it. Not sure if it makes\r\nsense
to bump it even higher, because we need this threshold to
be\r\nreasonably low, and more than 2 seconds doesn't sound low for
local test\r\nruns.\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests
changed","sha":"8e0561a27b99a99d1b279e6f152b17622a685f1c"}},{"branch":"8.x","label":"v8.18.0","branchLabelMappingKey":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.16","label":"v8.16.3","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.17","label":"v8.17.1","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Georgii Gorbachev <[email protected]>
  • Loading branch information
kibanamachine and banderror authored Dec 23, 2024
1 parent 316519c commit 9b9ce42
Showing 1 changed file with 3 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -174,11 +174,11 @@ describe('multiLineStringDiffAlgorithm', () => {
const result = multiLineStringDiffAlgorithm(mockVersions);
const endTime = performance.now();

// If the regex merge in this function takes over 1 sec, this test fails
// If the regex merge in this function takes over 2 sec, this test fails
// Performance measurements: https://github.com/elastic/kibana/pull/199388
// NOTE: despite the fact that this test runs in ~50ms locally, on CI it
// runs slower and can be flaky even with a 500ms threshold.
expect(endTime - startTime).toBeLessThan(1000);
// runs ~10x slower and can be flaky even with a 1000ms threshold.
expect(endTime - startTime).toBeLessThan(2000);

expect(result).toEqual(
expect.objectContaining({
Expand Down

0 comments on commit 9b9ce42

Please sign in to comment.