Refactor invocation of Action listeners in correlations #1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…en (opensearch-project#871) * remove actionGet() and change threat intel feed flow to event driven Signed-off-by: Surya Sashank Nistala <[email protected]> * fix javadocs Signed-off-by: Surya Sashank Nistala <[email protected]> * revert try catch removals Signed-off-by: Surya Sashank Nistala <[email protected]> * use action listener wrap() in detector threat intel code paths Signed-off-by: Surya Sashank Nistala <[email protected]> * add try catch Signed-off-by: Surya Sashank Nistala <[email protected]> --------- Signed-off-by: Surya Sashank Nistala <[email protected]>
Signed-off-by: Megha Goyal <[email protected]>
Signed-off-by: Megha Goyal <[email protected]>
goyamegh
force-pushed
the
main-correlations-copy
branch
from
f653429 to
5c77d65
Compare
Signed-off-by: Megha Goyal <[email protected]>
Signed-off-by: Megha Goyal <[email protected]>
|
Pushed in the main repo. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR is intended to fix the hanging tasks observed in _cat/tasks by refactoring the correlation workflow to ensure timely closure of parent action listeners upon successful completion or encountering exceptions, and consolidating exception handling logic into a centralized function. The aim is to optimize task management efficiency and enhance the overall reliability of our system.
This logic has been tested against a high indexing workload ( approx. 1 M docs/ minute) where the issue was observed prominently in a cluster of 3 or more data nodes, and generating findings with the help of a Cloudtrail logs detector running all 32 pre-packaged rules at a frequency of 1 minute. Further, the correlations were generated with the help of a single rule on the same log type for testing, where the findings are generated at a rate of 1~2k per minute.
Issues Resolved
opensearch-project#879
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.