Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for the part of the event stream query that generates an iter with a fixed key value #200

Open
kimhanbeom opened this issue Apr 8, 2024 · 0 comments

Comments

@kimhanbeom
Copy link
Contributor

kimhanbeom commented Apr 8, 2024

Background

To select the key value of the iterator to use on the next tick in the event_stream, it is processed in the following order

  1. If it is before the first tick call, assign start_time to the detection event time variables.
  2. Extract the time of each event from the iter created by start_time and store it in the detection event time variables.
  3. Select the smallest value of the detection event time variables as the key value for the iter lookup in the next tick.

However, not all detection events are detected due to actual operating environment conditions and protocol/model activation conditions of the hog. That's why on every tick, the key of the iter is fixed to the start_time of the event stream.

This causes unnecessary event time checks and degrades lookup performance over time. To resolve the above issues, we'd like to make the following modifications.

Task

  1. If it is the first tick, store start_time in a specific variable (for example, named iter_time_key).
  2. If the tick is called before a certain amount of time (15 minutes), proceed with the event handling as you were doing before.
  3. If the tick is called after a certain amount of time (15 minutes), find the smallest of the detection event time variables with a value greater than iter_key_time, and store that value in a temporary variable (for example, named min_time_key).
  4. If any of the detection event time variables have the same value as iter_key_time, change it to the value of min_time_key.
  5. Change iter_key_time to the value of min_time_key.
  6. repeat steps 2 through 5.

The reason for the 15 minutes above is to include detection events that are not stored in REview in real time, such as HttpThreat.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant