Performance analysis of cascade operations

[AlexRuiz7]

Description

As a preliminary step towards migrating Wazuh's RBAC from the Server to the Indexer, we need to be aware about the performance of the Indexer on cascade operations involving the change of agents' groups.

A single agent can generate hundreds to thousands of events that end up on indexes. These documents (events) are tied to a single agent, comprising a one-to-one relationship, meaning that a document in an index can only belong to an agent. In order to depict this relationship in the indices, every document contains the agent.id as a primary key that allows these entities to be correlated. Every document also has the field agent.groups to:

• allow groups-based queries over the documents.
• allow filtering the document based on RBAC (roles that can read specific groups of agents only).

The main drawback of this design is that when any agent changes its groups, all the data belonging to that agent until that moment needs to be updated with the new groups of the agent.

To better understand the problem, let's imagine an environment with 50K agents, 10K documents per agent and day, over 30 days.

n_docs = 50.000 * 10.000 * 30 = 15.000.000.000 documents

Over a month, such an environment would have 15K million documents. On a hypothetical, but possible, update of every agent's group, the Indexer would need to perform 15K million update operations as a result.

Environment details

• Comprised by 50K agents.
• Agents distribution by OS:
  • 50% Windows
  • 15% macOS
  • 35% Linux
• Number of groups per agent: 128

Plan

• Simulate the environment.
• Simulate an update on every agent's group (worst case scenario).
• Measure performance of the Indexer during the update of the documents.