Vulnerability Detector Syscollector initial scan E2E tests

[Rebits]

Related issue
closes #4590

---

Description

This pull request introduces the initial set of end-to-end (E2E) tests for the Vulnerability Detector module. These tests ensure the proper functionality of the Vulnerability and Syscollector modules in the early stages of the environment. The primary objective is to validate that agents initiate Syscollector scans, generating the expected vulnerabilities, without involving any package installation, removal, or upgrade operations in the environment.

Context

These tests are integral to the ongoing refactoring of the Vulnerability Detector module. Their purpose is to guarantee that the module functions correctly, aligning with the refactoring requirements.

This pull request establishes the foundational steps and essential tools required to proceed with the development of the complete test suite. For more detailed information about the test context, refer to GitHub Issue #4369.

Implemented Cases

The test cases were designed and defined in GitHub Issue #4531 using Gherkin syntax. Two scenarios are covered:

Scenario 1: First Syscollector Scan (Rsync)

1. Given the Manager has updated the feeds and a new Agent is connected.
2. When the Agent's Syscollector scan is executed.
3. Then Vulnerability alerts are generated as "added," and the Agent's System states are stored.

Scenario 2: Subsequent Scan (Dbsync) without any Installation

1. Given the Manager has an agent connected.
2. And a first Syscollector scan has been executed.
3. And no changes have been made to the agent.
4. When a second Syscollector Scan is initiated on the agent.
5. Then the Agent's System states and vulnerability alerts remain unchanged.

Test Environment

The test environment specifications can be found in GitHub Issue #4582. The environment includes multiple hosts with various roles and operating systems.

Vulnerability Detector E2E tests

manager1:
	roles: [manager, filebeat, indexer]
	os: ubuntu_22
	type: master
manager2:
	roles: [manager, filebeat]
	os: ubuntu_22
	type: worker
agent1:
	roles: [agent]
	os: centos_7
	manager: manager1
agent2:
	roles: [agent]
	os: windows_11
	manager: manager2
agent3:
	roles: [agent]
	os: ubuntu_22
	manager: manager1
agent4:
	roles: [agent]
	os: centos_7
	manager: manager1
	arch: arm64v8
agent5:
	roles: [agent]
	os: ubuntu_22
	manager: manager2
	architecture: amd64

How to Launch

pytest test_vulnerability_detector --inventory-path <PATH-INVENTORY>

Implementation Details

Setup Environment

Host Manager Session Fixture

All E2E Vulnerability detector tests will use the host_manager test, returning the HostManager object handling all instance operations. Sharing the same object in a fixture makes it easier to maintain and handle the environments.

Setup Fixture
This fixture prepares the environment by:

• Configuring the environment.
• Starting agents and manager services.
• Waiting until VD feeds are finished in all the managers.
• Truncating Managers Logs.
• Restoring the initial status of the environment.

Test

The test checks the following:

• Syscollector Scan starts and finishes in all the agents in the specified time (~3 minutes).
• Vulnerability Scan is performed over all the agents in the specified time (5 minutes per agent).
• Vulnerability Scan detects at least one vulnerability in the environment, validated using the vulnerability API endpoint.
• Agent's State indices and vulnerability API endpoint results are consistent (To Do).
• Second Syscollector scan is triggered.
• No new vulnerabilities are detected, and Agent's state indices and alerts do not change.

Note

A basic logger for the test has been created using the live log of pytest parameters --log-cli. To increase the default logging level (ERROR, specified in pytest.ini), use --log-cli-level=debug.

Warning

Development was not done following a TDD approach. Some checks might not be fully implemented.

Changes Made in Wazuh QA Framework

New Modules Included

• Included waiters module to pause the test until certain events occur (In this case, Feeds update).
• Changed and included some HostManager methods. This might introduce instabilities to the system tests, so it may be necessary to migrate to another class or perform exhaustive testing before including these tests in production.
• Included regex module to handle event regexes.
• Included a configurations module to handle remote host configurations.
• Included a monitoring module to handle monitoring files in the remote environment.
• Included wazuh_api and indexer_api classes.
• Included logs module to allow handling of the logs files in the environment.
• Remote operation handler module: A middle module that allows performing multiple operations using a basic YAML syntax.

Included a Basic Logger for E2E Tests

Vulnerability detectors E2E tests now have their own logger.

Change Default Enrollment Name in All Hosts

Now agents will be registered using the inventory name.

Known Issues/Limitations

Most of the functionality included in this issue should be performed following an Object-Oriented Programming (OOP) approach instead of using the HostManager class. However, these tests should be refactored in the future when the Framework (present in the qa-system-framework repo) is migrated to this repository.

Future Work

After development is ready for review, it would be necessary to adapt the test in order to:

• check_vuln_state_index: A function to ensure the consistency of the agent's vulnerabilities.
• Check if wait_until_vuln_scan_agents_finished function works as expected for the second scan.

---

Testing performed

Tester	Test path	Jenkins	Local	OS	Commit	Notes
@user (Developer)		🟢	⚫⚫⚫			Nothing to highlight
@user (Reviewer)		⚫⚫⚫	🚫 🚫 🚫			Nothing to highlight

[Rebits]

Currently performing testing after changes

[juliamagan]

LGTM