Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge 4.8.1 into 4.9.0 #5552

Merged
merged 94 commits into from
Jul 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
77da068
fix: update wazuh_template_branch to 4.8.1
Rebits Jun 14, 2024
a21ace0
feat: add method to unify process values
rafabailon Jun 14, 2024
854660f
feat: modify conditional
rafabailon Jun 14, 2024
01333c0
feat: fix linter errors
rafabailon Jun 14, 2024
053b78c
feat: include abuseControl timeout to initial VD tests
Rebits Jun 17, 2024
4f12863
feat: rename unify variable
rafabailon Jun 18, 2024
0c04723
feat: change in the conditional to unify
rafabailon Jun 18, 2024
76c2ea9
feat: modify the logic of unification
rafabailon Jun 18, 2024
91ad2fe
feat: Improvement of the unification logic
rafabailon Jun 18, 2024
7433697
feat: increase time in 30 minutes always
Rebits Jun 18, 2024
8b40e58
style: remove extra whitespaces
Rebits Jun 18, 2024
b84647a
refac: visualization module
Rebits Jun 18, 2024
aa36bf7
docs: include 5432 changelog
Rebits Jun 19, 2024
5757bb6
feat: change the argument information to unify
rafabailon Jun 19, 2024
ee85a1d
feat: change conditional, argument and docstring
rafabailon Jun 19, 2024
0a9dc21
Merge pull request #5499 from wazuh/fix/5498-change-provision-template
juliamagan Jun 19, 2024
94364c2
refac: DataVisualization class
Rebits Jun 19, 2024
0ed0388
Merge pull request #5507 from wazuh/fix/5501-timeout-avoid-cooldown
juliamagan Jun 20, 2024
e2a692b
fix: remove hardcoded branch
santipadilla Jun 20, 2024
365b76b
fix: remove hardcoded branch
santipadilla Jun 20, 2024
90d45d9
feat: update changelog
rafabailon Jun 20, 2024
548d189
Merge branch '4.8.1' into enhancement/5479-unify-process-values
rafabailon Jun 20, 2024
0e07651
Merge pull request #5500 from wazuh/enhancement/5479-unify-process-va…
juliamagan Jun 20, 2024
c294f2f
feat: adapt data_visualization script
Rebits Jun 20, 2024
38584b5
refac: remove csv binary headers unnused file
Rebits Jun 20, 2024
b6615d0
fix: replace cluster_log by disable_x_labels
Rebits Jun 20, 2024
4f39ed9
fix: extra . in statistics title
Rebits Jun 21, 2024
27e4dce
fix: extra . in statistics title
Rebits Jun 21, 2024
46816f5
fix: bug in statistics method
Rebits Jun 21, 2024
649237b
Merge branch '4.8.1' into feat/6571-indexer-stats-plot
Rebits Jun 21, 2024
1a7e0d7
docs: include 5518 changelog
Rebits Jun 21, 2024
9782a68
docs: improve 5432 changelog
Rebits Jun 21, 2024
25de9a1
style: fix pep8 to data visualization
Rebits Jun 21, 2024
3bd3ce2
style: replace " by '
Rebits Jun 21, 2024
7f90476
style: pep8 fix
Rebits Jun 21, 2024
e36d0a1
docs: include datavisualization class docstrings
Rebits Jun 21, 2024
40c0a94
fix: remove hardcode branch reference
santipadilla Jun 24, 2024
1c333bb
fix: temporal removal of macos
santipadilla Jun 24, 2024
70f1724
fix: add debug task
santipadilla Jun 24, 2024
ed291d2
fix: capital letters
santipadilla Jun 24, 2024
995d39b
fix: remove debug println
santipadilla Jun 24, 2024
0312c7e
refac: refactor strategy selection logic
Rebits Jun 25, 2024
6c405dc
fix: docstring indentation
Rebits Jun 25, 2024
be0afc8
docs: fix docstrings styling
Rebits Jun 25, 2024
9430c86
refac: supported daemon statistics list
Rebits Jun 25, 2024
6b3b248
style: pep8 fixes
Rebits Jun 25, 2024
9ff2f5b
fix: change variable name
santipadilla Jun 25, 2024
4cb7ed4
fix: change variable name
santipadilla Jun 25, 2024
8bf979f
Merge pull request #5518 from wazuh/feat/6571-indexer-stats-plot
juliamagan Jun 25, 2024
1f6f8f5
fix: change variable name
santipadilla Jun 25, 2024
592ec66
fix: temporary agent removal
santipadilla Jun 26, 2024
0672d00
fix: reset packages version
santipadilla Jun 26, 2024
69affd3
docs: update changelog
santipadilla Jun 26, 2024
9c692d9
fix: change parameter
santipadilla Jun 26, 2024
b90480c
fix: change parameter name
santipadilla Jun 26, 2024
fdd09d0
Merge branch '4.8.1' into enhancement/5509-remove-hardcoded-branch-re…
santipadilla Jun 26, 2024
0860d2b
fix: Indexer class initialization
Rebits Jun 26, 2024
f391ccb
fix: reestablish all agents
santipadilla Jun 26, 2024
5f3fa0d
docs: include 5520 changelog
Rebits Jun 27, 2024
48e94be
Merge pull request #5520 from wazuh/fix/5519-indexer-class
juliamagan Jun 27, 2024
925d6fe
fix: remove unnecesary repository installation tasks
santipadilla Jun 27, 2024
98b5d31
fix: remove unnecesary repository installation
santipadilla Jun 27, 2024
fb60cce
fix: remove unnecessary repository installation tasks
santipadilla Jun 27, 2024
b87f54f
fix: remove unnecessary repository installation tasks
santipadilla Jun 27, 2024
04d324e
fix: change hardcode cert variable
santipadilla Jun 27, 2024
622d79d
fix: add commas
santipadilla Jun 27, 2024
a5a4cae
fix: restore source variables
santipadilla Jun 27, 2024
548f064
fix: temporal remove of agents
santipadilla Jun 27, 2024
2135afa
docs: update changelog
santipadilla Jun 28, 2024
331da44
fix: reestablish all test agents
santipadilla Jun 28, 2024
15d8c2c
Update CHANGELOG.md
santipadilla Jun 28, 2024
dffbd1b
feat: include flake8 linting
Rebits Jun 28, 2024
946c2ed
feat: update pycodestyle version
Rebits Jun 28, 2024
5de7379
style: fix yaml style error
Rebits Jun 28, 2024
c2064a5
fix: datavisualization sublclasses constructors
Rebits Jun 28, 2024
85d27ab
style: pep8 and yamllint
Rebits Jun 28, 2024
bb6c8ff
style: yamllint format
Rebits Jun 28, 2024
47af0d5
style: avoid yamllint
Rebits Jun 28, 2024
53a9063
temporal change to test linter
Rebits Jun 28, 2024
36a8c61
fix: binary initialization
Rebits Jun 28, 2024
29bd234
style: yaml lint
Rebits Jun 28, 2024
08db0f4
Merge pull request #5526 from wazuh/fix/5519-binary-plot
juliamagan Jul 1, 2024
286fa11
fix: remove agents for testing
santipadilla Jul 1, 2024
c20dfd7
fix: reestablish all agents
santipadilla Jul 1, 2024
80ad6d5
Merge pull request #5517 from wazuh/enhancement/5509-remove-hardcoded…
juliamagan Jul 1, 2024
5899aac
docs: move changelog entry to 4.8.1
juliamagan Jul 2, 2024
516bce2
Merge pull request #5542 from wazuh/enhancement/5541-update-changelog
juliamagan Jul 2, 2024
bb5e0dc
refactor: bump revision
juliamagan Jul 2, 2024
c482e30
Merge pull request #5543 from wazuh/enhancement/5541-bump-revision
juliamagan Jul 2, 2024
755d9db
Bump revision to 40815
rauldpm Jul 4, 2024
7f17350
Merge pull request #5550 from wazuh/enhancement/5548-bump-revision
rauldpm Jul 4, 2024
497c30b
Resolve conflicts
fcaffieri Jul 8, 2024
7295129
Remove changelog
fcaffieri Jul 8, 2024
3624d76
Fix review comments
fcaffieri Jul 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,10 @@
]
},
"alerts_info": {
"title": "Alerts and events info.",
"title": "Alerts and events info",
"columns": [
"Events processed", "Events received", "Written alerts", "Written firewall", "Written fts"
"Events processed", "Events received", "Written alerts", "Written firewall", "Written fts",
"Written archives", "Written stats"
]
}
}
63 changes: 54 additions & 9 deletions deps/wazuh_testing/wazuh_testing/scripts/data_visualizations.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,36 +3,81 @@
from os.path import exists
from tempfile import gettempdir

from wazuh_testing.tools.performance.visualization import DataVisualizer
from wazuh_testing.tools.performance.visualization import (
BinaryDatavisualizer,
ClusterStatisticsVisualizer,
DaemonStatisticsVisualizer,
IndexerAlerts,
IndexerVulnerabilities,
LogcollectorStatisticsVisualizer,
)

supported_targets = ['binary', 'analysis', 'remote', 'wazuhdb', 'logcollector',
'cluster', 'indexer-alerts',
'indexer-vulnerabilities']
daemon_supported_statistics = ['analysis', 'remote', 'wazuhdb']
strategy_plot_by_target = {
'binary': BinaryDatavisualizer,
'daemon-statistics': DaemonStatisticsVisualizer,
'cluster': ClusterStatisticsVisualizer,
'logcollector': LogcollectorStatisticsVisualizer,
'indexer-alerts': IndexerAlerts,
'indexer-vulnerabilities': IndexerVulnerabilities
}


def create_destination_directory(destination_directory):
if not exists(destination_directory):
makedirs(destination_directory)


def validate_arguments(options):
if options.visualization_target != 'binary' and options.unify:
raise ValueError('Unify option is not allowed for non binary data plotting')


def get_script_arguments():
parser = argparse.ArgumentParser(usage="%(prog)s [options]", description="Script to generate data visualizations",
parser = argparse.ArgumentParser(usage='%(prog)s [options]', description='Script to generate data visualizations',
formatter_class=argparse.RawTextHelpFormatter)
parser.add_argument('-s', '--sources', dest='csv_list', required=True, type=str, nargs='+', action='store',
help='Paths to the CSV files separated by whitespace.')
parser.add_argument('-t', '--target', dest='visualization_target', default='binary',
choices=['binary', 'analysis', 'remote', 'agent', 'logcollector', 'cluster', 'api', 'wazuhdb'],
choices=supported_targets,
help='Generate data visualizations for a specific target. Default binary.')
parser.add_argument('-d', '--destination', dest='destination', default=gettempdir(),
help=f'Directory to store the images. Default {gettempdir()}')
parser.add_argument('-n', '--name', dest='name', default=None,
help=f'Base name for the images. Default {None}.')
parser.add_argument('-c', '--columns', dest='columns', default=None,
help=f'Path to Json with Columns to Plot. Default {None}.')
parser.add_argument('-u', '--unify', dest='unify', action='store_true',
help='Unify data of the binary processes with their subprocesses to plot.')

return parser.parse_args()


def main():
options = get_script_arguments()
destination = options.destination
create_destination_directory(options.destination)

target = options.visualization_target
validate_arguments(options)

visualization_options = {
'dataframes_paths': options.csv_list,
'store_path': options.destination,
'base_name': options.name
}

strategy = target
if target in daemon_supported_statistics:
visualization_options['daemon'] = target
strategy = 'daemon-statistics'
elif target == 'binary':
visualization_options['unify_child_daemon_metrics'] = options.unify

dv = strategy_plot_by_target[strategy](**visualization_options)

if not exists(destination):
makedirs(destination)
dv = DataVisualizer(dataframes=options.csv_list, target=options.visualization_target,
compare=False, store_path=options.destination, base_name=options.name,
columns_path=options.columns)
dv.plot()


Expand Down
Loading