Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test suite: initial tests #518

Closed
37 of 43 tasks
mmguero opened this issue Dec 2, 2024 · 0 comments
Closed
37 of 43 tasks

test suite: initial tests #518

mmguero opened this issue Dec 2, 2024 · 0 comments
Assignees
Labels
testing Related to automated system testing of Malcolm
Milestone

Comments

@mmguero
Copy link
Collaborator

mmguero commented Dec 2, 2024

I'm going to collect a list of initial tests that need to be created for the automated testing feature (#486):

  • connectivity
    • test_malcolm_exists - Asserts that the Malcolm VM is running
    • test_malcolm_response - Asserts that the Malcolm instance is up and API is responsive
    • test_malcolm_db_health - Asserts that the Malcolm opensearch API is responsive and returns a "yellow" or "green" health status
  • components/interfaces/features
    • dashboards/kibana
      • ensure that OpenSearch Dashboards page is served
      • if possible, try to use reporting (API?) to generate PDF for a dashboard
    • Arkime
    • local account management
      • ensure that local account management page is served
      • if possible, actually use it, but IDK if that's possible with pytest/requests
    • upload
    • cyberchef
      • ensure that cyberchef UI is served
    • netbox
      • ensure that netbox UI is served
      • after insertion with autodiscovery, check that various network entities have been populated
      • test that enrichment has been performed on zeek logs and suricata alerts
      • test a netbox restore operation (not supported by Malcolm-Test framework yet)
      • test a netbox backup operation (not supported by Malcolm-Test framework yet)
    • file extraction
      • ensure that file extraction download UI is served
      • ensure that files are extracted and quarantined and/or preserved
      • ensure extracted file can be downloaded and unzipped/decrypted
    • severity
      • checks for event severity scoring (all categories)
  • Malcolm APIs
  • protocols/enrichment
    • test_ot_protocols - perform field aggregation to check for all expected OT protocols
    • test_common_protocols - perform field aggregation to check for all expected common protocols
    • test_detection_packages - perform field aggregation to check for results from CVE/etc. detections
    • check for uploaded windows event (evtx) logs
    • spot check data for all enrichment sources (freq, geoip, etc.)
    • check that Zeek intel triggers (not supported by Malcolm-Test framework yet)
    • check for logs written by all zeek plugins

EDIT: I've stricken some that aren't going to be possible in this initial batch, but might be good for later additions.

@mmguero mmguero added the testing Related to automated system testing of Malcolm label Dec 2, 2024
@mmguero mmguero added this to the v24.12.0 milestone Dec 2, 2024
@mmguero mmguero self-assigned this Dec 2, 2024
@mmguero mmguero added this to Malcolm Dec 2, 2024
@mmguero mmguero moved this to In Progress in Malcolm Dec 2, 2024
mmguero added a commit to mmguero-dev/Malcolm-Test that referenced this issue Dec 2, 2024
mmguero added a commit to mmguero-dev/Malcolm-Test that referenced this issue Dec 2, 2024
@mmguero mmguero closed this as completed Dec 18, 2024
@github-project-automation github-project-automation bot moved this from In Progress to Done in Malcolm Dec 18, 2024
@mmguero mmguero moved this from Done to Released in Malcolm Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
testing Related to automated system testing of Malcolm
Projects
Status: Released
Development

No branches or pull requests

1 participant