Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SystemResources tested within the Ref integration test suite #1561

Open
ThibFrgsGmz opened this issue Jul 12, 2022 · 2 comments
Open

SystemResources tested within the Ref integration test suite #1561

ThibFrgsGmz opened this issue Jul 12, 2022 · 2 comments
Labels
Easy First Issue An issue that should be straight forward to implement, and easily tested via CI. enhancement Medium Priority

Comments

@ThibFrgsGmz
Copy link
Contributor

F´ Version 3.1.x
Affected Component Ref integration test suite

Feature Description

It would be great to have an integration test that verifies the system resources in the Ref integration test suite. These tests would verify:

  • The framework/project version as described in git describe --tags --always for example.
  • Memory usage (MEMORY_TOTAL, MEMORY_USED, NON_VOLATILE_TOTAL, NON_VOLATILE_FREE) greater than a certain value, 1KB for example.
  • A number of CPUs greater than or equal to 1
  • A percentage of CPUs greater than 1%.

Rationale

From a system perspective, it is interesting to:

  • monitor the memory consumption and CPU load according to the system's operating modes.
  • verify that it is the correct version of the system under test
  • check that the number of CPUs used is correct and that there are no underutilized or overutilized CPUs.

It also shows the user how to create predicates/assertions on specific data structure fields received in telemetry messages.

@LeStarch LeStarch added Easy First Issue An issue that should be straight forward to implement, and easily tested via CI. Medium Priority labels Sep 26, 2022
@LeStarch
Copy link
Collaborator

Agreed. We need to test this component.

@matt392code
Copy link
Contributor

system-resources-test.py.txt
Proposed comprehensive test suite for the system resources requirements.
Here's a breakdown of the key components:

  1. Framework Version Test:
  • Uses git describe --tags --always to get the current version
  • Verifies it matches the version reported by SystemResources component
  1. Memory Metrics Tests:
  • Validates total memory exceeds 1KB threshold
  • Checks that used memory is being tracked and reported
  • Verifies both total and free non-volatile memory reporting
  1. CPU Utilization Tests:
  • Confirms CPU count is at least 1
  • Verifies CPU usage exceeds 1% threshold
  • Includes historical tracking of CPU metrics
  1. Additional Features:
  • Proper test setup/teardown for clean state between tests
  • Resource history tracking to verify metrics over time
  • Integration with F' testing framework and predicates
  • Comprehensive error messages for failed assertions

To use this test suite:

  1. Place it in your F' reference implementation's test directory
  2. Run it using:
fprime-gds --test-mode
python SystemResourcesTest.py

The tests use F' standard testing frameworks and predicates to ensure consistent behavior with other integration tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Easy First Issue An issue that should be straight forward to implement, and easily tested via CI. enhancement Medium Priority
Projects
None yet
Development

No branches or pull requests

3 participants