Refactor and simplify e2e test traces loading and execution #2435
Labels
admin: epic
An EPIC -- meta issue used to track a body of work
other: decayed
Stale issues that need follow up from commentators. Were closed for inactivity
S: Productivity
Productivity: Developer tooling, infrastructure improvements enabling future growth
scope: testing
Code review, testing, making sure the code is following the specification.
type: feature-request
New feature or request improvement
type: tech-debt
Slows down development in the long run
At present the gaia e2e testing uses hand-written tests to ensure normal operations are passing (happy path tests) and that no sources of non-determinisim exist.
The ICS e2e testing framework also uses hand-written test traces and executes them using the test runner.
All tests are performed in an isolated environment (docker container).
It would be beneficial to adopt a standardized trace format that comprises the following features:
Adopting a standardized format would enable fast and reproducible test trace generation using MBT tooling (as opposed to writing tests traces manually), would help standardize the test output and enable meaningful and comprehensive test results.
In the first steps towards establishing e2e test trace generation it is important to maintain parity with the existing test frameworks. Existing test frameworks should be used as a benchmark for comparing all future work. The selected format should allow e2e testing framework refactoring with regards to ease of use and configuration.
Existing frameworks:
Existing trace formats:
Tasks
The text was updated successfully, but these errors were encountered: