Refactor and simplify e2e testing startup #2434
Labels
S: Productivity
Productivity: Developer tooling, infrastructure improvements enabling future growth
scope: testing
Code review, testing, making sure the code is following the specification.
type: feature-request
New feature or request improvement
type: tech-debt
Slows down development in the long run
At present, the e2e testing approach in gaia and ICS is essentially the same, but different initialization procedure, test trace execution and result output is used. The listed inconsistencies are preventing standardization of approach to e2e testing and requires custom CI integrations to be setup.
With adoptiong of ICS module, gaia needs to have the ability to test ICS protocol functionality with ICS consumer chains (either mock chains or actual consumer chain binaries). To mitigate risks of consumer misbehaviour, gaia should be able to include consumer chains in its e2e tests.
To facilitate this the ICS e2e testing framework needs to be adapted to allow easier provider and consumer chain initialization, test trace execution and test result reporting.
It would be beneficial to create a way to declaratively configure the testing framework with regards to test path execution, the number of chains and which chains to run.
Inspiration can be taken from existing test frameworks and the main focus should be allowing the tests to run in an isolated environment (such as a docker container). Additionally, the tests could be made to run solely on the host machine but this feature is optional (for now). The ability to run e2e tests in both docker environment and on host machine is beneficial but difficult to achieve at first since the two approaches have their own specific quirks which have to be addressed separately.
Frameworks to study and learn from:
Heavily linked to cosmos/interchain-security#877
The text was updated successfully, but these errors were encountered: