Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor and simplify e2e testing startup #2434

Closed
Tracked by #2406
MSalopek opened this issue Apr 24, 2023 · 0 comments
Closed
Tracked by #2406

Refactor and simplify e2e testing startup #2434

MSalopek opened this issue Apr 24, 2023 · 0 comments
Assignees
Labels
S: Productivity Productivity: Developer tooling, infrastructure improvements enabling future growth scope: testing Code review, testing, making sure the code is following the specification. type: feature-request New feature or request improvement type: tech-debt Slows down development in the long run

Comments

@MSalopek
Copy link
Contributor

MSalopek commented Apr 24, 2023

At present, the e2e testing approach in gaia and ICS is essentially the same, but different initialization procedure, test trace execution and result output is used. The listed inconsistencies are preventing standardization of approach to e2e testing and requires custom CI integrations to be setup.

With adoptiong of ICS module, gaia needs to have the ability to test ICS protocol functionality with ICS consumer chains (either mock chains or actual consumer chain binaries). To mitigate risks of consumer misbehaviour, gaia should be able to include consumer chains in its e2e tests.

To facilitate this the ICS e2e testing framework needs to be adapted to allow easier provider and consumer chain initialization, test trace execution and test result reporting.

It would be beneficial to create a way to declaratively configure the testing framework with regards to test path execution, the number of chains and which chains to run.

Inspiration can be taken from existing test frameworks and the main focus should be allowing the tests to run in an isolated environment (such as a docker container). Additionally, the tests could be made to run solely on the host machine but this feature is optional (for now). The ability to run e2e tests in both docker environment and on host machine is beneficial but difficult to achieve at first since the two approaches have their own specific quirks which have to be addressed separately.

Frameworks to study and learn from:

Heavily linked to cosmos/interchain-security#877

@mpoke mpoke added the more-info-needed Further information is requested label Apr 24, 2023
@MSalopek MSalopek self-assigned this Apr 24, 2023
@MSalopek MSalopek added type: feature-request New feature or request improvement scope: testing Code review, testing, making sure the code is following the specification. type: tech-debt Slows down development in the long run and removed more-info-needed Further information is requested labels Apr 24, 2023
@github-project-automation github-project-automation bot moved this to 🩹 Triage in Cosmos Hub Apr 24, 2023
@MSalopek MSalopek moved this from 🩹 Triage to 🏗 In progress in Cosmos Hub Apr 24, 2023
@MSalopek MSalopek moved this from 🏗 In progress to 📥 Todo in Cosmos Hub Apr 26, 2023
@mpoke mpoke added the S: Productivity Productivity: Developer tooling, infrastructure improvements enabling future growth label Sep 15, 2023
@mpoke mpoke closed this as completed Apr 4, 2024
@github-project-automation github-project-automation bot moved this from 📥 F2: Todo to 👍 F4: Assessment in Cosmos Hub Apr 4, 2024
@mpoke mpoke moved this from 👍 F4: Assessment to ✅ Done in Cosmos Hub Apr 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S: Productivity Productivity: Developer tooling, infrastructure improvements enabling future growth scope: testing Code review, testing, making sure the code is following the specification. type: feature-request New feature or request improvement type: tech-debt Slows down development in the long run
Projects
Status: ✅ Done
Development

No branches or pull requests

2 participants