Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[antithesis] Enable reuse of banff e2e test for antithesis testing #3554

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

marun
Copy link
Contributor

@marun marun commented Nov 19, 2024

Why this should be merged

Having to duplicate existing test coverage to benefit from execution with antithesis is not ideal. Better to be able to trivially refactor compatible e2e coverage for reuse. This PR refactors the banff e2e test for reuse with antithesis to demonstrate that this is possible.

How this works

  • adds new APITestFunction type that a compatible e2e test can implement to allow for execution with both ginkgo and antithesis
  • adds ExecuteAPITest helper to simplify execution of an APITestFunction with ginkgo
  • Refactors the banff e2e test into an APITestFunction executed by ExecuteAPITest
  • Refactors the avalanchego antithesis test setup to support execution of the banff test
    • Moves test dispatch into a new function so that panics can be recovered via a deferred function
      • This supports use of require by e2e tests without risking an unhandled panic exiting the test process

How this was tested

CI

TODO

 	Error Trace:	tests/e2e/banff/suites.go:95
	            				tests/simple_test_context.go:167
	            				tests/e2e/banff/suites.go:89
	Error:      	Received unexpected error:
	            	failed to decode client response: couldn't issue tx: failed verification: failed execution: standard tx Y6j5fJ84dxQYxt3R8CVh8YK87WXrzJjdPtrC9fWXYaa4dJaCm failed execution: failed to get shared memory: not found
Error Trace:	tests/e2e/banff/suites.go:95
               				tests/simple_test_context.go:167
               				tests/e2e/banff/suites.go:89
   Error:      	Received unexpected error:
               	failed to decode client response: couldn't issue tx: failed verification: failed execution: standard tx 2FXNbGVxrQ9qHUjwhVWzjrepAA2J71xkizC4gCQnJuZtQLvTYB failed execution: failed to get UTXO 11111111111111111111111111111111LpoYY:0: not found
  • Ensure a transaction is only attempted if sufficient funds exist to pay for it
Error Trace:	tests/e2e/banff/suites.go:95
               				tests/simple_test_context.go:167
               				tests/e2e/banff/suites.go:89
   Error:      	Received unexpected error:
               	couldn't generate tx inputs/outputs: insufficient funds: provided UTXOs needed 81008 more nAVAX ("2fombhL7aGPwj3KH4bfrmJwW6PVnMobf9Y2fn9GwxiAAJyFDbe")
  • Ensure failed connections caused by fault injection are retried or abort the test
Error Trace:	tests/e2e/banff/suites.go:95
	            				tests/simple_test_context.go:167
	            				tests/e2e/banff/suites.go:89
	Error:      	Received unexpected error:
	            	context deadline exceeded
	Error Trace:	tests/e2e/banff/suites.go:95
	            				tests/simple_test_context.go:167
	            				tests/e2e/banff/suites.go:89
	Error:      	Received unexpected error:
	            	failed to issue request: Post "http://10.0.20.4:9650/ext/P": EOF
	Error Trace:	tests/e2e/banff/suites.go:123
	            				tests/simple_test_context.go:167
	            				tests/e2e/banff/suites.go:117
	Error:      	Received unexpected error:
	            	failed to issue request: Post "http://10.0.20.3:9650/ext/bc/X": dial tcp 10.0.20.3:9650: connect: connection refused
	Error Trace:	tests/e2e/banff/suites.go:86
	            				tests/simple_test_context.go:167
	            				tests/e2e/banff/suites.go:70
	Error:      	Received unexpected error:
	            	failed to issue request: Post "http://10.0.20.3:9650/ext/bc/X": dial tcp 10.0.20.3:9650: i/o timeout
	Error Trace:	tests/e2e/banff/suites.go:66
	            				tests/simple_test_context.go:167
	            				tests/e2e/banff/suites.go:51
	Error:      	Received unexpected error:
	            	received status code: 404
	Error Trace:	tests/e2e/banff/suites.go:66
	            				tests/simple_test_context.go:167
	            				tests/e2e/banff/suites.go:51
	Error:      	Received unexpected error:
	            	failed to issue request: Post "http://10.0.20.4:9650/ext/bc/X": read tcp 10.0.20.129:60442->10.0.20.4:9650: read: connection reset by peer

Need to be documented in RELEASES.md?

N/A

@marun marun marked this pull request as draft November 19, 2024 15:24
@marun marun changed the title bE2e reuse [antithesis] Enable reuse of banff e2e test for antithesis testing Nov 19, 2024
@marun marun added the testing This primarily focuses on testing label Nov 19, 2024
@marun marun self-assigned this Nov 19, 2024
@marun marun marked this pull request as ready for review November 19, 2024 18:14
@marun marun marked this pull request as draft November 20, 2024 21:59
@marun
Copy link
Contributor Author

marun commented Nov 20, 2024

Switched to draft in light of wanting to rebase on #3557

@marun marun changed the base branch from master to testing-switch-to-zap November 21, 2024 13:52
@marun marun force-pushed the testing-switch-to-zap branch from cf8a4d4 to 292016a Compare November 21, 2024 19:27
Base automatically changed from testing-switch-to-zap to master November 21, 2024 22:39
@marun marun marked this pull request as ready for review November 22, 2024 16:12
@marun marun marked this pull request as draft December 5, 2024 13:44
@marun
Copy link
Contributor Author

marun commented Dec 5, 2024

Moving to draft and adding a bunch of TODOs prompted by a manually-triggered antithesis run. The error handling of the e2e test is not sufficiently robust, so there are a lot of unhelpful errors that need to be addressed before this PR will be mergeable.

@marun marun force-pushed the e2e-reuse branch 2 times, most recently from 2dbfdce to 6e0b41c Compare December 11, 2024 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
testing This primarily focuses on testing
Projects
Status: In Review 👀
Development

Successfully merging this pull request may close these issues.

1 participant