tests: separate e2e tests into a standalone module #1273
Conversation
MSalopek
added
the
scope: testing
github-actions
bot
added
C:Testing
| @@ -28,6 +28,8 @@ brew install jq | |||
|
|
|||
| **Installing and running binaries** | |||
|
|
|||
| This repository consists of multiple go modules. In order to have a smooth development experience, please add a `go.work` file. It is enough to copy and rename the `go.work.example` file. | |||
There was a problem hiding this comment.
The go.work file is also in this PR, this makes me think it should not. Should we remove the go.work file itself from the PR?
There was a problem hiding this comment.
Ideally, the go.work file should not be commited.
It is commited for now because the testing pipelines fail if you don't commit a go work file. The go tool sees that there are 2 modules but does not know how to build and execute normal tests without instructions from go.work.
This can also be fixed by some make magic, e.g.:
# create a go.work file on the fly by renaming go.work.example
mv ./go.work.example ./go.work
go test (...)I opted to not do that now since it would be more confusing than just commiting the file directly.
Notice how none of the make commands are actually changed, they just point to a different path (./e2e instead of ./testing/e2e).
There was a problem hiding this comment.
Ah, makes sense. Then maybe let's change that line in the README to account for the fact that the go.work file is in the repo already and keep it like this for now?
Makefile
Outdated
|
|
||
| # run only happy path E2E tests using latest tagged gaia | ||
| test-gaia-e2e-short: | ||
| go run ./tests/e2e/... --tc happy-path --use-gaia | ||
| go run ./e2e/... --happy-path-only --use-gaia |
There was a problem hiding this comment.
| go run ./e2e/... --happy-path-only --use-gaia | |
| go run ./e2e/... --tc happy-path --use-gaia |
Makefile
Outdated
|
|
||
| # run only happy path E2E tests using latest tagged gaia | ||
| # usage: GAIA_TAG=v9.0.0 make test-gaia-e2e-short-tagged | ||
| test-gaia-e2e-short-tagged: | ||
| go run ./tests/e2e/... --tc happy-path --use-gaia --gaia-tag $(GAIA_TAG) | ||
| go run ./e2e/... --happy-path-only --use-gaia --gaia-tag $(GAIA_TAG) |
There was a problem hiding this comment.
same here, what is the reason to go back to the 'old' flag and not using --tc
if I'm not mistaken that's not even supported anymore. did you test that?
There was a problem hiding this comment.
That test has not been working for a while. Gaia is on v45, the ICS main is on v47.
Since e2e was coupled to the ICS version, some of the commands to execute against a gaia node are not valid and cannot be executed. Notice that most of that was intended for gaia v9.
| @@ -0,0 +1,6 @@ | |||
| go 1.20 | |||
There was a problem hiding this comment.
Aren't these in the gitignore?
There was a problem hiding this comment.
Nvm addressed in other comment
|
Drafting until v50 is in to mitigate some risks |
MSalopek
added
the
other: decayed
|
Closing as the PR has decayed and takes a lot of work to get it working again. |
Description
Closes: #1132
e2e tests were moved outside
/testsdirectory and made independent of the rest of the repository.The
e2etests can still be used normally, no workflows were broken and there's no impact on theMakefilecommands.Some small refactors were made along the way.
Author Checklist
All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.
I have...
Reviewers Checklist
All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.
I have...