Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tests: 1175 ci refactor e2e #1191

Merged
merged 13 commits into from
Aug 11, 2023
Merged

Tests: 1175 ci refactor e2e #1191

Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
dad33d5
Refactor Tests
bermuell Aug 8, 2023
fb8ab73
Remove option happy-path-only, short-happy-path
bermuell Aug 9, 2023
e497694
Move E2E and Cometmock to dedicated workflow
bermuell Aug 9, 2023
be539b2
Fix Makefile
bermuell Aug 9, 2023
f0c2721
fix linter issues
bermuell Aug 9, 2023
657e674
Split test in e2e manual workflow
bermuell Aug 10, 2023
7b84042
Enforce go-relayer on cometmock usage
bermuell Aug 10, 2023
8c7a30c
Revert "Move E2E and Cometmock to dedicated workflow"
bermuell Aug 10, 2023
704e35d
Update description
bermuell Aug 10, 2023
7f2ac85
Introduce action for e2e setup
bermuell Aug 10, 2023
b57eb95
Revert e2e action
bermuell Aug 10, 2023
128ca6b
added error handling for flag.set
bermuell Aug 10, 2023
f0a700e
Update docstring for short-happy-path
p-offtermatt Aug 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 84 additions & 7 deletions .github/workflows/manual-e2e.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,99 @@ on:
workflow_dispatch:

jobs:
manual-integration-main:
happy-path-test:
runs-on: ubuntu-latest
timeout-minutes: 60
timeout-minutes: 20
steps:
- uses: actions/setup-go@v4
with:
go-version: "1.20"
- uses: actions/checkout@v3

- name: Checkout LFS objects
run: git lfs checkout

- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: "1.20" # The Go version to download (if necessary) and use.

- name: E2E tests
run: make test-e2e
- name: E2E happy-path test
run: go run ./tests/e2e/... --tc happy-path
changeover-test:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/setup-go@v4
with:
go-version: "1.20"
- uses: actions/checkout@v3
- name: Checkout LFS objects
run: git lfs checkout
- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: "1.20" # The Go version to download (if necessary) and use.
- name: E2E changeover test
run: go run ./tests/e2e/... --tc changeover
bermuell marked this conversation as resolved.
Show resolved Hide resolved
democracy-reward-test:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/setup-go@v4
with:
go-version: "1.20"
- uses: actions/checkout@v3
- name: Checkout LFS objects
run: git lfs checkout
- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: "1.20" # The Go version to download (if necessary) and use.
- name: E2E democracy-reward tests
run: go run ./tests/e2e/... --tc democracy-reward
democracy-test:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/setup-go@v4
with:
go-version: "1.20"
- uses: actions/checkout@v3
- name: Checkout LFS objects
run: git lfs checkout
- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: "1.20" # The Go version to download (if necessary) and use.
- name: E2E democracy tests
run: go run ./tests/e2e/... --tc democracy
slash-throttle-test:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/setup-go@v4
with:
go-version: "1.20"
- uses: actions/checkout@v3
- name: Checkout LFS objects
run: git lfs checkout
- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: "1.20" # The Go version to download (if necessary) and use.
- name: E2E slash-throttle tests
run: go run ./tests/e2e/... --tc slash-throttle
multiconsumer-test:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/setup-go@v4
with:
go-version: "1.20"
- uses: actions/checkout@v3
- name: Checkout LFS objects
run: git lfs checkout
- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: "1.20" # The Go version to download (if necessary) and use.
- name: E2E multi-consumer tests
run: go run ./tests/e2e/... --tc multiconsumer
104 changes: 97 additions & 7 deletions .github/workflows/nightly-e2e.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,21 +18,111 @@ on:
- cron: "0 3 * * *"

jobs:
nightly-test:
happy-path-test:
runs-on: ubuntu-latest
timeout-minutes: 60
timeout-minutes: 20
steps:
- uses: actions/setup-go@v4
with:
go-version: "1.20"

- uses: actions/checkout@v3

- name: E2E tests
run: make test-e2e
- name: Checkout LFS objects
run: git lfs checkout
- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: "1.20" # The Go version to download (if necessary) and use.
- name: E2E happy-path test
run: go run ./tests/e2e/... --tc happy-path
changeover-test:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/setup-go@v4
with:
go-version: "1.20"
- uses: actions/checkout@v3
- name: Checkout LFS objects
run: git lfs checkout
- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: "1.20" # The Go version to download (if necessary) and use.
- name: E2E changeover test
run: go run ./tests/e2e/... --tc changeover
democracy-reward-test:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/setup-go@v4
with:
go-version: "1.20"
- uses: actions/checkout@v3
- name: Checkout LFS objects
run: git lfs checkout
- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: "1.20" # The Go version to download (if necessary) and use.
- name: E2E democracy-reward tests
run: go run ./tests/e2e/... --tc democracy-reward
democracy-test:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/setup-go@v4
with:
go-version: "1.20"
- uses: actions/checkout@v3
- name: Checkout LFS objects
run: git lfs checkout
- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: "1.20" # The Go version to download (if necessary) and use.
- name: E2E democracy tests
run: go run ./tests/e2e/... --tc democracy
slash-throttle-test:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/setup-go@v4
with:
go-version: "1.20"
- uses: actions/checkout@v3
- name: Checkout LFS objects
run: git lfs checkout
- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: "1.20" # The Go version to download (if necessary) and use.
- name: E2E slash-throttle tests
run: go run ./tests/e2e/... --tc slash-throttle
multiconsumer-test:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/setup-go@v4
with:
go-version: "1.20"
- uses: actions/checkout@v3
- name: Checkout LFS objects
run: git lfs checkout
- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: "1.20" # The Go version to download (if necessary) and use.
- name: E2E multi-consumer tests
run: go run ./tests/e2e/... --tc multiconsumer

nightly-test-fail:
needs: nightly-test
needs:
- happy-path-test
- changeover-test
- democracy-reward-test
- democracy-test
- slash-throttle-test
- multiconsumer-test
if: ${{ failure() }}
runs-on: ubuntu-latest
steps:
Expand Down
12 changes: 6 additions & 6 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ install: go.sum
go install $(BUILD_FLAGS) ./cmd/interchain-security-sd

# run all tests: unit, integration, diff, and E2E
test:
go test ./... && go run ./tests/e2e/...
test:
go test ./... && go run ./tests/e2e/...

# run all unit tests
test-unit:
Expand All @@ -31,12 +31,12 @@ test-diff:

# run only happy path E2E tests
test-e2e-short:
go run ./tests/e2e/... --happy-path-only
go run ./tests/e2e/... --tc happy-path

# run only happy path E2E tests with cometmock
# this set of traces does not test equivocation but it does check downtime
test-e2e-short-cometmock:
go run ./tests/e2e/... --cometmock-happy-path --use-cometmock --use-gorelayer
go run ./tests/e2e/... --tc happy-path-short --use-cometmock --use-gorelayer

# run full E2E tests in sequence (including multiconsumer)
test-e2e-multi-consumer:
Expand All @@ -52,7 +52,7 @@ test-gaia-e2e:

# run only happy path E2E tests using latest tagged gaia
test-gaia-e2e-short:
go run ./tests/e2e/... --happy-path-only --use-gaia
go run ./tests/e2e/... --tc happy-path --use-gaia

# run full E2E tests in parallel (including multiconsumer) using latest tagged gaia
test-gaia-e2e-parallel:
Expand All @@ -66,7 +66,7 @@ test-gaia-e2e-tagged:
# run only happy path E2E tests using latest tagged gaia
# usage: GAIA_TAG=v9.0.0 make test-gaia-e2e-short-tagged
test-gaia-e2e-short-tagged:
go run ./tests/e2e/... --happy-path-only --use-gaia --gaia-tag $(GAIA_TAG)
go run ./tests/e2e/... --tc happy-path --use-gaia --gaia-tag $(GAIA_TAG)

# run full E2E tests in parallel (including multiconsumer) using specific tagged version of gaia
# usage: GAIA_TAG=v9.0.0 make test-gaia-e2e-parallel-tagged
Expand Down
Loading
Loading