Skip to content

Commit

Permalink
feat(ci): dynamic test scheduler / balancer
Browse files Browse the repository at this point in the history
This reverts commit e804fd4 effectively
reapplying 543004c.

Original commit message:

This commit adds an automatic scheduler for running busted tests.  It
replaces the static, shell script based scheduler by a mechanism that
distributes the load onto a number of runners.  Each runner gets to
work on a portion of the tests that need to be run.  The scheduler
uses historic run time information to distribute the work evenly
across runners, with the goal of making them all run for the same
amount of time.  With the 7 runners configured in the PR, the overall
time it takes to run tests is reduced from around 30 minutes to around
11 minutes.

Previously, the scheduling for tests was defined by what the
run_tests.sh shell script did.  This has now changed so that the new
JSON file `test_suites.json` is instead used to define the tests that
need to run.  Like before, each of the test suites can have its own
set of environment variables and test exclusions.

The test runner has been rewritten in Javascript in order to make it
easier to interface with the declarative configuration file and to
facilitate reporting and interfacing with busted.  It resides in the
https://github.com/Kong/gateway-test-scheduler repository and
provides its functionality through custom GitHub Actions.

A couple of tests had to be changed to isolate them from other tests
better.  As the tests are no longer run in identical order every time,
it has become more important that each test performs any required
cleanup before it runs.
  • Loading branch information
samugi committed Feb 14, 2024
1 parent b0940b2 commit da61296
Show file tree
Hide file tree
Showing 6 changed files with 216 additions and 352 deletions.
165 changes: 0 additions & 165 deletions .ci/run_tests.sh

This file was deleted.

34 changes: 34 additions & 0 deletions .ci/test_suites.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
[
{
"name": "unit",
"exclude_tags": "flaky,ipv6",
"specs": ["spec/01-unit/"]
},
{
"name": "integration",
"exclude_tags": "flaky,ipv6,off",
"environment": {
"KONG_TEST_DATABASE": "postgres"
},
"specs": ["spec/02-integration/"]
},
{
"name": "dbless",
"exclude_tags": "flaky,ipv6,postgres,db",
"specs": [
"spec/02-integration/02-cmd/",
"spec/02-integration/05-proxy/",
"spec/02-integration/04-admin_api/02-kong_routes_spec.lua",
"spec/02-integration/04-admin_api/15-off_spec.lua",
"spec/02-integration/08-status_api/01-core_routes_spec.lua",
"spec/02-integration/08-status_api/03-readiness_endpoint_spec.lua",
"spec/02-integration/11-dbless/",
"spec/02-integration/20-wasm/"
]
},
{
"name": "plugins",
"exclude_tags": "flaky,ipv6",
"specs": ["spec/03-plugins/"]
}
]
Loading

0 comments on commit da61296

Please sign in to comment.