Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration testing for round restart and multiple rounds #4

Merged
merged 12 commits into from
Jul 6, 2021
Merged

Integration testing for round restart and multiple rounds #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
5af114c
work in progress support for multiple rounds
kellpossible Jun 13, 2021
07d0f16
Merge branch 'main' into round-restart
kellpossible Jun 19, 2021
1639543
implemented multiple rounds
kellpossible Jun 19, 2021
77f5431
updated multi test specifications with latest changes to ron format
kellpossible Jun 19, 2021
59d2db2
add backtraces to processes and update some of the test specifications
kellpossible Jun 27, 2021
504a28d
removed the command line test specification in favour of the ron spec…
kellpossible Jun 27, 2021
fa411a8
updated readme
kellpossible Jun 27, 2021
57caa74
work in progress changes for round-restart allowing contributors to s…
kellpossible Jun 27, 2021
8ccb5a6
updated test specifications
kellpossible Jul 2, 2021
dc37a4e
refactor for waiter in preparation for contributor starts
kellpossible Jul 2, 2021
aa04d68
implemented starting a contributor after round has already started, a…
kellpossible Jul 2, 2021
281847f
Merge branch 'main' into round-restart
kellpossible Jul 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 3 additions & 23 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,33 +18,13 @@ rustup install stable

## Building/Running

### Single

You can run a single simple integration test directly from the command line:

```bash
cargo run
```

Available command line options can be found:

```bash
cargo run -- --help
```

### Multi

You can run the scripted development integration tests with the following `multi` sub-command:
You can run a integration tests with the following command where `test.ron` is the test specification:

```bash
cargo run -- multi dev-tests.ron
cargo run -- test.ron
```

See the other `*-tests.ron` files for tests for each of the setups `development`, `inner`, `outer`, and `universal` (not yet implemented).

## Configuration Format

You can see the `*-tests.ron` files in the repository root (e.g. [dev-tests.ron](./dev-tests.ron)) for an example of the configuration format. They use the [Rusty Object Notation (RON)](https://github.com/ron-rs/ron) format, there are editor extensions available. This format was chosen because it allows structured/nested data (like JSON) but also allows comments and looser formatting for handwritten files (like TOML).
See [example-config.ron](./example-config.ron) in the repository root for an example of the configuration format. They use the [Rusty Object Notation (RON)](https://github.com/ron-rs/ron) format, there are editor extensions available. This format was chosen because it allows structured/nested data (like JSON) but also allows comments and looser formatting for handwritten files (like TOML).

### Using Local Repositories

Expand Down
20 changes: 12 additions & 8 deletions dev-tests-fail.ron
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,18 +19,22 @@
tests: [
(
id: "8",
contributors: 3,
verifiers: 1,
replacement_contributors: 2,
replacement_contributors: 0,
environment: development,
round_timout: Some(18000),
timout: Some(18000),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it's "time"

skip: false,
contributor_drops: [
rounds: [
(
after_contributions: 10,
),
(
after_contributions: 11,
contributors: 3,
contributor_drops: [
(
after_contributions: 10,
),
(
after_contributions: 11,
),
],
),
],
),
Expand Down
201 changes: 133 additions & 68 deletions dev-tests.ron
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,196 +71,261 @@
(
/// Id for the individual test.
id: "1",
/// Number of contributor participants for the test.
contributors: 1,
/// Number of verifier participants for the test.
verifiers: 1,
/// Number of replacement contributors for the test.
replacement_contributors: 0,
/// What environment to use for the setup.
/// Possible options: `development`, `inner`, `outer`, `universal`.
environment: development,
/// Timout (in seconds) for running a ceremony round of the
/// integration test (not including setting up prerequisites). If
/// this time is exceeded for a given round, the test will fail.
round_timout: Some(18000),
/// Whether to skip running this test.
skip: false,
contributor_drops: [],
/// Configure the tests performed for each round of the ceremony.
rounds: [
(
/// Number of contributor participants for this round of the
/// ceremony.
contributors: 1,
),
],
),
(
id: "2",
contributors: 1,
verifiers: 1,
replacement_contributors: 1,
environment: development,
round_timout: Some(18000),
timout: Some(18000),
skip: false,
/// Configure expected contributor drops. A contributor is
/// assigned automatically to each specified config. The number of
/// configs should not exceed the number of contributors.
contributor_drops: [
rounds: [
(
/// A contributor is dropped (process killed) after having made
/// this number of contributions.
after_contributions: 10,
contributors: 1,
/// Configure expected contributor drops. A contributor is
/// assigned automatically to each specified config. The number of
/// configs should not exceed the number of contributors.
contributor_drops: [
(
/// A contributor is dropped (process killed) after having made
/// this number of contributions.
after_contributions: 10,
),
],
),
],
),
(
id: "3",
contributors: 2,
verifiers: 1,
replacement_contributors: 0,
environment: development,
round_timout: Some(18000),
timout: Some(18000),
skip: false,
contributor_drops: [],
rounds: [
(
contributors: 2,
),
],
),
(
id: "4",
contributors: 2,
verifiers: 1,
replacement_contributors: 1,
environment: development,
round_timout: Some(18000),
timout: Some(18000),
skip: false,
contributor_drops: [
rounds: [
(
after_contributions: 10,
contributors: 2,
contributor_drops: [
(
after_contributions: 10,
),
],
),
],
),
(
id: "5",
contributors: 2,
verifiers: 1,
replacement_contributors: 2,
environment: development,
round_timout: Some(18000),
timout: Some(18000),
skip: false,
contributor_drops: [
rounds: [
(
after_contributions: 10,
),
(
after_contributions: 15,
contributors: 2,
contributor_drops: [
(
after_contributions: 10,
),
(
after_contributions: 15,
),
],
),
],
),
(
id: "6",
contributors: 3,
verifiers: 1,
replacement_contributors: 0,
environment: development,
round_timout: Some(18000),
timout: Some(18000),
skip: false,
contributor_drops: [],
rounds: [
(
contributors: 3,
),
],
),
(
id: "7",
contributors: 3,
verifiers: 1,
replacement_contributors: 1,
environment: development,
round_timout: Some(18000),
timout: Some(18000),
skip: false,
contributor_drops: [
rounds: [
(
after_contributions: 10,
contributors: 3,
contributor_drops: [
(
after_contributions: 10,
),
],
),
],
),
(
id: "9",
contributors: 3,
verifiers: 1,
replacement_contributors: 3,
environment: development,
round_timout: Some(18000),
timout: Some(18000),
skip: false,
contributor_drops: [
(
after_contributions: 10,
),
(
after_contributions: 11,
),
rounds: [
(
after_contributions: 15,
contributors: 3,
contributor_drops: [
(
after_contributions: 10,
),
(
after_contributions: 11,
),
(
after_contributions: 15,
),
],
),
],
),
(
id: "10",
contributors: 4,
verifiers: 1,
replacement_contributors: 0,
environment: development,
round_timout: Some(18000),
timout: Some(18000),
skip: false,
contributor_drops: [],
rounds: [
(
contributors: 4,
),
],
),
(
id: "15",
contributors: 5,
verifiers: 1,
replacement_contributors: 0,
environment: development,
round_timout: Some(18000),
timout: Some(18000),
skip: false,
contributor_drops: [],
rounds: [
(
contributors: 5,
),
],
),
(
id: "21",
contributors: 1,
verifiers: 2,
replacement_contributors: 0,
environment: development,
round_timout: Some(18000),
timout: Some(18000),
skip: false,
contributor_drops: [],
rounds: [
(
contributors: 1,
),
],
),
(
id: "23",
contributors: 2,
verifiers: 2,
replacement_contributors: 0,
environment: development,
round_timout: Some(18000),
timout: Some(18000),
skip: false,
contributor_drops: [],
rounds: [
(
contributors: 2,
),
],
),
(
id: "26",
contributors: 3,
verifiers: 2,
replacement_contributors: 0,
environment: development,
round_timout: Some(18000),
timout: Some(18000),
skip: false,
contributor_drops: [],
rounds: [
(
contributors: 3,
),
],
),
(
id: "30",
contributors: 4,
verifiers: 2,
replacement_contributors: 0,
environment: development,
round_timout: Some(18000),
timout: Some(18000),
skip: false,
contributor_drops: [],
rounds: [
(
contributors: 4,
),
],
),
(
id: "35",
contributors: 5,
verifiers: 2,
replacement_contributors: 0,
environment: development,
round_timout: Some(18000),
timout: Some(18000),
skip: false,
contributor_drops: [],
rounds: [
(
contributors: 5,
),
],
),
(
id: "36",
verifiers: 2,
replacement_contributors: 0,
environment: development,
timout: Some(18000),
skip: false,
/// Configure the tests performed for each round of the ceremony.
rounds: [
(
contributors: 1
),
(
contributors: 1
),
],
),
],
)
Loading