WIP reference documentation

Viasat · Oct 16, 2024 · d390092 · d390092
1 parent 813594d
commit d390092
Show file tree

Hide file tree

Showing 10 changed files with 4,320 additions and 54 deletions.
diff --git a/docs/.nojekyll b/docs/.nojekyll
diff --git a/docs/.swp b/docs/.swp
diff --git a/docs/_render-local-README.md b/docs/_render-local-README.md
@@ -0,0 +1 @@
+../README.md
diff --git a/docs/_sidebar.md b/docs/_sidebar.md
@@ -0,0 +1,8 @@
+- **Home**
+  - [Getting Started](/)
+- **Reference**
+  - [Input Syntax](/reference/latest/input)
+  - [Results File Schema](/reference/latest/results-file)
+  - [Expressions](/reference/latest/expressions)
+- **Examples**
+  - [View on GitHub](https://github.com/Viasat/dctest/tree/main/examples)
diff --git a/docs/index.html b/docs/index.html
@@ -0,0 +1,29 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8">
+  <title>dctest - Integration Testing in Docker Compose</title>
+  <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" />
+  <meta name="description" content="Description">
+  <meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=1.0">
+  <link rel="stylesheet" href="//cdn.jsdelivr.net/npm/docsify@4/lib/themes/vue.css">
+</head>
+<body>
+  <div id="app"></div>
+  <script>
+    window.$docsify = {
+      name: 'dctest',
+      repo: 'Viasat/dctest',
+      homepage: 'https://raw.githubusercontent.com/Viasat/dctest/main/README.md',
+      // Uncomment to follow a symlink to root README.md and test rendering locally
+      //homepage: '_render-local-README.md',
+
+      auto2top: true,
+      loadSidebar: true,
+      subMaxLevel: 3
+    }
+  </script>
+  <!-- Docsify v4 -->
+  <script src="//cdn.jsdelivr.net/npm/docsify@4"></script>
+</body>
+</html>
diff --git a/docs/reference/latest/expressions.md b/docs/reference/latest/expressions.md
@@ -0,0 +1,54 @@
+# Expressions
+
+## Basics
+
+types and operators
+
+## Contexts
+
+### env
+
+* `env.COMPOSE_PROJECT_NAME` -
+
+### process
+
+* `process.platform` -
+* `process.pid` -
+* `process.ppid` -
+* `process.argv` -
+* `process.versions` -
+* `process.features` -
+* `process.env` -
+
+### step
+
+* `step.stdout` -
+* `step.stderr` -
+
+## Functions and Methods
+
+### Status
+
+* `always()` -
+* `success()` -
+* `failure()` -
+
+### Conversions
+
+* `toJSON(value)` -
+* `fromJSON(value)` -
+* `value.toString()` - ex: `[].toString()` would return the string "[]"
+
+### Collections
+
+* `coll.count()` - ex: `[1, 2, 3].count()` would return 3
+
+### String
+
+* `contains(txt, s)` -
+* `startsWith(txt, s)` -
+* `endsWith(txt, s)` -
+
+### Error
+
+* `throw(msg)` -
diff --git a/docs/reference/latest/input.md b/docs/reference/latest/input.md
@@ -0,0 +1,100 @@
+# Input Syntax
+
+## Overview
+
+dctest runs suites of tests, where each test can have any number of steps.
+Here is a very simple suite that defines a single test:
+
+``` example_suite.yaml
+name: Example Suite
+tests:
+  example-1:
+    name: Test echo in node1
+    steps:
+      - name: Step 1
+        exec: node1
+        run: echo "Hi from node1 in project ${{ env.COMPOSE_PROJECT_NAME }}"
+
+      - name: Step 2
+        exec: node1
+        run: |
+          echo "YAML multiline strings can be useful ..."
+          echo "... for more complicated commands."
+```
+
+> NOTE: The input format is defined in [JSON Schema](https://github.com/Viasat/dctest/blob/main/schemas/input.yaml).
+
+In the above example, our lone test has an id `example-1`, which is how this
+test could be referenced by other tests or in the CLI option `--test-filter`.
+The suite, test, and step names are for human readability and are shown in
+results output.
+
+The test above defines two steps. Steps are run in order, top to bottom. Each
+step defines a command to be run (`run`) and a place to run it (`exec`). In
+this example both steps run their commands in a Docker Compose service called
+`node1`, essentially doing `docker compose exec node1 <command>`.
+
+The first step is using `echo` in the `node1` service, but it's also using the
+special syntax `${{ ... }}`. dctest will evaluate the [expression](/reference/latest/expressions)
+inside the `${{ ... }}` and interpolate the result into the command before
+running it. In this case, the step is simply getting the Docker Compose project
+name.
+
+The second step demonstrates another common pattern of using YAML multiline
+strings for commands. Sometimes this is done to improve readability, but
+sometimes when using expressions and interpolation, this will be necessary for
+the suite file to be valid YAML.
+
+If both of the steps' commands return a zero exit code, the test will pass!
+
+There are more [runnable examples](https://github.com/Viasat/dctest/tree/main/examples)
+that demonstrate other features. The following reference documentation
+covers what is available in suites, tests, and steps
+
+## Suite
+
+> NOTE: `env` is interpolated before all other keys.
+
+* `env` - A mapping of env variable names and interpolatable values, used as
+  part of the environment in all tests and steps in the suite
+* `name` - An interpolatable string to serve as a human-readable suite name
+* `tests` - A mapping of test ids to tests for this suite
+
+## Test
+
+> NOTE: `env` is interpolated before all other keys.
+
+* `env` - A mapping of env variable names and interpolatable values, used as
+  part of the environment in all steps in the test
+* `depends` - Either a string or list of strings referring to test ids
+  that must be run before this test is eligible to be run
+* `name` - An interpolatable string to serve as a human-readable test name
+* `steps` - A list of steps to be run in order for this test
+
+## Step
+
+> NOTE: `if` is evaluated before `env` interpolation. `env` is interpolated,
+  before all other keys.
+
+* `env` - A mapping of env variable names and interpolatable values, used as
+  part of the environment in the step
+* `exec` - An interpolatable string representing the location to run the `run`
+  command, either a Docker Compose service name or the special value `:host`
+* `expect` - An expression or list of expressions (no `${{ ... }}`) that are
+   evaluated after the `run` command. If all results are truthy, the step
+   succeeds; otherwise, it fails (or is retried, if `repeat` is set)
+* `if` - An expression (no `${{ ... }}`), evaluated to determine if this step
+   should be run. Default is `success()`
+* `index` - An integer referencing the index of the container, if there are
+   multiple (default is 1)
+* `name` - An interpolatable string to serve as a human-readable step name
+* `repeat` - A mapping to indicate a step should be retried if the `run` or
+  `expect` conditions failed
+  * `repeat.interval` - Time to wait between retries in Docker Compose
+    healthcheck format, ex: `1m20s`
+  * `repeat.retries` - An integer indicating number of retry attempts. If
+    omitted, retry indefinitely
+* `run` - An interpolatable string or list of interpolatable strings to
+  be run as a command
+* `shell` - A string to define the shell for the step, either `sh` or `bash`.
+  (default is `sh`)
diff --git a/docs/reference/latest/results-file.md b/docs/reference/latest/results-file.md
@@ -0,0 +1,45 @@
+# Results File Schema
+
+## Overview
+
+dctest can optionally write a summary of results as JSON data to a file (see
+the `--results-file` CLI option). This JSON data contains the followin top-level
+keys, which are further described below:
+
+* `summary` - overall outcome, runtime, and test outcome counts
+* `tests` - all tests executed as part of dctest run
+* `errors` - a list of errors encountered during execution
+
+## Summary
+
+* `summary.outcome` -
+* `summary.passed` -
+* `summary.failed` -
+* `summary.start` -
+* `summary.stop` -
+
+## Tests
+
+* `tests[].id` -
+* `tests[].name` -
+* `tests[].outcome` -
+* `tests[].error` -
+* `tests[].start` -
+* `tests[].stop` -
+* `tests[].steps` -
+
+For steps:
+
+* `tests[].steps[].name` -
+* `tests[].steps[].outcome` -
+* `tests[].steps[].error` -
+* `tests[].steps[].start` -
+* `tests[].steps[].stop` -
+
+
+## Errors
+
+> NOTE: `errors` may contain additional information that is not specified.
+  Any additional keys should be considered experimental.
+
+* `errors[].message` -