diff --git a/main/404.html b/main/404.html index b4f24a025..11eb66351 100644 --- a/main/404.html +++ b/main/404.html @@ -1 +1 @@ - Chainsaw

404 - Not found

\ No newline at end of file + Chainsaw

404 - Not found

\ No newline at end of file diff --git a/main/apis/chainsaw.v1alpha1/index.html b/main/apis/chainsaw.v1alpha1/index.html index 5bb2a1de9..a2e49fc7c 100644 --- a/main/apis/chainsaw.v1alpha1/index.html +++ b/main/apis/chainsaw.v1alpha1/index.html @@ -1 +1 @@ - chainsaw (v1alpha1) - Chainsaw
Skip to content

v1alpha1

Package v1alpha1 contains API Schema definitions for the v1alpha1 API group.

Resource Types

Configuration

Configuration is the resource that contains the configuration used to run tests.

Field Type Required Inline Description
apiVersion string ✅ chainsaw.kyverno.io/v1alpha1
kind string ✅ Configuration
metadata meta/v1.ObjectMeta

Standard object's metadata.
spec ConfigurationSpec ✅

Configuration spec.

Test

Test is the resource that contains a test definition.

Field Type Required Inline Description
apiVersion string ✅ chainsaw.kyverno.io/v1alpha1
kind string ✅ Test
metadata meta/v1.ObjectMeta

Standard object's metadata.
spec TestSpec ✅

Test spec.

Apply

Appears in:

Apply represents a set of configurations or resources that should be applied during testing.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
outputs []Output

Outputs defines output bindings.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
FileRefOrResource FileRefOrResource ✅ ✅

FileRefOrResource provides a reference to the resources to be applied.
template bool

Template determines whether resources should be considered for templating.
dryRun bool

DryRun determines whether the file should be applied in dry run mode.
expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

Assert

Appears in:

Assert represents a test condition that is expected to hold true during the testing process.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
FileRefOrCheck FileRefOrCheck ✅ ✅

FileRefOrAssert provides a reference to the assertion.
template bool

Template determines whether resources should be considered for templating.

Binding

Appears in:

Binding represents a key/value set as a binding in an executing test.

Field Type Required Inline Description
name string ✅

Name the name of the binding.
value policy/v1alpha1.Any ✅

Value value of the binding.

Catch

Appears in:

Catch defines actions to be executed on failure.

Field Type Required Inline Description
description string

Description contains a description of the operation.
podLogs PodLogs

PodLogs determines the pod logs collector to execute.
events Events

Events determines the events collector to execute.
describe Describe

Describe determines the resource describe collector to execute.
wait Wait

Wait determines the resource wait collector to execute.
get Get

Get determines the resource get collector to execute.
delete Delete

Delete represents a deletion operation.
command Command

Command defines a command to run.
script Script

Script defines a script to run.
sleep Sleep

Sleep defines zzzz.

Cluster

Appears in:

Cluster defines cluster config and context.

Field Type Required Inline Description
kubeconfig string ✅

Kubeconfig is the path to the referenced file.
context string

Context is the name of the context to use.

Command

Appears in:

Command describes a command to run as a part of a test step.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
outputs []Output

Outputs defines output bindings.
env []Binding

Env defines additional environment variables.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
entrypoint string ✅

Entrypoint is the command entry point to run.
args []string

Args is the command arguments.
skipLogOutput bool

SkipLogOutput removes the output from the command. Useful for sensitive logs or to reduce noise.
check policy/v1alpha1.Any

Check is an assertion tree to validate the operation outcome.

Condition

Appears in:

Condition represents parameters for waiting on a specific condition of a resource.

Field Type Required Inline Description
name string ✅

Name defines the specific condition to wait for, e.g., "Available", "Ready".
value string

Value defines the specific condition status to wait for, e.g., "True", "False".

ConfigurationSpec

Appears in:

ConfigurationSpec contains the configuration used to run tests.

Field Type Required Inline Description
timeouts Timeouts

Global timeouts configuration. Applies to all tests/test steps if not overridden.
skipDelete bool

If set, do not delete the resources after running the tests (implies SkipClusterDelete).
template bool

Template determines whether resources should be considered for templating.
failFast bool

FailFast determines whether the test should stop upon encountering the first failure.
parallel int

The maximum number of tests to run at once.
reportFormat ReportFormatType

ReportFormat determines test report format (JSON
reportPath string

ReportPath defines the path.
reportName string

ReportName defines the name of report to create. It defaults to "chainsaw-report".
namespace string

Namespace defines the namespace to use for tests. If not specified, every test will execute in a random ephemeral namespace unless the namespace is overridden in a the test spec.
namespaceTemplate policy/v1alpha1.Any

NamespaceTemplate defines a template to create the test namespace.
fullName bool

FullName makes use of the full test case folder path instead of the folder name.
excludeTestRegex string

ExcludeTestRegex is used to exclude tests based on a regular expression.
includeTestRegex string

IncludeTestRegex is used to include tests based on a regular expression.
repeatCount int

RepeatCount indicates how many times the tests should be executed.
testFile string

TestFile is the name of the file containing the test to run. If no extension is provided, chainsaw will try with .yaml first and .yml if needed.
forceTerminationGracePeriod meta/v1.Duration

ForceTerminationGracePeriod forces the termination grace period on pods, statefulsets, daemonsets and deployments.
delayBeforeCleanup meta/v1.Duration

DelayBeforeCleanup adds a delay between the time a test ends and the time cleanup starts.
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
catch []Catch

Catch defines what the tests steps will execute when an error happens. This will be combined with catch handlers defined at the test and step levels.

Create

Appears in:

Create represents a set of resources that should be created. If a resource already exists in the cluster it will fail.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
outputs []Output

Outputs defines output bindings.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
FileRefOrResource FileRefOrResource ✅ ✅

FileRefOrResource provides a reference to the file containing the resources to be created.
template bool

Template determines whether resources should be considered for templating.
dryRun bool

DryRun determines whether the file should be applied in dry run mode.
expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

Delete

Appears in:

Delete is a reference to an object that should be deleted

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
template bool

Template determines whether resources should be considered for templating.
ref ObjectReference ✅

ObjectReference determines objects to be deleted.
expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

Deletion

Appears in:

Deletion represents parameters for waiting on a resource's deletion.

Describe

Appears in:

Describe defines how to describe resources.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
ResourceReference ResourceReference ✅ ✅

ResourceReference referenced resource type.
ObjectLabelsSelector ObjectLabelsSelector ✅ ✅

ObjectLabelsSelector determines the selection process of referenced objects.
showEvents bool

Show Events indicates whether to include related events.

Error

Appears in:

Error represents an anticipated error condition that may arise during testing. Instead of treating such an error as a test failure, it acknowledges it as expected.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
FileRefOrCheck FileRefOrCheck ✅ ✅

FileRefOrAssert provides a reference to the expected error.
template bool

Template determines whether resources should be considered for templating.

Events

Appears in:

Events defines how to collect events.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
ObjectLabelsSelector ObjectLabelsSelector ✅ ✅

ObjectLabelsSelector determines the selection process of referenced objects.
format Format

Format determines the output format (json or yaml).

Expectation

Appears in:

Expectation represents a check to be applied on the result of an operation with a match filter to determine if the verification should be considered.

Field Type Required Inline Description
match policy/v1alpha1.Any

Match defines the matching statement.
check policy/v1alpha1.Any ✅

Check defines the verification statement.

FileRef

Appears in:

FileRef represents a file reference.

Field Type Required Inline Description
file string ✅

File is the path to the referenced file. This can be a direct path to a file or an expression that matches multiple files, such as "manifest/*.yaml" for all YAML files within the "manifest" directory.

FileRefOrCheck

Appears in:

FileRefOrCheck represents a file reference or resource.

Field Type Required Inline Description
FileRef FileRef ✅

FileRef provides a reference to the file containing the resources to be applied.
resource policy/v1alpha1.Any

Check provides a check used in assertions.

FileRefOrResource

Appears in:

FileRefOrResource represents a file reference or resource.

Field Type Required Inline Description
FileRef FileRef ✅

FileRef provides a reference to the file containing the resources to be applied.
resource meta/v1/unstructured.Unstructured

Resource provides a resource to be applied.

Finally

Appears in:

Finally defines actions to be executed at the end of a test.

Field Type Required Inline Description
description string

Description contains a description of the operation.
podLogs PodLogs

PodLogs determines the pod logs collector to execute.
events Events

Events determines the events collector to execute.
describe Describe

Describe determines the resource describe collector to execute.
wait Wait

Wait determines the resource wait collector to execute.
get Get

Get determines the resource get collector to execute.
delete Delete

Delete represents a deletion operation.
command Command

Command defines a command to run.
script Script

Script defines a script to run.
sleep Sleep

Sleep defines zzzz.

For

Appears in:

For specifies the condition to wait for.

Field Type Required Inline Description
deletion Deletion

Deletion specifies parameters for waiting on a resource's deletion.
condition Condition

Condition specifies the condition to wait for.
jsonPath JsonPath

JsonPath specifies the json path condition to wait for.

Format

(Alias of string)

Appears in:

Format determines the output format (json or yaml).

Get

Appears in:

Get defines how to get resources.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
ResourceReference ResourceReference ✅ ✅

ResourceReference referenced resource type.
ObjectLabelsSelector ObjectLabelsSelector ✅ ✅

ObjectLabelsSelector determines the selection process of referenced objects.
format Format

Format determines the output format (json or yaml).

JsonPath

Appears in:

JsonPath represents parameters for waiting on a json path of a resource.

Field Type Required Inline Description
path string ✅

Path defines the json path to wait for, e.g. '{.status.phase}'.
value string ✅

Value defines the expected value to wait for, e.g., "Running".

ObjectLabelsSelector

Appears in:

ObjectLabelsSelector represents a strategy to select objects. For a single object name and namespace are used to identify the object. For multiple objects use selector.

Field Type Required Inline Description
namespace string

Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/
name string

Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
selector string

Selector defines labels selector.

ObjectReference

Appears in:

ObjectReference represents one or more objects with a specific apiVersion and kind. For a single object name and namespace are used to identify the object. For multiple objects use labels.

Field Type Required Inline Description
ObjectType ObjectType ✅ ✅

ObjectType determines the type of referenced objects.
ObjectSelector ObjectSelector ✅ ✅

ObjectSelector determines the selection process of referenced objects.

ObjectSelector

Appears in:

ObjectSelector represents a strategy to select objects. For a single object name and namespace are used to identify the object. For multiple objects use labels.

Field Type Required Inline Description
namespace string

Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/
name string

Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
labels map[string]string

Label selector to match objects to delete

ObjectType

Appears in:

ObjectType represents a specific apiVersion and kind.

Field Type Required Inline Description
apiVersion string ✅

API version of the referent.
kind string ✅

Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

Operation

Appears in:

Operation defines a single operation, only one action is permitted for a given operation.

Field Type Required Inline Description
description string

Description contains a description of the operation.
continueOnError bool

ContinueOnError determines whether a test should continue or not in case the operation was not successful. Even if the test continues executing, it will still be reported as failed.
apply Apply

Apply represents resources that should be applied for this test step. This can include things like configuration settings or any other resources that need to be available during the test.
assert Assert

Assert represents an assertion to be made. It checks whether the conditions specified in the assertion hold true.
command Command

Command defines a command to run.
create Create

Create represents a creation operation.
delete Delete

Delete represents a deletion operation.
error Error

Error represents the expected errors for this test step. If any of these errors occur, the test will consider them as expected; otherwise, they will be treated as test failures.
patch Patch

Patch represents a patch operation.
script Script

Script defines a script to run.
sleep Sleep

Sleep defines zzzz.
update Update

Update represents an update operation.
wait Wait

Wait determines the resource wait collector to execute.

Output

Appears in:

Output represents an output binding with a match to determine if the binding must be considered or not.

Field Type Required Inline Description
Binding Binding ✅ ✅

Binding determines the binding to create when the match succeeds.
match policy/v1alpha1.Any

Match defines the matching statement.

Patch

Appears in:

Patch represents a set of resources that should be patched. If a resource doesn't exist yet in the cluster it will fail.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
outputs []Output

Outputs defines output bindings.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
FileRefOrResource FileRefOrResource ✅ ✅

FileRefOrResource provides a reference to the file containing the resources to be patched.
template bool

Template determines whether resources should be considered for templating.
dryRun bool

DryRun determines whether the file should be applied in dry run mode.
expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

PodLogs

Appears in:

PodLogs defines how to collect pod logs.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
ObjectLabelsSelector ObjectLabelsSelector ✅ ✅

ObjectLabelsSelector determines the selection process of referenced objects.
container string

Container in pod to get logs from else --all-containers is used.
tail int

Tail is the number of last lines to collect from pods. If omitted or zero, then the default is 10 if you use a selector, or -1 (all) if you use a pod name. This matches default behavior of kubectl logs.

ReportFormatType

(Alias of string)

Appears in:

ResourceReference

Appears in:

ResourceReference represents a resource (API), it can be represented with a resource or a kind. Optionally an apiVersion can be specified.

Field Type Required Inline Description
apiVersion string

API version of the referent.
kind string

Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
resource string

Resource name of the referent.

Script

Appears in:

Script describes a script to run as a part of a test step.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
outputs []Output

Outputs defines output bindings.
env []Binding

Env defines additional environment variables.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
content string

Content defines a shell script (run with "sh -c ...").
skipLogOutput bool

SkipLogOutput removes the output from the command. Useful for sensitive logs or to reduce noise.
check policy/v1alpha1.Any

Check is an assertion tree to validate the operation outcome.

Sleep

Appears in:

Sleep represents a duration while nothing happens.

Field Type Required Inline Description
duration meta/v1.Duration ✅

Duration is the delay used for sleeping.

TestSpec

Appears in:

TestSpec contains the test spec.

Field Type Required Inline Description
description string

Description contains a description of the test.
timeouts Timeouts

Timeouts for the test. Overrides the global timeouts set in the Configuration on a per operation basis.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
skip bool

Skip determines whether the test should skipped.
concurrent bool

Concurrent determines whether the test should run concurrently with other tests.
skipDelete bool

SkipDelete determines whether the resources created by the test should be deleted after the test is executed.
template bool

Template determines whether resources should be considered for templating.
namespace string

Namespace determines whether the test should run in a random ephemeral namespace or not.
namespaceTemplate policy/v1alpha1.Any

NamespaceTemplate defines a template to create the test namespace.
bindings []Binding

Bindings defines additional binding key/values.
steps []TestStep ✅

Steps defining the test.
catch []Catch

Catch defines what the steps will execute when an error happens. This will be combined with catch handlers defined at the step level.
forceTerminationGracePeriod meta/v1.Duration

ForceTerminationGracePeriod forces the termination grace period on pods, statefulsets, daemonsets and deployments.
delayBeforeCleanup meta/v1.Duration

DelayBeforeCleanup adds a delay between the time a test ends and the time cleanup starts.

TestStep

Appears in:

TestStep contains the test step definition used in a test spec.

Field Type Required Inline Description
name string

Name of the step.
TestStepSpec TestStepSpec ✅ ✅

TestStepSpec of the step.

TestStepSpec

Appears in:

TestStepSpec defines the desired state and behavior for each test step.

Field Type Required Inline Description
description string

Description contains a description of the test step.
timeouts Timeouts

Timeouts for the test step. Overrides the global timeouts set in the Configuration and the timeouts eventually set in the Test.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
skipDelete bool

SkipDelete determines whether the resources created by the step should be deleted after the test step is executed.
template bool

Template determines whether resources should be considered for templating.
bindings []Binding

Bindings defines additional binding key/values.
try []Operation ✅

Try defines what the step will try to execute.
catch []Catch

Catch defines what the step will execute when an error happens.
finally []Finally

Finally defines what the step will execute after the step is terminated.
cleanup []Finally

Cleanup defines what will be executed after the test is terminated.

Timeouts

Appears in:

Timeouts contains timeouts per operation.

Field Type Required Inline Description
apply meta/v1.Duration ✅

Apply defines the timeout for the apply operation
assert meta/v1.Duration ✅

Assert defines the timeout for the assert operation
cleanup meta/v1.Duration ✅

Cleanup defines the timeout for the cleanup operation
delete meta/v1.Duration ✅

Delete defines the timeout for the delete operation
error meta/v1.Duration ✅

Error defines the timeout for the error operation
exec meta/v1.Duration ✅

Exec defines the timeout for exec operations

Update

Appears in:

Update represents a set of resources that should be updated. If a resource does not exist in the cluster it will fail.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
outputs []Output

Outputs defines output bindings.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
FileRefOrResource FileRefOrResource ✅ ✅

FileRefOrResource provides a reference to the file containing the resources to be created.
template bool

Template determines whether resources should be considered for templating.
dryRun bool

DryRun determines whether the file should be applied in dry run mode.
expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

Wait

Appears in:

Wait specifies how to perform wait operations on resources.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Specifies how long to wait for the condition to be met before timing out.
cluster string

Cluster defines the target cluster where the wait operation will be performed (default cluster will be used if not specified).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
ResourceReference ResourceReference ✅ ✅

ResourceReference referenced resource type.
ObjectLabelsSelector ObjectLabelsSelector ✅ ✅

ObjectLabelsSelector determines the selection process of referenced objects.
for For ✅

For specifies the condition to wait for.
format Format

Format determines the output format (json or yaml).
\ No newline at end of file + chainsaw (v1alpha1) - Chainsaw
Skip to content

v1alpha1

Package v1alpha1 contains API Schema definitions for the v1alpha1 API group.

Resource Types

Configuration

Configuration is the resource that contains the configuration used to run tests.

Field Type Required Inline Description
apiVersion string ✅ chainsaw.kyverno.io/v1alpha1
kind string ✅ Configuration
metadata meta/v1.ObjectMeta

Standard object's metadata.
spec ConfigurationSpec ✅

Configuration spec.

Test

Test is the resource that contains a test definition.

Field Type Required Inline Description
apiVersion string ✅ chainsaw.kyverno.io/v1alpha1
kind string ✅ Test
metadata meta/v1.ObjectMeta

Standard object's metadata.
spec TestSpec ✅

Test spec.

Apply

Appears in:

Apply represents a set of configurations or resources that should be applied during testing.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
outputs []Output

Outputs defines output bindings.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
FileRefOrResource FileRefOrResource ✅ ✅

FileRefOrResource provides a reference to the resources to be applied.
template bool

Template determines whether resources should be considered for templating.
dryRun bool

DryRun determines whether the file should be applied in dry run mode.
expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

Assert

Appears in:

Assert represents a test condition that is expected to hold true during the testing process.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
FileRefOrCheck FileRefOrCheck ✅ ✅

FileRefOrAssert provides a reference to the assertion.
template bool

Template determines whether resources should be considered for templating.

Binding

Appears in:

Binding represents a key/value set as a binding in an executing test.

Field Type Required Inline Description
name string ✅

Name the name of the binding.
value policy/v1alpha1.Any ✅

Value value of the binding.

Catch

Appears in:

Catch defines actions to be executed on failure.

Field Type Required Inline Description
description string

Description contains a description of the operation.
podLogs PodLogs

PodLogs determines the pod logs collector to execute.
events Events

Events determines the events collector to execute.
describe Describe

Describe determines the resource describe collector to execute.
wait Wait

Wait determines the resource wait collector to execute.
get Get

Get determines the resource get collector to execute.
delete Delete

Delete represents a deletion operation.
command Command

Command defines a command to run.
script Script

Script defines a script to run.
sleep Sleep

Sleep defines zzzz.

Cluster

Appears in:

Cluster defines cluster config and context.

Field Type Required Inline Description
kubeconfig string ✅

Kubeconfig is the path to the referenced file.
context string

Context is the name of the context to use.

Command

Appears in:

Command describes a command to run as a part of a test step.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
outputs []Output

Outputs defines output bindings.
env []Binding

Env defines additional environment variables.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
entrypoint string ✅

Entrypoint is the command entry point to run.
args []string

Args is the command arguments.
skipLogOutput bool

SkipLogOutput removes the output from the command. Useful for sensitive logs or to reduce noise.
check policy/v1alpha1.Any

Check is an assertion tree to validate the operation outcome.

Condition

Appears in:

Condition represents parameters for waiting on a specific condition of a resource.

Field Type Required Inline Description
name string ✅

Name defines the specific condition to wait for, e.g., "Available", "Ready".
value string

Value defines the specific condition status to wait for, e.g., "True", "False".

ConfigurationSpec

Appears in:

ConfigurationSpec contains the configuration used to run tests.

Field Type Required Inline Description
timeouts Timeouts

Global timeouts configuration. Applies to all tests/test steps if not overridden.
skipDelete bool

If set, do not delete the resources after running the tests (implies SkipClusterDelete).
template bool

Template determines whether resources should be considered for templating.
failFast bool

FailFast determines whether the test should stop upon encountering the first failure.
parallel int

The maximum number of tests to run at once.
reportFormat ReportFormatType

ReportFormat determines test report format (JSON
reportPath string

ReportPath defines the path.
reportName string

ReportName defines the name of report to create. It defaults to "chainsaw-report".
namespace string

Namespace defines the namespace to use for tests. If not specified, every test will execute in a random ephemeral namespace unless the namespace is overridden in a the test spec.
namespaceTemplate policy/v1alpha1.Any

NamespaceTemplate defines a template to create the test namespace.
fullName bool

FullName makes use of the full test case folder path instead of the folder name.
excludeTestRegex string

ExcludeTestRegex is used to exclude tests based on a regular expression.
includeTestRegex string

IncludeTestRegex is used to include tests based on a regular expression.
repeatCount int

RepeatCount indicates how many times the tests should be executed.
testFile string

TestFile is the name of the file containing the test to run. If no extension is provided, chainsaw will try with .yaml first and .yml if needed.
forceTerminationGracePeriod meta/v1.Duration

ForceTerminationGracePeriod forces the termination grace period on pods, statefulsets, daemonsets and deployments.
delayBeforeCleanup meta/v1.Duration

DelayBeforeCleanup adds a delay between the time a test ends and the time cleanup starts.
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
catch []Catch

Catch defines what the tests steps will execute when an error happens. This will be combined with catch handlers defined at the test and step levels.

Create

Appears in:

Create represents a set of resources that should be created. If a resource already exists in the cluster it will fail.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
outputs []Output

Outputs defines output bindings.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
FileRefOrResource FileRefOrResource ✅ ✅

FileRefOrResource provides a reference to the file containing the resources to be created.
template bool

Template determines whether resources should be considered for templating.
dryRun bool

DryRun determines whether the file should be applied in dry run mode.
expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

Delete

Appears in:

Delete is a reference to an object that should be deleted

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
template bool

Template determines whether resources should be considered for templating.
ref ObjectReference ✅

ObjectReference determines objects to be deleted.
expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

Deletion

Appears in:

Deletion represents parameters for waiting on a resource's deletion.

Describe

Appears in:

Describe defines how to describe resources.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
ResourceReference ResourceReference ✅ ✅

ResourceReference referenced resource type.
ObjectLabelsSelector ObjectLabelsSelector ✅ ✅

ObjectLabelsSelector determines the selection process of referenced objects.
showEvents bool

Show Events indicates whether to include related events.

Error

Appears in:

Error represents an anticipated error condition that may arise during testing. Instead of treating such an error as a test failure, it acknowledges it as expected.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
FileRefOrCheck FileRefOrCheck ✅ ✅

FileRefOrAssert provides a reference to the expected error.
template bool

Template determines whether resources should be considered for templating.

Events

Appears in:

Events defines how to collect events.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
ObjectLabelsSelector ObjectLabelsSelector ✅ ✅

ObjectLabelsSelector determines the selection process of referenced objects.
format Format

Format determines the output format (json or yaml).

Expectation

Appears in:

Expectation represents a check to be applied on the result of an operation with a match filter to determine if the verification should be considered.

Field Type Required Inline Description
match policy/v1alpha1.Any

Match defines the matching statement.
check policy/v1alpha1.Any ✅

Check defines the verification statement.

FileRef

Appears in:

FileRef represents a file reference.

Field Type Required Inline Description
file string ✅

File is the path to the referenced file. This can be a direct path to a file or an expression that matches multiple files, such as "manifest/*.yaml" for all YAML files within the "manifest" directory.

FileRefOrCheck

Appears in:

FileRefOrCheck represents a file reference or resource.

Field Type Required Inline Description
FileRef FileRef ✅

FileRef provides a reference to the file containing the resources to be applied.
resource policy/v1alpha1.Any

Check provides a check used in assertions.

FileRefOrResource

Appears in:

FileRefOrResource represents a file reference or resource.

Field Type Required Inline Description
FileRef FileRef ✅

FileRef provides a reference to the file containing the resources to be applied.
resource meta/v1/unstructured.Unstructured

Resource provides a resource to be applied.

Finally

Appears in:

Finally defines actions to be executed at the end of a test.

Field Type Required Inline Description
description string

Description contains a description of the operation.
podLogs PodLogs

PodLogs determines the pod logs collector to execute.
events Events

Events determines the events collector to execute.
describe Describe

Describe determines the resource describe collector to execute.
wait Wait

Wait determines the resource wait collector to execute.
get Get

Get determines the resource get collector to execute.
delete Delete

Delete represents a deletion operation.
command Command

Command defines a command to run.
script Script

Script defines a script to run.
sleep Sleep

Sleep defines zzzz.

For

Appears in:

For specifies the condition to wait for.

Field Type Required Inline Description
deletion Deletion

Deletion specifies parameters for waiting on a resource's deletion.
condition Condition

Condition specifies the condition to wait for.
jsonPath JsonPath

JsonPath specifies the json path condition to wait for.

Format

(Alias of string)

Appears in:

Format determines the output format (json or yaml).

Get

Appears in:

Get defines how to get resources.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
ResourceReference ResourceReference ✅ ✅

ResourceReference referenced resource type.
ObjectLabelsSelector ObjectLabelsSelector ✅ ✅

ObjectLabelsSelector determines the selection process of referenced objects.
format Format

Format determines the output format (json or yaml).

JsonPath

Appears in:

JsonPath represents parameters for waiting on a json path of a resource.

Field Type Required Inline Description
path string ✅

Path defines the json path to wait for, e.g. '{.status.phase}'.
value string ✅

Value defines the expected value to wait for, e.g., "Running".

ObjectLabelsSelector

Appears in:

ObjectLabelsSelector represents a strategy to select objects. For a single object name and namespace are used to identify the object. For multiple objects use selector.

Field Type Required Inline Description
namespace string

Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/
name string

Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
selector string

Selector defines labels selector.

ObjectReference

Appears in:

ObjectReference represents one or more objects with a specific apiVersion and kind. For a single object name and namespace are used to identify the object. For multiple objects use labels.

Field Type Required Inline Description
ObjectType ObjectType ✅ ✅

ObjectType determines the type of referenced objects.
ObjectSelector ObjectSelector ✅ ✅

ObjectSelector determines the selection process of referenced objects.

ObjectSelector

Appears in:

ObjectSelector represents a strategy to select objects. For a single object name and namespace are used to identify the object. For multiple objects use labels.

Field Type Required Inline Description
namespace string

Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/
name string

Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
labels map[string]string

Label selector to match objects to delete

ObjectType

Appears in:

ObjectType represents a specific apiVersion and kind.

Field Type Required Inline Description
apiVersion string ✅

API version of the referent.
kind string ✅

Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

Operation

Appears in:

Operation defines a single operation, only one action is permitted for a given operation.

Field Type Required Inline Description
description string

Description contains a description of the operation.
continueOnError bool

ContinueOnError determines whether a test should continue or not in case the operation was not successful. Even if the test continues executing, it will still be reported as failed.
apply Apply

Apply represents resources that should be applied for this test step. This can include things like configuration settings or any other resources that need to be available during the test.
assert Assert

Assert represents an assertion to be made. It checks whether the conditions specified in the assertion hold true.
command Command

Command defines a command to run.
create Create

Create represents a creation operation.
delete Delete

Delete represents a deletion operation.
error Error

Error represents the expected errors for this test step. If any of these errors occur, the test will consider them as expected; otherwise, they will be treated as test failures.
patch Patch

Patch represents a patch operation.
script Script

Script defines a script to run.
sleep Sleep

Sleep defines zzzz.
update Update

Update represents an update operation.
wait Wait

Wait determines the resource wait collector to execute.

Output

Appears in:

Output represents an output binding with a match to determine if the binding must be considered or not.

Field Type Required Inline Description
Binding Binding ✅ ✅

Binding determines the binding to create when the match succeeds.
match policy/v1alpha1.Any

Match defines the matching statement.

Patch

Appears in:

Patch represents a set of resources that should be patched. If a resource doesn't exist yet in the cluster it will fail.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
outputs []Output

Outputs defines output bindings.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
FileRefOrResource FileRefOrResource ✅ ✅

FileRefOrResource provides a reference to the file containing the resources to be patched.
template bool

Template determines whether resources should be considered for templating.
dryRun bool

DryRun determines whether the file should be applied in dry run mode.
expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

PodLogs

Appears in:

PodLogs defines how to collect pod logs.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
ObjectLabelsSelector ObjectLabelsSelector ✅ ✅

ObjectLabelsSelector determines the selection process of referenced objects.
container string

Container in pod to get logs from else --all-containers is used.
tail int

Tail is the number of last lines to collect from pods. If omitted or zero, then the default is 10 if you use a selector, or -1 (all) if you use a pod name. This matches default behavior of kubectl logs.

ReportFormatType

(Alias of string)

Appears in:

ResourceReference

Appears in:

ResourceReference represents a resource (API), it can be represented with a resource or a kind. Optionally an apiVersion can be specified.

Field Type Required Inline Description
apiVersion string

API version of the referent.
kind string

Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
resource string

Resource name of the referent.

Script

Appears in:

Script describes a script to run as a part of a test step.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
outputs []Output

Outputs defines output bindings.
env []Binding

Env defines additional environment variables.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
content string

Content defines a shell script (run with "sh -c ...").
skipLogOutput bool

SkipLogOutput removes the output from the command. Useful for sensitive logs or to reduce noise.
check policy/v1alpha1.Any

Check is an assertion tree to validate the operation outcome.

Sleep

Appears in:

Sleep represents a duration while nothing happens.

Field Type Required Inline Description
duration meta/v1.Duration ✅

Duration is the delay used for sleeping.

TestSpec

Appears in:

TestSpec contains the test spec.

Field Type Required Inline Description
description string

Description contains a description of the test.
timeouts Timeouts

Timeouts for the test. Overrides the global timeouts set in the Configuration on a per operation basis.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
skip bool

Skip determines whether the test should skipped.
concurrent bool

Concurrent determines whether the test should run concurrently with other tests.
skipDelete bool

SkipDelete determines whether the resources created by the test should be deleted after the test is executed.
template bool

Template determines whether resources should be considered for templating.
namespace string

Namespace determines whether the test should run in a random ephemeral namespace or not.
namespaceTemplate policy/v1alpha1.Any

NamespaceTemplate defines a template to create the test namespace.
bindings []Binding

Bindings defines additional binding key/values.
steps []TestStep ✅

Steps defining the test.
catch []Catch

Catch defines what the steps will execute when an error happens. This will be combined with catch handlers defined at the step level.
forceTerminationGracePeriod meta/v1.Duration

ForceTerminationGracePeriod forces the termination grace period on pods, statefulsets, daemonsets and deployments.
delayBeforeCleanup meta/v1.Duration

DelayBeforeCleanup adds a delay between the time a test ends and the time cleanup starts.

TestStep

Appears in:

TestStep contains the test step definition used in a test spec.

Field Type Required Inline Description
name string

Name of the step.
TestStepSpec TestStepSpec ✅ ✅

TestStepSpec of the step.

TestStepSpec

Appears in:

TestStepSpec defines the desired state and behavior for each test step.

Field Type Required Inline Description
description string

Description contains a description of the test step.
timeouts Timeouts

Timeouts for the test step. Overrides the global timeouts set in the Configuration and the timeouts eventually set in the Test.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
skipDelete bool

SkipDelete determines whether the resources created by the step should be deleted after the test step is executed.
template bool

Template determines whether resources should be considered for templating.
bindings []Binding

Bindings defines additional binding key/values.
try []Operation ✅

Try defines what the step will try to execute.
catch []Catch

Catch defines what the step will execute when an error happens.
finally []Finally

Finally defines what the step will execute after the step is terminated.
cleanup []Finally

Cleanup defines what will be executed after the test is terminated.

Timeouts

Appears in:

Timeouts contains timeouts per operation.

Field Type Required Inline Description
apply meta/v1.Duration ✅

Apply defines the timeout for the apply operation
assert meta/v1.Duration ✅

Assert defines the timeout for the assert operation
cleanup meta/v1.Duration ✅

Cleanup defines the timeout for the cleanup operation
delete meta/v1.Duration ✅

Delete defines the timeout for the delete operation
error meta/v1.Duration ✅

Error defines the timeout for the error operation
exec meta/v1.Duration ✅

Exec defines the timeout for exec operations

Update

Appears in:

Update represents a set of resources that should be updated. If a resource does not exist in the cluster it will fail.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.
bindings []Binding

Bindings defines additional binding key/values.
outputs []Output

Outputs defines output bindings.
cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
FileRefOrResource FileRefOrResource ✅ ✅

FileRefOrResource provides a reference to the file containing the resources to be created.
template bool

Template determines whether resources should be considered for templating.
dryRun bool

DryRun determines whether the file should be applied in dry run mode.
expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

Wait

Appears in:

Wait specifies how to perform wait operations on resources.

Field Type Required Inline Description
timeout meta/v1.Duration

Timeout for the operation. Specifies how long to wait for the condition to be met before timing out.
cluster string

Cluster defines the target cluster where the wait operation will be performed (default cluster will be used if not specified).
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
ResourceReference ResourceReference ✅ ✅

ResourceReference referenced resource type.
ObjectLabelsSelector ObjectLabelsSelector ✅ ✅

ObjectLabelsSelector determines the selection process of referenced objects.
for For ✅

For specifies the condition to wait for.
format Format

Format determines the output format (json or yaml).
\ No newline at end of file diff --git a/main/apis/chainsaw.v1alpha2/index.html b/main/apis/chainsaw.v1alpha2/index.html index c3f668926..6acb49e48 100644 --- a/main/apis/chainsaw.v1alpha2/index.html +++ b/main/apis/chainsaw.v1alpha2/index.html @@ -1 +1 @@ - chainsaw (v1alpha2) - Chainsaw
Skip to content

v1alpha2

Package v1alpha2 contains API Schema definitions for the v1alpha2 API group.

Resource Types

Configuration

Configuration is the resource that contains the configuration used to run tests.

Field Type Required Inline Description
apiVersion string ✅ chainsaw.kyverno.io/v1alpha2
kind string ✅ Configuration
metadata meta/v1.ObjectMeta

Standard object's metadata.
spec ConfigurationSpec ✅

Configuration spec.

Cleanup

Appears in:

Cleanup contains the cleanup configuration.

Field Type Required Inline Description
skipDelete bool

If set, do not delete the resources after running a test.
delayBeforeCleanup meta/v1.Duration

DelayBeforeCleanup adds a delay between the time a test ends and the time cleanup starts.

ConfigurationSpec

Appears in:

ConfigurationSpec contains the configuration used to run tests.

Field Type Required Inline Description
catch []Catch

Catch defines what the tests steps will execute when an error happens. This will be combined with catch handlers defined at the test and step levels.
cleanup Cleanup

Cleanup contains cleanup configuration.
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
discovery Discovery

Discovery contains tests discovery configuration.
execution Execution

Execution contains tests execution configuration.
namespace Namespace

Namespace contains properties for the namespace to use for tests.
report Report

Report contains properties for the report.
templating Templating

Templating contains the templating config.
timeouts Timeouts

Global timeouts configuration. Applies to all tests/test steps if not overridden.

Discovery

Appears in:

Discovery contains the tests discovery configuration.

Field Type Required Inline Description
excludeTestRegex string

ExcludeTestRegex is used to exclude tests based on a regular expression.
includeTestRegex string

IncludeTestRegex is used to include tests based on a regular expression.
testFile string

TestFile is the name of the file containing the test to run. If no extension is provided, chainsaw will try with .yaml first and .yml if needed.
fullName bool

FullName makes use of the full test case folder path instead of the folder name.

Execution

Appears in:

Execution contains the runner configuration.

Field Type Required Inline Description
failFast bool

FailFast determines whether the test should stop upon encountering the first failure.
parallel int

The maximum number of tests to run at once.
repeatCount int

RepeatCount indicates how many times the tests should be executed.
forceTerminationGracePeriod meta/v1.Duration

ForceTerminationGracePeriod forces the termination grace period on pods, statefulsets, daemonsets and deployments.

Namespace

Appears in:

Namespace contains info about the namespace used for testing.

Field Type Required Inline Description
name string

Name defines the namespace to use for tests. If not specified, every test will execute in a random ephemeral namespace unless the namespace is overridden in a the test spec.
template policy/v1alpha1.Any

Template defines a template to create the test namespace.

Report

Appears in:

Report contains info about the report.

Field Type Required Inline Description
format ReportFormatType

ReportFormat determines test report format (JSON
path string

ReportPath defines the path.
name string

ReportName defines the name of report to create. It defaults to "chainsaw-report".

ReportFormatType

(Alias of string)

Appears in:

Templating

Appears in:

Templating contains the templating configuration.

Field Type Required Inline Description
enabled bool

Enabled determines whether resources should be considered for templating.
\ No newline at end of file + chainsaw (v1alpha2) - Chainsaw
Skip to content

v1alpha2

Package v1alpha2 contains API Schema definitions for the v1alpha2 API group.

Resource Types

Configuration

Configuration is the resource that contains the configuration used to run tests.

Field Type Required Inline Description
apiVersion string ✅ chainsaw.kyverno.io/v1alpha2
kind string ✅ Configuration
metadata meta/v1.ObjectMeta

Standard object's metadata.
spec ConfigurationSpec ✅

Configuration spec.

Cleanup

Appears in:

Cleanup contains the cleanup configuration.

Field Type Required Inline Description
skipDelete bool

If set, do not delete the resources after running a test.
delayBeforeCleanup meta/v1.Duration

DelayBeforeCleanup adds a delay between the time a test ends and the time cleanup starts.

ConfigurationSpec

Appears in:

ConfigurationSpec contains the configuration used to run tests.

Field Type Required Inline Description
catch []Catch

Catch defines what the tests steps will execute when an error happens. This will be combined with catch handlers defined at the test and step levels.
cleanup Cleanup

Cleanup contains cleanup configuration.
clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.
discovery Discovery

Discovery contains tests discovery configuration.
execution Execution

Execution contains tests execution configuration.
namespace Namespace

Namespace contains properties for the namespace to use for tests.
report Report

Report contains properties for the report.
templating Templating

Templating contains the templating config.
timeouts Timeouts

Global timeouts configuration. Applies to all tests/test steps if not overridden.

Discovery

Appears in:

Discovery contains the tests discovery configuration.

Field Type Required Inline Description
excludeTestRegex string

ExcludeTestRegex is used to exclude tests based on a regular expression.
includeTestRegex string

IncludeTestRegex is used to include tests based on a regular expression.
testFile string

TestFile is the name of the file containing the test to run. If no extension is provided, chainsaw will try with .yaml first and .yml if needed.
fullName bool

FullName makes use of the full test case folder path instead of the folder name.

Execution

Appears in:

Execution contains the runner configuration.

Field Type Required Inline Description
failFast bool

FailFast determines whether the test should stop upon encountering the first failure.
parallel int

The maximum number of tests to run at once.
repeatCount int

RepeatCount indicates how many times the tests should be executed.
forceTerminationGracePeriod meta/v1.Duration

ForceTerminationGracePeriod forces the termination grace period on pods, statefulsets, daemonsets and deployments.

Namespace

Appears in:

Namespace contains info about the namespace used for testing.

Field Type Required Inline Description
name string

Name defines the namespace to use for tests. If not specified, every test will execute in a random ephemeral namespace unless the namespace is overridden in a the test spec.
template policy/v1alpha1.Any

Template defines a template to create the test namespace.

Report

Appears in:

Report contains info about the report.

Field Type Required Inline Description
format ReportFormatType

ReportFormat determines test report format (JSON
path string

ReportPath defines the path.
name string

ReportName defines the name of report to create. It defaults to "chainsaw-report".

ReportFormatType

(Alias of string)

Appears in:

Templating

Appears in:

Templating contains the templating configuration.

Field Type Required Inline Description
enabled bool

Enabled determines whether resources should be considered for templating.
\ No newline at end of file diff --git a/main/bindings/index.html b/main/bindings/index.html index ef9327b9d..f34346a7e 100644 --- a/main/bindings/index.html +++ b/main/bindings/index.html @@ -1,4 +1,4 @@ - Bindings - Chainsaw
Skip to content

Bindings

Chainsaw has a concept of bindings which can be seen as an execution context.

Bindings are used in assertion trees and resource templating, as well as when using the --values flag when invoking chainsaw.

While some bindings are built-in and provided by chainsaw, it's possible to define custom bindings at the test, step, or individual operation level. Those bindings can in turn be used to create custom environment variables in command and script operations.

Immutability

It's important to note that bindings are immutable, it's not possible to overwrite a binding and two bindings with the same name can exist without overwriting each other.

When a binding value is resolved, chainsaw will walk the binding chain upwards until it finds a binding with the expected name. Effectively, the last one registered in the chain will be used.

Templating

A binding name supports templating.

The name of a binding can therefore be dynamic and depend on values or other bindings.

Usage

The example below defines custom bindings at the test level.

Test level bindings

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Bindings - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Bindings
 
Chainsaw has a concept of bindings which can be seen as an execution context.
 
Bindings are used in assertion trees and resource templating, as well as when using the --values flag when invoking chainsaw.
 
While some bindings are built-in and provided by chainsaw, it's possible to define custom bindings at the test, step, or individual operation level. Those bindings can in turn be used to create custom environment variables in command and script operations.
 
Immutability
 
It's important to note that bindings are immutable, it's not possible to overwrite a binding and two bindings with the same name can exist without overwriting each other.
 
When a binding value is resolved, chainsaw will walk the binding chain upwards until it finds a binding with the expected name. Effectively, the last one registered in the chain will be used.
 
Templating
 
A binding name supports templating.
 
The name of a binding can therefore be dynamic and depend on values or other bindings.
 
Usage
 
The example below defines custom bindings at the test level.
 
 
Test level bindings
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -76,4 +76,4 @@
         check:
           ($error): ~
           ($stdout): hello chainsaw
-
 
 
Outputs
 
Under certain conditions, bindings can also be used to add computed results to the context.
 
See Outputs for details.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Outputs

Under certain conditions, bindings can also be used to add computed results to the context.

See Outputs for details.

\ No newline at end of file diff --git a/main/bindings/outputs/index.html b/main/bindings/outputs/index.html index a1e965fd4..724cbd847 100644 --- a/main/bindings/outputs/index.html +++ b/main/bindings/outputs/index.html @@ -1,4 +1,4 @@ - Outputs - Chainsaw
Skip to content

Outputs

Binding outputs can be useful to communicate and reuse computation results between operations.

Supported operations

Currently, only script and command operations support outputs.

Lifetime of outputs

Once an output has been added in the form of a binding, this binding will be available to all following operations in the same step.

Currently, outputs do not cross the step boundaries.

Matching

An output supports an optional match field. The match is used to conditionally create a binding.

In the case of applying a file, for example, the file may contain multiple resources. The match can be used to select the resource to use for creating the binding.

Examples

The example below defines invokes a kubectl command to get a configmap from the cluster in json format.

The json output is then parsed and added to the $cm binding and the next operation performs an assertion on it by reading the binding instead of querying the cluster.

Output in script

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Outputs - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Outputs
 
Binding outputs can be useful to communicate and reuse computation results between operations.
 
Supported operations
 
Currently, only script and command operations support outputs.
 
Lifetime of outputs
 
Once an output has been added in the form of a binding, this binding will be available to all following operations in the same step.
 
Currently, outputs do not cross the step boundaries.
 
Matching
 
An output supports an optional match field. The match is used to conditionally create a binding.
 
In the case of applying a file, for example, the file may contain multiple resources. The match can be used to select the resource to use for creating the binding.
 
Examples
 
The example below defines invokes a kubectl command to get a configmap from the cluster in json format.
 
The json output is then parsed and added to the $cm binding and the next operation performs an assertion on it by reading the binding instead of querying the cluster.
 
 
Output in script
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -52,4 +52,4 @@
             metadata:
               (uid != null): true
   # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/collectors/describe/index.html b/main/collectors/describe/index.html index 8d21b10c7..b1e061fdd 100644 --- a/main/collectors/describe/index.html +++ b/main/collectors/describe/index.html @@ -1,4 +1,4 @@ - Describe - Chainsaw
Skip to content

Describe

Describing resources present in the cluster can help understand what happened and troubleshoot test failures.

Configuration

Reference documentation

  • The full structure of the Describe resource is documented here.

Deprecated syntax

You can specify the resource directly instead of using apiVersion and kind.

This is a deprecated syntax though and will be removed in a future version.

Clustered resources

When used with a clustered resource, the namespace is ignored and is not added to the corresponding kubectl command.

All namespaces

When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.

Usage examples

Describe pods

If a name is specified, Chainsaw will describe resources that have a name starting with the provided name in the test namespace (or in the cluster if it is a clustered-level resource).

Describe pods in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Describe - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Describe
 
Describing resources present in the cluster can help understand what happened and troubleshoot test failures.
 
Configuration
 
 
Reference documentation
 
     
  • The full structure of the Describe resource is documented here.
    •  
 
 
 
Deprecated syntax
 
You can specify the resource directly instead of using apiVersion and kind.
 
This is a deprecated syntax though and will be removed in a future version.
 
 
Clustered resources
 
When used with a clustered resource, the namespace is ignored and is not added to the corresponding kubectl command.
 
All namespaces
 
When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.
 
Usage examples
 
Describe pods
 
If a name is specified, Chainsaw will describe resources that have a name starting with the provided name in the test namespace (or in the cluster if it is a clustered-level resource).
 
 
Describe pods in the test namespace
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -102,4 +102,4 @@
         namespace: foo
         showEvents: false
     # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/collectors/events/index.html b/main/collectors/events/index.html index b17c0e9b7..35764e14b 100644 --- a/main/collectors/events/index.html +++ b/main/collectors/events/index.html @@ -1,4 +1,4 @@ - Events - Chainsaw
Skip to content

Events

Collecting namespace events can help understand what happened inside the cluster.

Configuration

Reference documentation

  • The full structure of the Events resource is documented here.

All namespaces

When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.

Usage examples

Single event

If a name is specified, Chainsaw will retrieve the specified event in the test namespace.

Collect event in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Events - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Events
 
Collecting namespace events can help understand what happened inside the cluster.
 
Configuration
 
 
Reference documentation
 
     
  • The full structure of the Events resource is documented here.
    •  
 
 
All namespaces
 
When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.
 
Usage examples
 
Single event
 
If a name is specified, Chainsaw will retrieve the specified event in the test namespace.
 
 
Collect event in the test namespace
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -128,4 +128,4 @@
     - events:
         format: yaml
     # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/collectors/get/index.html b/main/collectors/get/index.html index bdc334499..9ef2fed3a 100644 --- a/main/collectors/get/index.html +++ b/main/collectors/get/index.html @@ -1,4 +1,4 @@ - Get - Chainsaw
Skip to content

Get

The get collector is used to list and print resources in the cluster.

Configuration

Reference documentation

  • The full structure of the Get resource is documented here.

Deprecated syntax

You can specify the resource directly instead of using apiVersion and kind.

This is a deprecated syntax though and will be removed in a future version.

Clustered resources

When used with a clustered resource, the namespace is ignored and is not added to the corresponding kubectl command.

All namespaces

When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.

Usage examples

Single resource

If a name is specified, Chainsaw will retrieve the specified resource in the test namespace.

Get pod in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Get - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Get
 
The get collector is used to list and print resources in the cluster.
 
Configuration
 
 
Reference documentation
 
     
  • The full structure of the Get resource is documented here.
    •  
 
 
 
Deprecated syntax
 
You can specify the resource directly instead of using apiVersion and kind.
 
This is a deprecated syntax though and will be removed in a future version.
 
 
Clustered resources
 
When used with a clustered resource, the namespace is ignored and is not added to the corresponding kubectl command.
 
All namespaces
 
When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.
 
Usage examples
 
Single resource
 
If a name is specified, Chainsaw will retrieve the specified resource in the test namespace.
 
 
Get pod in the test namespace
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -159,4 +159,4 @@
         kind: Pod
         format: yaml
     # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/collectors/index.html b/main/collectors/index.html index ac621636d..547d5bfff 100644 --- a/main/collectors/index.html +++ b/main/collectors/index.html @@ -1 +1 @@ - Collectors - Chainsaw
Skip to content

Collectors

Purpose

The purpose of collectors is to collect certain information about the outcome of a step should it fail (in the case of catch) or at the end of the step (in the case of finally).

The ultimate goal of collectors is to gather information about the failure of a step and therefore help understand what caused it to fail.

A test step can have an arbitrary number of collectors.

Available collectors

Templating

All string fields in collectors support templating.

\ No newline at end of file + Collectors - Chainsaw
Skip to content

Collectors

Purpose

The purpose of collectors is to collect certain information about the outcome of a step should it fail (in the case of catch) or at the end of the step (in the case of finally).

The ultimate goal of collectors is to gather information about the failure of a step and therefore help understand what caused it to fail.

A test step can have an arbitrary number of collectors.

Available collectors

Templating

All string fields in collectors support templating.

\ No newline at end of file diff --git a/main/collectors/pod-logs/index.html b/main/collectors/pod-logs/index.html index 0eed396b6..f42325d1d 100644 --- a/main/collectors/pod-logs/index.html +++ b/main/collectors/pod-logs/index.html @@ -1,4 +1,4 @@ - Pod logs - Chainsaw
Skip to content

Pod logs

Collecting pod logs can help understand what happened inside one or more pods.

Configuration

Reference documentation

  • The full structure of the PodLogs resource is documented here.

All namespaces

When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.

Usage examples

Single pod

If a pod name is specified, Chainsaw will retrieve logs from this specific pod in the test namespace.

Collect pod logs in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Pod logs - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Pod logs
 
Collecting pod logs can help understand what happened inside one or more pods.
 
Configuration
 
 
Reference documentation
 
     
  • The full structure of the PodLogs resource is documented here.
    •  
 
 
All namespaces
 
When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.
 
Usage examples
 
Single pod
 
If a pod name is specified, Chainsaw will retrieve logs from this specific pod in the test namespace.
 
 
Collect pod logs in the test namespace
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -136,4 +136,4 @@
         namespace: foo
         container: nginx
     # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/commands/chainsaw/index.html b/main/commands/chainsaw/index.html index 108fe97cb..69e3d3da7 100644 --- a/main/commands/chainsaw/index.html +++ b/main/commands/chainsaw/index.html @@ -1,3 +1,3 @@ - Chainsaw - Chainsaw
Skip to content

Chainsaw

chainsaw

Stronger tool for e2e testing

chainsaw [flags]
+ Chainsaw - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw
 
chainsaw
 
Stronger tool for e2e testing
 
chainsaw [flags]
 
 
Options
 
  -h, --help   help for chainsaw
-
 
SEE ALSO
  
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

\ No newline at end of file diff --git a/main/commands/chainsaw_assert/index.html b/main/commands/chainsaw_assert/index.html index cdb86a437..6b1970ddb 100644 --- a/main/commands/chainsaw_assert/index.html +++ b/main/commands/chainsaw_assert/index.html @@ -1,4 +1,4 @@ - Chainsaw assert - Chainsaw
Skip to content

Chainsaw assert

chainsaw assert

Evaluate assertion

chainsaw assert [flags] [FILE]
+ Chainsaw assert - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw assert
 
chainsaw assert
 
Evaluate assertion
 
chainsaw assert [flags] [FILE]
 
 
Options
 
      --clustered                           Defines if the resource is clustered (only applies when resource is loaded from a file)
   -f, --file string                         Path to the file to assert or '-' to read from stdin
   -h, --help                                help for assert
@@ -25,4 +25,4 @@
       --no-color                            Removes output colors
   -r, --resource string                     Path to the file containing the resource
       --timeout duration                    The assert timeout to use (default 30s)
-
 
SEE ALSO
 
     
  • chainsaw - Stronger tool for e2e testing
    •  
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

  • chainsaw - Stronger tool for e2e testing
\ No newline at end of file diff --git a/main/commands/chainsaw_build/index.html b/main/commands/chainsaw_build/index.html index f1cffc7f2..00fa73f35 100644 --- a/main/commands/chainsaw_build/index.html +++ b/main/commands/chainsaw_build/index.html @@ -1,3 +1,3 @@ - Chainsaw build - Chainsaw
Skip to content

Chainsaw build

chainsaw build

Build commands

chainsaw build [flags]
+ Chainsaw build - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw build
 
chainsaw build
 
Build commands
 
chainsaw build [flags]
 
 
Options
 
  -h, --help   help for build
-
 
SEE ALSO
  
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

\ No newline at end of file diff --git a/main/commands/chainsaw_build_docs/index.html b/main/commands/chainsaw_build_docs/index.html index 059133422..975b902e2 100644 --- a/main/commands/chainsaw_build_docs/index.html +++ b/main/commands/chainsaw_build_docs/index.html @@ -1,7 +1,7 @@ - Chainsaw build docs - Chainsaw
Skip to content

Chainsaw build docs

chainsaw build docs

Build tests documentation

chainsaw build docs [flags]
+ Chainsaw build docs - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw build docs
 
chainsaw build docs
 
Build tests documentation
 
chainsaw build docs [flags]
 
 
Options
 
      --catalog string         Path to the built test catalog file
   -h, --help                   help for docs
       --readme-file string     Name of the built docs file (default "README.md")
       --test-dir stringArray   Directories containing test cases to run
       --test-file string       Name of the test file (default "chainsaw-test")
-
 
SEE ALSO
  
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

\ No newline at end of file diff --git a/main/commands/chainsaw_completion/index.html b/main/commands/chainsaw_completion/index.html index 24e6b0685..01d156dbb 100644 --- a/main/commands/chainsaw_completion/index.html +++ b/main/commands/chainsaw_completion/index.html @@ -1,2 +1,2 @@ - Chainsaw completion - Chainsaw
Skip to content

Chainsaw completion

chainsaw completion

Generate the autocompletion script for the specified shell

Synopsis

Generate the autocompletion script for chainsaw for the specified shell. See each sub-command's help for details on how to use the generated script.

Options

  -h, --help   help for completion
-

SEE ALSO

\ No newline at end of file + Chainsaw completion - Chainsaw
Skip to content

Chainsaw completion

chainsaw completion

Generate the autocompletion script for the specified shell

Synopsis

Generate the autocompletion script for chainsaw for the specified shell. See each sub-command's help for details on how to use the generated script.

Options

  -h, --help   help for completion
+

SEE ALSO

\ No newline at end of file diff --git a/main/commands/chainsaw_completion_bash/index.html b/main/commands/chainsaw_completion_bash/index.html index 2770fe60a..b92b9d5e3 100644 --- a/main/commands/chainsaw_completion_bash/index.html +++ b/main/commands/chainsaw_completion_bash/index.html @@ -1,7 +1,7 @@ - Chainsaw completion bash - Chainsaw
Skip to content

Chainsaw completion bash

chainsaw completion bash

Generate the autocompletion script for bash

Synopsis

Generate the autocompletion script for the bash shell.

This script depends on the 'bash-completion' package. If it is not installed already, you can install it via your OS's package manager.

To load completions in your current shell session:

source <(chainsaw completion bash)
+ Chainsaw completion bash - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw completion bash
 
chainsaw completion bash
 
Generate the autocompletion script for bash
 
Synopsis
 
Generate the autocompletion script for the bash shell.
 
This script depends on the 'bash-completion' package. If it is not installed already, you can install it via your OS's package manager.
 
To load completions in your current shell session:
 
source <(chainsaw completion bash)
 
 
To load completions for every new session, execute once:
 
Linux:
 
chainsaw completion bash > /etc/bash_completion.d/chainsaw
 
 
macOS:
 
chainsaw completion bash > $(brew --prefix)/etc/bash_completion.d/chainsaw
 
 
You will need to start a new shell for this setup to take effect.
 
chainsaw completion bash
 
 
Options
 
  -h, --help              help for bash
       --no-descriptions   disable completion descriptions
-
 
SEE ALSO
  
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

\ No newline at end of file diff --git a/main/commands/chainsaw_completion_fish/index.html b/main/commands/chainsaw_completion_fish/index.html index 10b41e0a0..a6a750299 100644 --- a/main/commands/chainsaw_completion_fish/index.html +++ b/main/commands/chainsaw_completion_fish/index.html @@ -1,6 +1,6 @@ - Chainsaw completion fish - Chainsaw
Skip to content

Chainsaw completion fish

chainsaw completion fish

Generate the autocompletion script for fish

Synopsis

Generate the autocompletion script for the fish shell.

To load completions in your current shell session:

chainsaw completion fish | source
+ Chainsaw completion fish - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw completion fish
 
chainsaw completion fish
 
Generate the autocompletion script for fish
 
Synopsis
 
Generate the autocompletion script for the fish shell.
 
To load completions in your current shell session:
 
chainsaw completion fish | source
 
 
To load completions for every new session, execute once:
 
chainsaw completion fish > ~/.config/fish/completions/chainsaw.fish
 
 
You will need to start a new shell for this setup to take effect.
 
chainsaw completion fish [flags]
 
 
Options
 
  -h, --help              help for fish
       --no-descriptions   disable completion descriptions
-
 
SEE ALSO
  
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

\ No newline at end of file diff --git a/main/commands/chainsaw_completion_powershell/index.html b/main/commands/chainsaw_completion_powershell/index.html index ae4128f7b..3ff18aa56 100644 --- a/main/commands/chainsaw_completion_powershell/index.html +++ b/main/commands/chainsaw_completion_powershell/index.html @@ -1,5 +1,5 @@ - Chainsaw completion powershell - Chainsaw
Skip to content

Chainsaw completion powershell

chainsaw completion powershell

Generate the autocompletion script for powershell

Synopsis

Generate the autocompletion script for powershell.

To load completions in your current shell session:

chainsaw completion powershell | Out-String | Invoke-Expression
+ Chainsaw completion powershell - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw completion powershell
 
chainsaw completion powershell
 
Generate the autocompletion script for powershell
 
Synopsis
 
Generate the autocompletion script for powershell.
 
To load completions in your current shell session:
 
chainsaw completion powershell | Out-String | Invoke-Expression
 
 
To load completions for every new session, add the output of the above command to your powershell profile.
 
chainsaw completion powershell [flags]
 
 
Options
 
  -h, --help              help for powershell
       --no-descriptions   disable completion descriptions
-
 
SEE ALSO
  
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

\ No newline at end of file diff --git a/main/commands/chainsaw_completion_zsh/index.html b/main/commands/chainsaw_completion_zsh/index.html index 0f92cff42..44357ad46 100644 --- a/main/commands/chainsaw_completion_zsh/index.html +++ b/main/commands/chainsaw_completion_zsh/index.html @@ -1,8 +1,8 @@ - Chainsaw completion zsh - Chainsaw
Skip to content

Chainsaw completion zsh

chainsaw completion zsh

Generate the autocompletion script for zsh

Synopsis

Generate the autocompletion script for the zsh shell.

If shell completion is not already enabled in your environment you will need to enable it. You can execute the following once:

echo "autoload -U compinit; compinit" >> ~/.zshrc
+ Chainsaw completion zsh - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw completion zsh
 
chainsaw completion zsh
 
Generate the autocompletion script for zsh
 
Synopsis
 
Generate the autocompletion script for the zsh shell.
 
If shell completion is not already enabled in your environment you will need to enable it. You can execute the following once:
 
echo "autoload -U compinit; compinit" >> ~/.zshrc
 
 
To load completions in your current shell session:
 
source <(chainsaw completion zsh)
 
 
To load completions for every new session, execute once:
 
Linux:
 
chainsaw completion zsh > "${fpath[1]}/_chainsaw"
 
 
macOS:
 
chainsaw completion zsh > $(brew --prefix)/share/zsh/site-functions/_chainsaw
 
 
You will need to start a new shell for this setup to take effect.
 
chainsaw completion zsh [flags]
 
 
Options
 
  -h, --help              help for zsh
       --no-descriptions   disable completion descriptions
-
 
SEE ALSO
  
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

\ No newline at end of file diff --git a/main/commands/chainsaw_create/index.html b/main/commands/chainsaw_create/index.html index 403138100..e7da646cf 100644 --- a/main/commands/chainsaw_create/index.html +++ b/main/commands/chainsaw_create/index.html @@ -1,3 +1,3 @@ - Chainsaw create - Chainsaw
Skip to content

Chainsaw create

chainsaw create

Create Chainsaw resources

chainsaw create [flags]
+ Chainsaw create - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw create
 
chainsaw create
 
Create Chainsaw resources
 
chainsaw create [flags]
 
 
Options
 
  -h, --help   help for create
-
 
SEE ALSO
  
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

\ No newline at end of file diff --git a/main/commands/chainsaw_create_test/index.html b/main/commands/chainsaw_create_test/index.html index 432a42748..65aa4d5dd 100644 --- a/main/commands/chainsaw_create_test/index.html +++ b/main/commands/chainsaw_create_test/index.html @@ -1,6 +1,6 @@ - Chainsaw create test - Chainsaw
Skip to content

Chainsaw create test

chainsaw create test

Create a Chainsaw test

chainsaw create test [flags]
+ Chainsaw create test - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw create test
 
chainsaw create test
 
Create a Chainsaw test
 
chainsaw create test [flags]
 
 
Options
 
      --description   If set, adds description when applicable (default true)
       --force         If set, existing test will be deleted if needed
   -h, --help          help for test
       --save          If set, created test will be saved
-
 
SEE ALSO
  
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

\ No newline at end of file diff --git a/main/commands/chainsaw_docs/index.html b/main/commands/chainsaw_docs/index.html index 1435c412b..2120e903d 100644 --- a/main/commands/chainsaw_docs/index.html +++ b/main/commands/chainsaw_docs/index.html @@ -1,6 +1,6 @@ - Chainsaw docs - Chainsaw
Skip to content

Chainsaw docs

chainsaw docs

Generate reference documentation

chainsaw docs [flags]
+ Chainsaw docs - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw docs
 
chainsaw docs
 
Generate reference documentation
 
chainsaw docs [flags]
 
 
Options
 
      --autogenTag      Determines if the generated docs should contain a timestamp (default true)
   -h, --help            help for docs
   -o, --output string   Output path (default ".")
       --website         Website version
-
 
SEE ALSO
 
     
  • chainsaw - Stronger tool for e2e testing
    •  
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

  • chainsaw - Stronger tool for e2e testing
\ No newline at end of file diff --git a/main/commands/chainsaw_export/index.html b/main/commands/chainsaw_export/index.html index 4cf64a4e9..0357573da 100644 --- a/main/commands/chainsaw_export/index.html +++ b/main/commands/chainsaw_export/index.html @@ -1,3 +1,3 @@ - Chainsaw export - Chainsaw
Skip to content

Chainsaw export

chainsaw export

Export commands

chainsaw export [flags]
+ Chainsaw export - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw export
 
chainsaw export
 
Export commands
 
chainsaw export [flags]
 
 
Options
 
  -h, --help   help for export
-
 
SEE ALSO
  
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

\ No newline at end of file diff --git a/main/commands/chainsaw_export_schemas/index.html b/main/commands/chainsaw_export_schemas/index.html index 4f0077705..449ca0306 100644 --- a/main/commands/chainsaw_export_schemas/index.html +++ b/main/commands/chainsaw_export_schemas/index.html @@ -1,3 +1,3 @@ - Chainsaw export schemas - Chainsaw
Skip to content

Chainsaw export schemas

chainsaw export schemas

Export JSON schemas

chainsaw export schemas [flags]
+ Chainsaw export schemas - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw export schemas
 
chainsaw export schemas
 
Export JSON schemas
 
chainsaw export schemas [flags]
 
 
Options
 
  -h, --help   help for schemas
-
 
SEE ALSO
  
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

\ No newline at end of file diff --git a/main/commands/chainsaw_lint/index.html b/main/commands/chainsaw_lint/index.html index 17a7bf840..7dc4c9133 100644 --- a/main/commands/chainsaw_lint/index.html +++ b/main/commands/chainsaw_lint/index.html @@ -1,4 +1,4 @@ - Chainsaw lint - Chainsaw
Skip to content

Chainsaw lint

chainsaw lint

Lint a file or read from standard input

Synopsis

Use chainsaw lint to lint a specific file or read from standard input for either test or configuration.

chainsaw lint [test|configuration] [flags]
+ Chainsaw lint - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw lint
 
chainsaw lint
 
Lint a file or read from standard input
 
Synopsis
 
Use chainsaw lint to lint a specific file or read from standard input for either test or configuration.
 
chainsaw lint [test|configuration] [flags]
 
 
Options
 
  -f, --file string   Specify the file to lint or '-' for standard input
   -h, --help          help for lint
-
 
SEE ALSO
 
     
  • chainsaw - Stronger tool for e2e testing
    •  
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

  • chainsaw - Stronger tool for e2e testing
\ No newline at end of file diff --git a/main/commands/chainsaw_migrate/index.html b/main/commands/chainsaw_migrate/index.html index 6dd50eebf..dec56e1c3 100644 --- a/main/commands/chainsaw_migrate/index.html +++ b/main/commands/chainsaw_migrate/index.html @@ -1,3 +1,3 @@ - Chainsaw migrate - Chainsaw
Skip to content

Chainsaw migrate

chainsaw migrate

Migrate resources to Chainsaw

chainsaw migrate [flags]
+ Chainsaw migrate - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw migrate
 
chainsaw migrate
 
Migrate resources to Chainsaw
 
chainsaw migrate [flags]
 
 
Options
 
  -h, --help   help for migrate
-
 
SEE ALSO
  
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

\ No newline at end of file diff --git a/main/commands/chainsaw_migrate_kuttl/index.html b/main/commands/chainsaw_migrate_kuttl/index.html index b6157aca3..fb77b4de3 100644 --- a/main/commands/chainsaw_migrate_kuttl/index.html +++ b/main/commands/chainsaw_migrate_kuttl/index.html @@ -1,3 +1,3 @@ - Chainsaw migrate kuttl - Chainsaw
Skip to content

Chainsaw migrate kuttl

chainsaw migrate kuttl

Migrate KUTTL resources to Chainsaw

chainsaw migrate kuttl [flags]
+ Chainsaw migrate kuttl - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
  
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

\ No newline at end of file diff --git a/main/commands/chainsaw_migrate_kuttl_config/index.html b/main/commands/chainsaw_migrate_kuttl_config/index.html index fa0aff2d6..1892c4b43 100644 --- a/main/commands/chainsaw_migrate_kuttl_config/index.html +++ b/main/commands/chainsaw_migrate_kuttl_config/index.html @@ -1,5 +1,5 @@ - Chainsaw migrate kuttl config - Chainsaw
Skip to content

Chainsaw migrate kuttl config

chainsaw migrate kuttl config

Migrate KUTTL config to Chainsaw

chainsaw migrate kuttl config [flags]
+ Chainsaw migrate kuttl config - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw migrate kuttl config
 
chainsaw migrate kuttl config
 
Migrate KUTTL config to Chainsaw
 
chainsaw migrate kuttl config [flags]
 
 
Options
 
      --cleanup   If set, delete converted files
   -h, --help      help for config
       --save      If set, converted files will be saved
-
 
SEE ALSO
  
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

\ No newline at end of file diff --git a/main/commands/chainsaw_migrate_kuttl_tests/index.html b/main/commands/chainsaw_migrate_kuttl_tests/index.html index 94341ac52..9954d794f 100644 --- a/main/commands/chainsaw_migrate_kuttl_tests/index.html +++ b/main/commands/chainsaw_migrate_kuttl_tests/index.html @@ -1,5 +1,5 @@ - Chainsaw migrate kuttl tests - Chainsaw
Skip to content

Chainsaw migrate kuttl tests

chainsaw migrate kuttl tests

Migrate KUTTL tests to Chainsaw

chainsaw migrate kuttl tests [flags]
+ Chainsaw migrate kuttl tests - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw migrate kuttl tests
 
chainsaw migrate kuttl tests
 
Migrate KUTTL tests to Chainsaw
 
chainsaw migrate kuttl tests [flags]
 
 
Options
 
      --cleanup   If set, delete converted files
   -h, --help      help for tests
       --save      If set, converted files will be saved
-
 
SEE ALSO
  
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

\ No newline at end of file diff --git a/main/commands/chainsaw_test/index.html b/main/commands/chainsaw_test/index.html index a441f6ebd..262a910c3 100644 --- a/main/commands/chainsaw_test/index.html +++ b/main/commands/chainsaw_test/index.html @@ -1,4 +1,4 @@ - Chainsaw test - Chainsaw
Skip to content

Chainsaw test

chainsaw test

Run tests

chainsaw test [flags]... [test directories]...
+ Chainsaw test - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw test
 
chainsaw test
 
Run tests
 
chainsaw test [flags]... [test directories]...
 
 
Options
 
      --apply-timeout duration                    The apply timeout to use as default for configuration (default 5s)
       --assert-timeout duration                   The assert timeout to use as default for configuration (default 30s)
       --cleanup-delay duration                    Adds a delay between the time a test ends and the time cleanup starts
@@ -48,4 +48,4 @@
       --test-dir strings                          Directories containing test cases to run
       --test-file string                          Name of the test file (default "chainsaw-test")
       --values strings                            Values passed to the tests
-
 
SEE ALSO
 
     
  • chainsaw - Stronger tool for e2e testing
    •  
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

  • chainsaw - Stronger tool for e2e testing
\ No newline at end of file diff --git a/main/commands/chainsaw_version/index.html b/main/commands/chainsaw_version/index.html index 9a6293055..4af71dca0 100644 --- a/main/commands/chainsaw_version/index.html +++ b/main/commands/chainsaw_version/index.html @@ -1,3 +1,3 @@ - Chainsaw version - Chainsaw
Skip to content

Chainsaw version

chainsaw version

Print the version informations

chainsaw version [flags]
+ Chainsaw version - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Chainsaw version
 
chainsaw version
 
Print the version informations
 
chainsaw version [flags]
 
 
Options
 
  -h, --help   help for version
-
 
SEE ALSO
 
     
  • chainsaw - Stronger tool for e2e testing
    •  
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

SEE ALSO

  • chainsaw - Stronger tool for e2e testing
\ No newline at end of file diff --git a/main/community/index.html b/main/community/index.html index b0c2b588a..444132a1b 100644 --- a/main/community/index.html +++ b/main/community/index.html @@ -1 +1 @@ - Community - Chainsaw
Skip to content

Community

Chainsaw has a growing community and we would definitely love to see you join and contribute.

Everyone is welcome to make suggestions, report bugs, open feature requests, contribute code or docs, participate in discussions, write blogs or anything that can benefit the project.

Chainsaw is built and maintained under the Kyverno umbrella but decisions are

Community driven

Everyone's voice matters

Slack channel

Join our slack channel #kyverno-chainsaw to meet with users, contributors and maintainers.

Community Meetings

To attend our community meetings, join the Chainsaw group. You will then be sent a meeting invite and will have access to the agenda and meeting notes. Any member may suggest topics for discussion.

This is a public, weekly for Kyverno-Chainsaw maintainers to make announcements and provide project updates, and request input and feedback. This forum allows community members to raise agenda items of any sort, including but not limited to any PRs or issues on which they are working.

Weekly every Thursday at 2:00 PM UTC

RoadMap

For detailed information on our planned features and upcoming updates, please view our Roadmap.

Contributing

Please read the contributing guide for details around:

  1. Code of Conduct
  2. Code Culture
  3. Details on how to contribute

Adopters

If you are using Chainsaw and want to share it publicly we always appreciate a bit of support. Pull requests to the ADOPTERS LIST will put a smile on our faces 😄

\ No newline at end of file + Community - Chainsaw
Skip to content

Community

Chainsaw has a growing community and we would definitely love to see you join and contribute.

Everyone is welcome to make suggestions, report bugs, open feature requests, contribute code or docs, participate in discussions, write blogs or anything that can benefit the project.

Chainsaw is built and maintained under the Kyverno umbrella but decisions are

Community driven

Everyone's voice matters

Slack channel

Join our slack channel #kyverno-chainsaw to meet with users, contributors and maintainers.

Community Meetings

To attend our community meetings, join the Chainsaw group. You will then be sent a meeting invite and will have access to the agenda and meeting notes. Any member may suggest topics for discussion.

This is a public, weekly for Kyverno-Chainsaw maintainers to make announcements and provide project updates, and request input and feedback. This forum allows community members to raise agenda items of any sort, including but not limited to any PRs or issues on which they are working.

Weekly every Thursday at 2:00 PM UTC

RoadMap

For detailed information on our planned features and upcoming updates, please view our Roadmap.

Contributing

Please read the contributing guide for details around:

  1. Code of Conduct
  2. Code Culture
  3. Details on how to contribute

Adopters

If you are using Chainsaw and want to share it publicly we always appreciate a bit of support. Pull requests to the ADOPTERS LIST will put a smile on our faces 😄

\ No newline at end of file diff --git a/main/configuration/cleanup-delay/index.html b/main/configuration/cleanup-delay/index.html index 111f67841..f45b36df9 100644 --- a/main/configuration/cleanup-delay/index.html +++ b/main/configuration/cleanup-delay/index.html @@ -1,4 +1,4 @@ - Delay before cleanup - Chainsaw
Skip to content

Delay before cleanup

At the end of each test, Chainsaw will delete the resources it created during the test.

When testing operators, it can be useful to wait a little bit before starting the cleanup process to make sure the operator/controller has the necessary time to update the internal state.

For this reason, Chainsaw provides the delayBeforeCleanup configuration option and the corresponding --delay-before-cleanup flag.

Configuration

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Delay before cleanup - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Delay before cleanup
 
At the end of each test, Chainsaw will delete the resources it created during the test.
 
When testing operators, it can be useful to wait a little bit before starting the cleanup process to make sure the operator/controller has the necessary time to update the internal state.
 
For this reason, Chainsaw provides the delayBeforeCleanup configuration option and the corresponding --delay-before-cleanup flag.
 
Configuration
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Configuration
 metadata:
   name: custom-config
@@ -7,4 +7,4 @@
   delayBeforeCleanup: 5s
   # ...
 
 
Flag
 
chainsaw test --delay-before-cleanup 5s ...
-
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/configuration/file/index.html b/main/configuration/file/index.html index cce34dc7a..e9c61272d 100644 --- a/main/configuration/file/index.html +++ b/main/configuration/file/index.html @@ -1,4 +1,4 @@ - Configuration file - Chainsaw
Skip to content

Configuration file

Chainsaw is described as a Stronger tool for e2e testing.

With its versatile configuration options, you can customize the testing process to fit your needs.

Chainsaw prioritizes its configuration in the following order:

  1. User-specified configuration

    If you explicitly provide a configuration file using a command-line flag

  2. Default configuration file

    If no configuration is specified, Chainsaw will look for a default file named .chainsaw.yaml in the current working directory

  3. Internal default configuration

    In the absence of both the above, Chainsaw will use a default configuration file embedded in the Chainsaw binary

Example

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Configuration file - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Configuration file
 
Chainsaw is described as a Stronger tool for e2e testing.
 
With its versatile configuration options, you can customize the testing process to fit your needs.
 
Chainsaw prioritizes its configuration in the following order:
 
     
  1.  
    User-specified configuration
     
    If you explicitly provide a configuration file using a command-line flag
     
    2.  
  2.  
    Default configuration file
     
    If no configuration is specified, Chainsaw will look for a default file named .chainsaw.yaml in the current working directory
     
    3.  
  3.  
    Internal default configuration
     
    In the absence of both the above, Chainsaw will use a default configuration file embedded in the Chainsaw binary
     
    4.  
 
Example
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Configuration
 metadata:
   name: custom-config
@@ -20,4 +20,4 @@
 metadata:
   name: default
 spec: {}
-
 
Reference documentation
 
See Configuration API reference for more details.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Reference documentation

See Configuration API reference for more details.

\ No newline at end of file diff --git a/main/configuration/flags/index.html b/main/configuration/flags/index.html index 280cd21ff..5f055db2c 100644 --- a/main/configuration/flags/index.html +++ b/main/configuration/flags/index.html @@ -1,4 +1,4 @@ - Command line flags - Chainsaw
Skip to content

Command line flags

Even after a configuration file is loaded, you can override specific settings using command-line flags.

Example

chainsaw test                         \
+ Command line flags - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Command line flags
 
Even after a configuration file is loaded, you can override specific settings using command-line flags.
 
Example
 
chainsaw test                         \
   path/to/test/dir                    \
   --config path/to/your/config.yaml   \
   --assert-timeout 45s                \
@@ -7,4 +7,4 @@
   --parallel 4                        \
   ...
 
 
In this example, Chainsaw will load a configuration file but the timeout configuration and other settings will be overridden by the values set in the flags, regardless of the value in the loaded configuration file.
 
Usage
 
The command below will run tests using the configuration from my-config.yaml, taking tests from /path/to/tests, and running a maximum of 10 tests simultaneously.
 
chainsaw test /path/to/tests --config my-config.yaml --parallel 10
-
 
Reference documentation
 
See Chainsaw test command reference for more details.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Reference documentation

See Chainsaw test command reference for more details.

\ No newline at end of file diff --git a/main/configuration/grace/index.html b/main/configuration/grace/index.html index 904d9e237..ce56e78a8 100644 --- a/main/configuration/grace/index.html +++ b/main/configuration/grace/index.html @@ -1,4 +1,4 @@ - Termination graceful period - Chainsaw
Skip to content

Termination graceful period

Some Kubernetes resources can take time before being stopped. For example, deleting a Pod can take time if the underlying container doesn't quit quickly enough.

For this reason, Chainsaw provides the forceTerminationGracePeriod configuration option and the corresponding --force-termination-grace-period flag. If set, Chainsaw will override the terminationGracePeriodSeconds when working with the following resource kinds:

  • Pod
  • Deployment
  • StatefulSet
  • DaemonSet
  • Job
  • CronJob

Configuration

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Termination graceful period - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Termination graceful period
 
Some Kubernetes resources can take time before being stopped. For example, deleting a Pod can take time if the underlying container doesn't quit quickly enough.
 
For this reason, Chainsaw provides the forceTerminationGracePeriod configuration option and the corresponding --force-termination-grace-period flag. If set, Chainsaw will override the terminationGracePeriodSeconds when working with the following resource kinds:
 
     
  • Pod
    •  
  • Deployment
    •  
  • StatefulSet
    •  
  • DaemonSet
    •  
  • Job
    •  
  • CronJob
    •  
 
Configuration
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Configuration
 metadata:
   name: custom-config
@@ -7,4 +7,4 @@
   forceTerminationGracePeriod: 5s
   # ...
 
 
Flag
 
chainsaw test --force-termination-grace-period 5s ...
-
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/configuration/index.html b/main/configuration/index.html index 6117598b7..8cb73a474 100644 --- a/main/configuration/index.html +++ b/main/configuration/index.html @@ -1 +1 @@ - Configuring Chainsaw - Chainsaw
Skip to content

Configuring Chainsaw

Chainsaw is a comprehensive tool designed to facilitate end-to-end testing in Kubernetes.

This documentation will focus on providing a breakdown of its configuration structure and how to use it.

Chainsaw can be configured in two different and complementary ways:

Precedence

If both are specified, command-line flags will take precedence over configuration coming from a configuration file.

Specific configuration options

Please pay attention to the configuration options below, they may or may not be relevant in your case but can be useful in certain cases:

\ No newline at end of file + Configuring Chainsaw - Chainsaw
Skip to content

Configuring Chainsaw

Chainsaw is a comprehensive tool designed to facilitate end-to-end testing in Kubernetes.

This documentation will focus on providing a breakdown of its configuration structure and how to use it.

Chainsaw can be configured in two different and complementary ways:

Precedence

If both are specified, command-line flags will take precedence over configuration coming from a configuration file.

Specific configuration options

Please pay attention to the configuration options below, they may or may not be relevant in your case but can be useful in certain cases:

\ No newline at end of file diff --git a/main/configuration/multi-cluster/index.html b/main/configuration/multi-cluster/index.html index d43c5de42..d9837d9b4 100644 --- a/main/configuration/multi-cluster/index.html +++ b/main/configuration/multi-cluster/index.html @@ -1,4 +1,4 @@ - Multi cluster - Chainsaw
Skip to content

Multi cluster

Chainsaw supports testing against multiple clusters.

To use a specific cluster in a test (or test step) you will need to register the cluster either using the config file or using command line flags.

Since v0.2.1 you can also register clusters dynamically at the test, step and operation levels. This is particularly useful when a cluster is created in a test step and used in subsequent steps.

Configuration

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Multi cluster - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Multi cluster
 
Chainsaw supports testing against multiple clusters.
 
To use a specific cluster in a test (or test step) you will need to register the cluster either using the config file or using command line flags.
 
Since v0.2.1 you can also register clusters dynamically at the test, step and operation levels. This is particularly useful when a cluster is created in a test step and used in subsequent steps.
 
Configuration
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Configuration
 metadata:
   name: custom-config
@@ -150,4 +150,4 @@
         # for the specified cluster
         content: kubectl get pods
     # ...
-
\ No newline at end of file +
\ No newline at end of file diff --git a/main/configuration/no-cluster/index.html b/main/configuration/no-cluster/index.html index 82a552dc1..c9213c591 100644 --- a/main/configuration/no-cluster/index.html +++ b/main/configuration/no-cluster/index.html @@ -1,3 +1,3 @@ - Running without a cluster - Chainsaw
Skip to content

Running without a cluster

Chainsaw can be run without connection to a Kubernetes cluster. In this case, chainsaw will not try to create an ephemeral namespace and all operations requiring a Kubernetes cluster will fail.

To run chainsaw in this mode pass the --no-cluster flag.

Example

# run chainsaw without connection to a Kubernetes cluster
+ Running without a cluster - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Running without a cluster
 
Chainsaw can be run without connection to a Kubernetes cluster. In this case, chainsaw will not try to create an ephemeral namespace and all operations requiring a Kubernetes cluster will fail.
 
To run chainsaw in this mode pass the --no-cluster flag.
 
Example
 
# run chainsaw without connection to a Kubernetes cluster
 chainsaw test --no-cluster
-
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/configuration/reports/index.html b/main/configuration/reports/index.html index b531ab179..8538a6c5e 100644 --- a/main/configuration/reports/index.html +++ b/main/configuration/reports/index.html @@ -1,4 +1,4 @@ - Reports - Chainsaw
Skip to content

Reports

Chainsaw can generate JUnit reports in XML or JSON format.

To produce a test report, configure the report format, report path and report name in the configuration or using CLI flags.

Configuration

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Reports - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Reports
 
Chainsaw can generate JUnit reports in XML or JSON format.
 
To produce a test report, configure the report format, report path and report name in the configuration or using CLI flags.
 
Configuration
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Configuration
 metadata:
   name: custom-config
@@ -9,4 +9,4 @@
   reportPath: /home/chainsaw
   # ...
 
 
Flag
 
chainsaw test --report-format JSON --report-name chainsaw-report --report-path /path/to/save/report ...
-
 
 
Note: The reportPath can be specified as either a relative or an absolute path.
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Note: The reportPath can be specified as either a relative or an absolute path.

\ No newline at end of file diff --git a/main/configuration/selector/index.html b/main/configuration/selector/index.html index 5c91e30cd..3145912e3 100644 --- a/main/configuration/selector/index.html +++ b/main/configuration/selector/index.html @@ -1,4 +1,4 @@ - Label selectors - Chainsaw
Skip to content

Label selectors

Chainsaw can filter the tests to run using label selectors.

You can pass label selectors using the --selector flag when invoking the chainsaw test command.

Example

Given the test below:

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Label selectors - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Label selectors
 
Chainsaw can filter the tests to run using label selectors.
 
You can pass label selectors using the --selector flag when invoking the chainsaw test command.
 
Example
 
Given the test below:
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: basic
@@ -7,4 +7,4 @@
 spec:
   # ...
 
 
Invoking Chainsaw with the command below will take the test above into account:
 
chainsaw test --selector foo=bar
-
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/configuration/templating/index.html b/main/configuration/templating/index.html index 53d08a4d2..3f8fa9205 100644 --- a/main/configuration/templating/index.html +++ b/main/configuration/templating/index.html @@ -1,4 +1,4 @@ - Resource templating - Chainsaw
Skip to content

Resource templating

Experimental status

This is an experimental feature, and implementation could change slightly in the next versions.

Info

Templating was disabled by default in v0.1.* but is now enabled by default since v0.2.1.

Chainsaw can apply transformations to the resources before they are processed by the operation.

This is useful when a resource needs some runtime configuration.

Templating must be enabled at the configuration, test, step, or operation level for the templating process to kick in. Alternatively, templating can be enabled using the --template flag when invoking chainsaw from the command line.

Note

Unlike assertion trees, templating can only be specified in leave nodes of the YAML tree.

Example

The test below will create, assert, and delete a ConfigMap with a dynamic name configured at runtime using the $namespace binding.

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Resource templating - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Resource templating
 
 
Experimental status
 
This is an experimental feature, and implementation could change slightly in the next versions.
 
 
 
Info
 
Templating was disabled by default in v0.1.* but is now enabled by default since v0.2.1.
 
 
Chainsaw can apply transformations to the resources before they are processed by the operation.
 
This is useful when a resource needs some runtime configuration.
 
Templating must be enabled at the configuration, test, step, or operation level for the templating process to kick in. Alternatively, templating can be enabled using the --template flag when invoking chainsaw from the command line.
 
 
Note
 
Unlike assertion trees, templating can only be specified in leave nodes of the YAML tree.
 
 
Example
 
The test below will create, assert, and delete a ConfigMap with a dynamic name configured at runtime using the $namespace binding.
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: values
@@ -22,4 +22,4 @@
           apiVersion: v1
           kind: ConfigMap
           name: ($namespace)
-
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/configuration/timeouts/index.html b/main/configuration/timeouts/index.html index bbe344dc0..85c967965 100644 --- a/main/configuration/timeouts/index.html +++ b/main/configuration/timeouts/index.html @@ -1,4 +1,4 @@ - Timeouts - Chainsaw
Skip to content

Timeouts

Timeouts in Chainsaw are specified per type of operation. This is required because the timeout varies greatly depending on the nature of an operation.

For example, applying a manifest in a cluster is expected to be reasonably fast, while validating a resource can be a long operation.

Chainsaw supports separately configuring the timeouts below:

  • Apply

    When Chainsaw applies manifests in a cluster

  • Assert

    When Chainsaw validates resources in a cluster

  • Cleanup

    When Chainsaw removes resources from a cluster created for a test

  • Delete

    When Chainsaw deletes resources from a cluster

  • Error

    When Chainsaw validates resources in a cluster

  • Exec

    When Chainsaw executes arbitrary commands or scripts

Overriding timeouts

Each timeout can be overridden at the test level, test step level, or individual operation level.

Timeouts defined in the Configuration are used in operations when not overridden.

Configuration

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Timeouts - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Timeouts
 
Timeouts in Chainsaw are specified per type of operation. This is required because the timeout varies greatly depending on the nature of an operation.
 
For example, applying a manifest in a cluster is expected to be reasonably fast, while validating a resource can be a long operation.
 
Chainsaw supports separately configuring the timeouts below:
 
     
  •  
    Apply
     
    When Chainsaw applies manifests in a cluster
     
    •  
  •  
    Assert
     
    When Chainsaw validates resources in a cluster
     
    •  
  •  
    Cleanup
     
    When Chainsaw removes resources from a cluster created for a test
     
    •  
  •  
    Delete
     
    When Chainsaw deletes resources from a cluster
     
    •  
  •  
    Error
     
    When Chainsaw validates resources in a cluster
     
    •  
  •  
    Exec
     
    When Chainsaw executes arbitrary commands or scripts
     
    •  
 
 
Overriding timeouts
 
Each timeout can be overridden at the test level, test step level, or individual operation level.
 
Timeouts defined in the Configuration are used in operations when not overridden.
 
 
Configuration
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Configuration
 metadata:
   name: custom-config
@@ -20,4 +20,4 @@
     --error-timeout 45s             \
     --exec-timeout 45s              \
     ...
-
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/configuration/values/index.html b/main/configuration/values/index.html index 1ca5b8911..4c33f214d 100644 --- a/main/configuration/values/index.html +++ b/main/configuration/values/index.html @@ -1,4 +1,4 @@ - Passing data to tests - Chainsaw
Skip to content

Passing data to tests

Chainsaw can pass arbitrary values when running tests using the --values flag. Values will be available to tests under the $values binding.

This is useful when a test needs to be configured externally.

Example

The test below expects the $value.foo to be provided when chainsaw is invoked.

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Passing data to tests - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Passing data to tests
 
Chainsaw can pass arbitrary values when running tests using the --values flag. Values will be available to tests under the $values binding.
 
This is useful when a test needs to be configured externally.
 
Example
 
The test below expects the $value.foo to be provided when chainsaw is invoked.
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: values
@@ -19,4 +19,4 @@
 chainsaw test --values - <<EOF
 foo: bar
 EOF
-
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/examples/array-assertions/index.html b/main/examples/array-assertions/index.html index 173f34f59..a1cab3025 100644 --- a/main/examples/array-assertions/index.html +++ b/main/examples/array-assertions/index.html @@ -1,4 +1,4 @@ - Array assertions - Chainsaw
Skip to content

Array assertions

This example demonstrates how to perform complex assertions on arrays.

Setup

See Setup docs

Manifests

resources.yaml

apiVersion: v1
+ Array assertions - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Array assertions
 
This example demonstrates how to perform complex assertions on arrays.
 
Setup
 
See Setup docs
 
Manifests
 
resources.yaml
 
apiVersion: v1
 kind: Pod
 metadata:
   name: example
@@ -48,4 +48,4 @@
     - assert:
         file: assertions.yaml
 
 
Execute
 
chainsaw test
-
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/examples/index.html b/main/examples/index.html index 6146032af..9a71653dd 100644 --- a/main/examples/index.html +++ b/main/examples/index.html @@ -1,6 +1,6 @@ - Examples - Chainsaw
Skip to content

Examples

Setup

To use Chainsaw you will need a Kubernetes cluster, Chainsaw won't create one for you.

In these examples, we will use kind but feel free to use the tool of your choice.

Not a cluster management tool

We consider this is not the responsibility of Chainsaw to manage clusters.

There are plenty of solutions to create and manage local clusters that will do that better than Chainsaw.

Create a kind cluster

Please refer to the kind install docs to install it locally.

Once kind creating a local cluster is as simple as running:

# define kind image
+ Examples - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Examples
 
Setup
 
To use Chainsaw you will need a Kubernetes cluster, Chainsaw won't create one for you.
 
In these examples, we will use kind but feel free to use the tool of your choice.
 
 
Not a cluster management tool
 
We consider this is not the responsibility of Chainsaw to manage clusters.
 
There are plenty of solutions to create and manage local clusters that will do that better than Chainsaw.
 
 
Create a kind cluster
 
Please refer to the kind install docs to install it locally.
 
Once kind creating a local cluster is as simple as running:
 
# define kind image
 export KIND_IMAGE="kindest/node:v1.28.0"
 
 # create cluster
 kind create cluster --image $KIND_IMAGE
-
 
Install Chainsaw
 
Refer to install docs to install Chainsaw.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Install Chainsaw

Refer to install docs to install Chainsaw.

\ No newline at end of file diff --git a/main/gh-action/index.html b/main/gh-action/index.html index 449b2eab9..be0ea8d1c 100644 --- a/main/gh-action/index.html +++ b/main/gh-action/index.html @@ -1,4 +1,4 @@ - GitHub action - Chainsaw
Skip to content

GitHub action

A GitHub action is available to easily install Chainsaw in your workflows.

The GitHub action is available at kyverno/action-install-chainsaw or in the marketplace.

Usage

This action currently supports GitHub-provided Linux, macOS and Windows runners (self-hosted runners may not work).

Add the following entry to your Github workflow YAML file:

uses: kyverno/action-install-chainsaw@v0.1.0
+ GitHub action - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
GitHub action
 
A GitHub action is available to easily install Chainsaw in your workflows.
 
The GitHub action is available at kyverno/action-install-chainsaw or in the marketplace.
 
Usage
 
This action currently supports GitHub-provided Linux, macOS and Windows runners (self-hosted runners may not work).
 
Add the following entry to your Github workflow YAML file:
 
uses: kyverno/action-install-chainsaw@v0.1.0
 with:
   release: v0.1.0 # optional
 
 
Example using a pinned version:
 
jobs:
@@ -61,4 +61,4 @@
           release: main
       - name: Check install
         run: chainsaw version
-
 
Optional Inputs
 
The following optional inputs:
 
  
 Input Description 
   
 release chainsaw version to use instead of the default. 
 
 install-dir directory to place the chainsaw binary into instead of the default ($HOME/.chainsaw). 
 
 use-sudo set to true if install-dir location requires sudo privs. Defaults to false. 
 
 verify set to true to enable cosign verification of the downloaded archive. 
  
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Optional Inputs

The following optional inputs:

Input Description
release chainsaw version to use instead of the default.
install-dir directory to place the chainsaw binary into instead of the default ($HOME/.chainsaw).
use-sudo set to true if install-dir location requires sudo privs. Defaults to false.
verify set to true to enable cosign verification of the downloaded archive.
\ No newline at end of file diff --git a/main/index.html b/main/index.html index f8a291190..b8b4a9aef 100644 --- a/main/index.html +++ b/main/index.html @@ -190,4 +190,4 @@ min-height: 130px; background-color: var(--md-primary-bg-color); } -

Kyverno Chainsaw

Kyverno end to end testing tool !

  • Used internally to continuously test Kyverno
  • Entirely low code and declarative
  • Provides an advanced and flexible assertion model
  • Run it locally or in your CI/CD pipelines
  • ... and much more!
Get started Learn more

Everything you would expect

  • Easy to install

    Install locally using a package manager like brew or nix, or simply download the binary from one of our releases. If you want to run using a Docker image, we have that too.

  • Easy to use

    Write tests in minutes, not hours. All it takes is a YAML file to define the steps of a test. Chainsaw will do the rest, no need to learn a programing language or write a single line of code!

  • Comprehensive reporting

    Understand and diagnose failures easily, thanks to a comprehensive output showing precisely what failed and why. Generate JUnit compatible reports to integrate with other test reporting tools.

  • Resource templating

    Kubernetes is all about resouces and tests need to work with resources. Chainsaw has built-in support for bindings, operation outputs and resource templating to describe complex test scenarios.

  • Stay focused

    Focus on the software you are building, write test scenarios using YAML and let Chainsaw tell you what passes or not. Integrate in your CI pipeline to prevent regressions and release with better confidence.

  • Multi cluster

    Native support for tests involving multiple clusters, either static or dynamically created ones, make Chainsaw an excellent tool for testing highly complex environments and architectures.

Widely adopted

Running all end to end tests for both Kyverno and the policies catalog.

Chainsaw replaced and improved upon our bash test framework for testing the RbacDefinition CRD.

Chainsaw cut down on hacks, improved code-reuse and enhanced test debugging.

Chainsaw enabled easier e2e testing and CI debugging after replacing kuttl.

Chainsaw helped a lot for declarative assertion of Redis Cluster state against various e2e test.

Chainsaw replaced Kuttl, and helped us get rid of many unreadable bash scripts.

Chainsaw cranked up the tempo, making our e2e tests dance to a rhythm of reliability and efficiency.

Chainsaw replaced Kuttl and made our e2e tests much more readable and easier to debug.

Chainsaw replaced Kuttl and made our e2e tests much more readable and easier to debug.

Discover Chainsaw in video

Watch this excellent review video to discover the unique power of Chainsaw.

\ No newline at end of file +

Kyverno Chainsaw

Kyverno end to end testing tool !

  • Used internally to continuously test Kyverno
  • Entirely low code and declarative
  • Provides an advanced and flexible assertion model
  • Run it locally or in your CI/CD pipelines
  • ... and much more!
Get started Learn more

Everything you would expect

  • Easy to install

    Install locally using a package manager like brew or nix, or simply download the binary from one of our releases. If you want to run using a Docker image, we have that too.

  • Easy to use

    Write tests in minutes, not hours. All it takes is a YAML file to define the steps of a test. Chainsaw will do the rest, no need to learn a programing language or write a single line of code!

  • Comprehensive reporting

    Understand and diagnose failures easily, thanks to a comprehensive output showing precisely what failed and why. Generate JUnit compatible reports to integrate with other test reporting tools.

  • Resource templating

    Kubernetes is all about resouces and tests need to work with resources. Chainsaw has built-in support for bindings, operation outputs and resource templating to describe complex test scenarios.

  • Stay focused

    Focus on the software you are building, write test scenarios using YAML and let Chainsaw tell you what passes or not. Integrate in your CI pipeline to prevent regressions and release with better confidence.

  • Multi cluster

    Native support for tests involving multiple clusters, either static or dynamically created ones, make Chainsaw an excellent tool for testing highly complex environments and architectures.

Widely adopted

Running all end to end tests for both Kyverno and the policies catalog.

Chainsaw replaced and improved upon our bash test framework for testing the RbacDefinition CRD.

Chainsaw cut down on hacks, improved code-reuse and enhanced test debugging.

Chainsaw enabled easier e2e testing and CI debugging after replacing kuttl.

Chainsaw helped a lot for declarative assertion of Redis Cluster state against various e2e test.

Chainsaw replaced Kuttl, and helped us get rid of many unreadable bash scripts.

Chainsaw cranked up the tempo, making our e2e tests dance to a rhythm of reliability and efficiency.

Chainsaw replaced Kuttl and made our e2e tests much more readable and easier to debug.

Chainsaw replaced Kuttl and made our e2e tests much more readable and easier to debug.

Discover Chainsaw in video

Watch this excellent review video to discover the unique power of Chainsaw.

\ No newline at end of file diff --git a/main/jp/functions/index.html b/main/jp/functions/index.html index dba04070f..2b479926e 100644 --- a/main/jp/functions/index.html +++ b/main/jp/functions/index.html @@ -1 +1 @@ - Functions - Chainsaw
Skip to content

Functions

Experimental functions

Experimental functions are denoted by the x_ prefix.

These are functions that are subject to signature change in a future version.

built-in functions

Name Signature
abs abs(number)
avg avg(array[number])
ceil ceil(number)
contains contains(array|string, any)
ends_with ends_with(string, string)
find_first find_first(string, string, number, number)
find_last find_last(string, string, number, number)
floor floor(number)
from_items from_items(array[array])
group_by group_by(array, expref)
items items(object)
join join(string, array[string])
keys keys(object)
length length(string|array|object)
lower lower(string)
map map(expref, array)
max max(array[number]|array[string])
max_by max_by(array, expref)
merge merge(object)
min min(array[number]|array[string])
min_by min_by(array, expref)
not_null not_null(any)
pad_left pad_left(string, number, string)
pad_right pad_right(string, number, string)
replace replace(string, string, string, number)
reverse reverse(array|string)
sort sort(array[string]|array[number])
sort_by sort_by(array, expref)
split split(string, string, number)
starts_with starts_with(string, string)
sum sum(array[number])
to_array to_array(any)
to_number to_number(any)
to_string to_string(any)
trim trim(string, string)
trim_left trim_left(string, string)
trim_right trim_right(string, string)
type type(any)
upper upper(string)
values values(object)
zip zip(array, array)

kyverno-json functions

Name Signature
at at(array, any)
concat concat(string, string)
json_parse json_parse(string)
wildcard wildcard(string, string)

kyverno functions

Name Signature
compare compare(string, string)
equal_fold equal_fold(string, string)
replace replace(string, string, string, number)
replace_all replace_all(string, string, string)
to_upper to_upper(string)
to_lower to_lower(string)
trim trim(string, string)
trim_prefix trim_prefix(string, string)
split split(string, string)
regex_replace_all regex_replace_all(string, string|number, string|number)
regex_replace_all_literal regex_replace_all_literal(string, string|number, string|number)
regex_match regex_match(string, string|number)
pattern_match pattern_match(string, string|number)
label_match label_match(object, object)
to_boolean to_boolean(string)
add add(any, any)
sum sum(array)
subtract subtract(any, any)
multiply multiply(any, any)
divide divide(any, any)
modulo modulo(any, any)
round round(number, number)
base64_decode base64_decode(string)
base64_encode base64_encode(string)
time_since time_since(string, string, string)
time_now time_now()
time_now_utc time_now_utc()
path_canonicalize path_canonicalize(string)
truncate truncate(string, number)
semver_compare semver_compare(string, string)
parse_json parse_json(string)
parse_yaml parse_yaml(string)
lookup lookup(object|array, string|number)
items items(object|array, string, string)
object_from_lists object_from_lists(array, array)
random random(string)
x509_decode x509_decode(string)
time_to_cron time_to_cron(string)
time_add time_add(string, string)
time_parse time_parse(string, string)
time_utc time_utc(string)
time_diff time_diff(string, string)
time_before time_before(string, string)
time_after time_after(string, string)
time_between time_between(string, string, string)
time_truncate time_truncate(string, string)

chainsaw functions

Name Signature
env env(string)
x_k8s_get x_k8s_get(any, string, string, string, string)
x_k8s_list x_k8s_list(any, string, string, string)
x_k8s_exists x_k8s_exists(any, string, string, string, string)
x_k8s_resource_exists x_k8s_resource_exists(any, string, string)
x_k8s_server_version x_k8s_server_version(any)
\ No newline at end of file + Functions - Chainsaw
Skip to content

Functions

Experimental functions

Experimental functions are denoted by the x_ prefix.

These are functions that are subject to signature change in a future version.

built-in functions

Name Signature
abs abs(number)
avg avg(array[number])
ceil ceil(number)
contains contains(array|string, any)
ends_with ends_with(string, string)
find_first find_first(string, string, number, number)
find_last find_last(string, string, number, number)
floor floor(number)
from_items from_items(array[array])
group_by group_by(array, expref)
items items(object)
join join(string, array[string])
keys keys(object)
length length(string|array|object)
lower lower(string)
map map(expref, array)
max max(array[number]|array[string])
max_by max_by(array, expref)
merge merge(object)
min min(array[number]|array[string])
min_by min_by(array, expref)
not_null not_null(any)
pad_left pad_left(string, number, string)
pad_right pad_right(string, number, string)
replace replace(string, string, string, number)
reverse reverse(array|string)
sort sort(array[string]|array[number])
sort_by sort_by(array, expref)
split split(string, string, number)
starts_with starts_with(string, string)
sum sum(array[number])
to_array to_array(any)
to_number to_number(any)
to_string to_string(any)
trim trim(string, string)
trim_left trim_left(string, string)
trim_right trim_right(string, string)
type type(any)
upper upper(string)
values values(object)
zip zip(array, array)

kyverno-json functions

Name Signature
at at(array, any)
concat concat(string, string)
json_parse json_parse(string)
wildcard wildcard(string, string)

kyverno functions

Name Signature
compare compare(string, string)
equal_fold equal_fold(string, string)
replace replace(string, string, string, number)
replace_all replace_all(string, string, string)
to_upper to_upper(string)
to_lower to_lower(string)
trim trim(string, string)
trim_prefix trim_prefix(string, string)
split split(string, string)
regex_replace_all regex_replace_all(string, string|number, string|number)
regex_replace_all_literal regex_replace_all_literal(string, string|number, string|number)
regex_match regex_match(string, string|number)
pattern_match pattern_match(string, string|number)
label_match label_match(object, object)
to_boolean to_boolean(string)
add add(any, any)
sum sum(array)
subtract subtract(any, any)
multiply multiply(any, any)
divide divide(any, any)
modulo modulo(any, any)
round round(number, number)
base64_decode base64_decode(string)
base64_encode base64_encode(string)
time_since time_since(string, string, string)
time_now time_now()
time_now_utc time_now_utc()
path_canonicalize path_canonicalize(string)
truncate truncate(string, number)
semver_compare semver_compare(string, string)
parse_json parse_json(string)
parse_yaml parse_yaml(string)
lookup lookup(object|array, string|number)
items items(object|array, string, string)
object_from_lists object_from_lists(array, array)
random random(string)
x509_decode x509_decode(string)
time_to_cron time_to_cron(string)
time_add time_add(string, string)
time_parse time_parse(string, string)
time_utc time_utc(string)
time_diff time_diff(string, string)
time_before time_before(string, string)
time_after time_after(string, string)
time_between time_between(string, string, string)
time_truncate time_truncate(string, string)

chainsaw functions

Name Signature
env env(string)
x_k8s_get x_k8s_get(any, string, string, string, string)
x_k8s_list x_k8s_list(any, string, string, string)
x_k8s_exists x_k8s_exists(any, string, string, string, string)
x_k8s_resource_exists x_k8s_resource_exists(any, string, string)
x_k8s_server_version x_k8s_server_version(any)
\ No newline at end of file diff --git a/main/json-schemas/index.html b/main/json-schemas/index.html index 1ba596bbc..8f3feb3e7 100644 --- a/main/json-schemas/index.html +++ b/main/json-schemas/index.html @@ -1,4 +1,4 @@ - JSON schemas - Chainsaw
Skip to content

JSON schemas

JSON schemas for Chainsaw resources are available to enable validation and autocompletion in your IDE:

VS code

In VS code, simply add a comment on top of your YAML resources.

Test

# yaml-language-server: $schema=https://raw.githubusercontent.com/kyverno/chainsaw/main/.schemas/json/test-chainsaw-v1alpha1.json
+ JSON schemas - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
JSON schemas
 
JSON schemas for Chainsaw resources are available to enable validation and autocompletion in your IDE:
  
VS code
 
In VS code, simply add a comment on top of your YAML resources.
 
Test
 
# yaml-language-server: $schema=https://raw.githubusercontent.com/kyverno/chainsaw/main/.schemas/json/test-chainsaw-v1alpha1.json
 apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
@@ -22,4 +22,4 @@
 
 
Applying CRDs
 
An alternative is to apply the Chainsaw CRDs in a kubernetes cluster and let the Kubernetes extension do the rest.
 
CRD definitions are provided in our GitHub repository.
 
The command below will apply the Chainsaw CRDs to the configured cluster:
 
kubectl apply -f https://raw.githubusercontent.com/kyverno/chainsaw/main/config/crds/chainsaw.kyverno.io_configurations.yaml
 kubectl apply -f https://raw.githubusercontent.com/kyverno/chainsaw/main/config/crds/chainsaw.kyverno.io_tests.yaml
 
 
Exporting schemas
 
Chainsaw can export JSON schemas locally.
 
chainsaw export schemas <local path>
-
 
See Chainsaw export schemas reference for more details.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

See Chainsaw export schemas reference for more details.

\ No newline at end of file diff --git a/main/more/events/index.html b/main/more/events/index.html index 85d562823..014a5970d 100644 --- a/main/more/events/index.html +++ b/main/more/events/index.html @@ -1,4 +1,4 @@ - Working with events - Chainsaw
Skip to content

Working with events

Kubernetes events are regular Kubernetes objects and can be asserted on just like any other object:

apiVersion: v1
+ Working with events - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Working with events
 
Kubernetes events are regular Kubernetes objects and can be asserted on just like any other object:
 
apiVersion: v1
 kind: Event
 reason: Started
 source:
@@ -7,4 +7,4 @@
   apiVersion: v1
   kind: Pod
   name: my-pod
-
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/more/kuttl-migration/index.html b/main/more/kuttl-migration/index.html index bb81be254..f47a10f07 100644 --- a/main/more/kuttl-migration/index.html +++ b/main/more/kuttl-migration/index.html @@ -1,3 +1,3 @@ - Migration from KUTTL - Chainsaw
Skip to content

Migration from KUTTL

Overview

The chainsaw migrate kuttl tests and chainsaw migrate kuttl config commands are designed for the migration of KUTTL tests to Chainsaw.

Reference documentation

You can view the full command documentation here.

Examples

Migrate tests

The command below will migrate KUTTL tests to Chainsaw and overwrite original files with converted ones.

chainsaw migrate kuttl tests path/to/kuttl/tests --save --cleanup
+ Migration from KUTTL - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Migration from KUTTL
 
Overview
 
The chainsaw migrate kuttl tests and chainsaw migrate kuttl config commands are designed for the migration of KUTTL tests to Chainsaw.
  
 
Reference documentation
 
You can view the full command documentation here.
 
 
Examples
 
Migrate tests
 
The command below will migrate KUTTL tests to Chainsaw and overwrite original files with converted ones.
 
chainsaw migrate kuttl tests path/to/kuttl/tests --save --cleanup
 
 
This will generate a chainsaw-test.yaml for every KUTTL test discovered.
 
Migrate configuration
 
The command below will migrate a KUTTL test suite file to the corresponding Chainsaw Configuration.
 
chainsaw migrate kuttl config path/to/kuttl/testsuite --save --cleanup
-
 
This will generate a .chainsaw.yaml configuration file.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

This will generate a .chainsaw.yaml configuration file.

\ No newline at end of file diff --git a/main/more/lint/index.html b/main/more/lint/index.html index 165ad2aa7..636c1e9ac 100644 --- a/main/more/lint/index.html +++ b/main/more/lint/index.html @@ -1,4 +1,4 @@ - Lint tests - Chainsaw
Skip to content

Lint tests

Overview

Chainsaw comes with a lint command to detect ill-formated tests.

Reference documentation

You can view the full command documentation here.

Usage

To build the docs of a test, Chainsaw provides the chainsaw lint test -f path/to/chainsaw-test.yaml command.

chainsaw lint test -f - <<EOF
+ Lint tests - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Lint tests
 
Overview
 
Chainsaw comes with a lint command to detect ill-formated tests.
 
 
Reference documentation
 
You can view the full command documentation here.
 
 
Usage
 
To build the docs of a test, Chainsaw provides the chainsaw lint test -f path/to/chainsaw-test.yaml command.
 
chainsaw lint test -f - <<EOF
 apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
@@ -11,4 +11,4 @@
 EOF
 
 
Processing input...
 The document is valid
-
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/more/test-docs/index.html b/main/more/test-docs/index.html index 91f4dca41..f039698d6 100644 --- a/main/more/test-docs/index.html +++ b/main/more/test-docs/index.html @@ -1,4 +1,4 @@ - Building test docs - Chainsaw
Skip to content

Building test docs

Overview

Chainsaw makes it simple to build the documentation of your tests.

As test suites grow, it becomes important to document what a test does and how it is supposed to work.

Going through the implementation of a test to understand its purpose is not an efficient strategy.

Reference documentation

You can view the full command documentation here.

Usage

To build the docs of a test, Chainsaw provides the chainsaw build docs command.

chainsaw build docs --test-dir path/to/chainsaw/tests
+ Building test docs - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Building test docs
 
Overview
 
Chainsaw makes it simple to build the documentation of your tests.
 
As test suites grow, it becomes important to document what a test does and how it is supposed to work.
 
Going through the implementation of a test to understand its purpose is not an efficient strategy.
 
 
Reference documentation
 
You can view the full command documentation here.
 
 
Usage
 
To build the docs of a test, Chainsaw provides the chainsaw build docs command.
 
chainsaw build docs --test-dir path/to/chainsaw/tests
 
 
This will automatically discover tests and document steps and operations in try, catch and finally statements.
 
The description field
 
Additionally, you can set the description field in:
 
     
  • TestSpec
    •  
  • TestStepSpec
    •  
  • Operation
    •  
  • Catch
    •  
  • Finally
    •  
 
Chainsaw will output them nicely in the built docs.
 
Example
 
See below for an example test and the corresponding built docs.
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
@@ -14,4 +14,4 @@
     - description: Check the configmap content.
       assert:
         file: configmap-assert.yaml
-
 
 
Test: basic
 
This is a very simple test that creates a configmap and checks the content is as expected.
 
Steps
 
  
 # Name Try Catch Finally 
   
 1 step-1 2 0 0 
  
 
Step: step-1
 
This step applies the configmap in the cluster and checks the configmap content.
 
Try
 
  
 # Operation Description 
   
 1 apply Create the configmap. 
 
 2 assert Check the configmap content. 
  
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Test: basic

This is a very simple test that creates a configmap and checks the content is as expected.

Steps

# Name Try Catch Finally
1 step-1 2 0 0

Step: step-1

This step applies the configmap in the cluster and checks the configmap content.

Try

# Operation Description
1 apply Create the configmap.
2 assert Check the configmap content.
\ No newline at end of file diff --git a/main/operations/apply/index.html b/main/operations/apply/index.html index 9981cfde5..fa07f4fd3 100644 --- a/main/operations/apply/index.html +++ b/main/operations/apply/index.html @@ -1,4 +1,4 @@ - Apply - Chainsaw
Skip to content

Apply

The apply operation lets you define resources that should be applied to the Kubernetes cluster during the test step.

These can be configurations, deployments, services, or any other Kubernetes resource.

Configuration

Reference documentation

  • The full structure of the Apply is documented here.
  • This operation supports bindings.
  • This operation supports outputs.

Usage examples

Below is an example of using apply in a Test resource.

Using a specific file

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Apply - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Apply
 
The apply operation lets you define resources that should be applied to the Kubernetes cluster during the test step.
 
These can be configurations, deployments, services, or any other Kubernetes resource.
 
Configuration
 
 
Reference documentation
 
     
  • The full structure of the Apply is documented here.
    •  
  • This operation supports bindings.
    •  
  • This operation supports outputs.
    •  
 
 
Usage examples
 
Below is an example of using apply in a Test resource.
 
 
Using a specific file
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -84,4 +84,4 @@
         # - fail if the operation succeeded
         ($error != null): true
 # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/operations/assert/index.html b/main/operations/assert/index.html index 2443e3bdb..d2e2d0c45 100644 --- a/main/operations/assert/index.html +++ b/main/operations/assert/index.html @@ -1,4 +1,4 @@ - Assert - Chainsaw
Skip to content

Assert

The assert operation allows you to specify conditions that should hold true for a successful test.

For example, after applying resources, you might want to ensure that a particular pod is running or a service is accessible.

Assertion trees

Assertions in Chainsaw are based on assertion trees.

Assertion trees are a solution to declaratively represent complex conditions like partial array comparisons or complex operations against an incoming data structure.

Assertion trees are compatible with standard assertions that exist in tools like KUTTL but can do a lot more. Please see the assertion trees documentation in kyverno-json for details.

Configuration

Reference documentation

  • The full structure of the Assert is documented here.
  • This operation supports bindings.

Usage examples

Below is an example of using assert in a Test resource.

Using a specific file for assertions

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Assert - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Assert
 
The assert operation allows you to specify conditions that should hold true for a successful test.
 
For example, after applying resources, you might want to ensure that a particular pod is running or a service is accessible.
 
 
Assertion trees
 
Assertions in Chainsaw are based on assertion trees.
 
Assertion trees are a solution to declaratively represent complex conditions like partial array comparisons or complex operations against an incoming data structure.
 
Assertion trees are compatible with standard assertions that exist in tools like KUTTL but can do a lot more. Please see the assertion trees documentation in kyverno-json for details.
 
 
Configuration
 
 
Reference documentation
 
     
  • The full structure of the Assert is documented here.
    •  
  • This operation supports bindings.
    •  
 
 
Usage examples
 
Below is an example of using assert in a Test resource.
 
 
Using a specific file for assertions
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -48,4 +48,4 @@
           spec:
             (replicas > 3): true
     # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/operations/check/index.html b/main/operations/check/index.html index fa3d947c5..2eff0e005 100644 --- a/main/operations/check/index.html +++ b/main/operations/check/index.html @@ -1 +1 @@ - Operation checks - Chainsaw
Skip to content

Operation checks

Considering an operation's success or failure is not always as simple as checking an error code.

  • Sometimes an operation can fail but the failure is what you expected, hence the operation should be reported as successful.
  • Sometimes an operation can succeed but the result is not what you expected, in this case, the operation should be reported as a failure.

To support those kinds of use cases, some operations support an additional check field to evaluate the operation result against an assertion tree.

Assertion trees

Assertions in Chainsaw are based on assertion trees.

Assertion trees are a solution to declaratively represent complex conditions like partial array comparisons or complex operations against an incoming data structure.

Assertion trees are compatible with standard assertions that exist in tools like KUTTL but can do a lot more. Please see the assertion trees documentation in kyverno-json for details.

Checked model

Different operations have a different model passed through the assertion tree.

The object passed to the assertion tree is the output object of the operation. Additional data like error or standard logs are passed using bindings ($error, $stdout, $stderr)

Expect vs Check

While a simple check is enough to determine the result of a single operation, we needed a more advanced construct to cover apply and create operations. Those operations can operate on files containing multiple manifests and every manifest can have a different result.

To support more granular checks we use the expect field that contains an array of Expectations. Every expectation is made of an optional match and a check statement.

This way it is possible to control the scope of a check.

Null match

If the match statement is null, the check statement applies to all manifests in the operation.

If no expectation matches a given manifest, the default expectation will be used, checking that no error occurred.

Apply

apply supports expect and has the following elements to be checked:

Name Purpose Type
$values Values provided when invoking chainsaw with --values flag object
$namespace Name of the current test namespace string
$client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object
$config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object
$test.id Current test id int
$step.id Current step id int
$operation.id Current operation id int
$operation.resourceId Current resource id int
$error The error message (if any) at the end of the operation string
@ The state of the resource (if any) at the end of the operation object

Command

command supports check and has the following elements to be checked:

Name Purpose Type
$values Values provided when invoking chainsaw with --values flag object
$namespace Name of the current test namespace string
$client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object
$config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object
$test.id Current test id int
$step.id Current step id int
$operation.id Current operation id int
$operation.resourceId Current resource id int
$error The error message (if any) at the end of the operation string
$stdout The content of the standard console output (if any) at the end of the operation string
$stderr The content of the standard console error output (if any) at the end of the operation string
@ Always null

Create

create supports expect and has the following elements to be checked:

Name Purpose Type
$values Values provided when invoking chainsaw with --values flag object
$namespace Name of the current test namespace string
$client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object
$config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object
$test.id Current test id int
$step.id Current step id int
$operation.id Current operation id int
$operation.resourceId Current resource id int
$error The error message (if any) at the end of the operation string
@ The state of the resource (if any) at the end of the operation object

Delete

delete supports check and has the following elements to be checked:

Name Purpose Type
$values Values provided when invoking chainsaw with --values flag object
$namespace Name of the current test namespace string
$client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object
$config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object
$test.id Current test id int
$step.id Current step id int
$operation.id Current operation id int
$operation.resourceId Current resource id int
$error The error message (if any) at the end of the operation string
@ The state of the resource (if any) at the end of the operation object

Patch

patch supports expect and has the following elements to be checked:

Name Purpose Type
$values Values provided when invoking chainsaw with --values flag object
$namespace Name of the current test namespace string
$client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object
$config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object
$test.id Current test id int
$step.id Current step id int
$operation.id Current operation id int
$operation.resourceId Current resource id int
$error The error message (if any) at the end of the operation string
@ The state of the resource (if any) at the end of the operation object

Script

script supports check and has the following elements to be checked:

Name Purpose Type
$values Values provided when invoking chainsaw with --values flag object
$namespace Name of the current test namespace string
$client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object
$config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object
$test.id Current test id int
$step.id Current step id int
$operation.id Current operation id int
$operation.resourceId Current resource id int
$error The error message (if any) at the end of the operation string
$stdout The content of the standard console output (if any) at the end of the operation string
$stderr The content of the standard console error output (if any) at the end of the operation string
@ Always null

Update

update supports expect and has the following elements to be checked:

Name Purpose Type
$values Values provided when invoking chainsaw with --values flag object
$namespace Name of the current test namespace string
$client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object
$config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object
$test.id Current test id int
$step.id Current step id int
$operation.id Current operation id int
$operation.resourceId Current resource id int
$error The error message (if any) at the end of the operation string
@ The state of the resource (if any) at the end of the operation object
\ No newline at end of file + Operation checks - Chainsaw
Skip to content

Operation checks

Considering an operation's success or failure is not always as simple as checking an error code.

  • Sometimes an operation can fail but the failure is what you expected, hence the operation should be reported as successful.
  • Sometimes an operation can succeed but the result is not what you expected, in this case, the operation should be reported as a failure.

To support those kinds of use cases, some operations support an additional check field to evaluate the operation result against an assertion tree.

Assertion trees

Assertions in Chainsaw are based on assertion trees.

Assertion trees are a solution to declaratively represent complex conditions like partial array comparisons or complex operations against an incoming data structure.

Assertion trees are compatible with standard assertions that exist in tools like KUTTL but can do a lot more. Please see the assertion trees documentation in kyverno-json for details.

Checked model

Different operations have a different model passed through the assertion tree.

The object passed to the assertion tree is the output object of the operation. Additional data like error or standard logs are passed using bindings ($error, $stdout, $stderr)

Expect vs Check

While a simple check is enough to determine the result of a single operation, we needed a more advanced construct to cover apply and create operations. Those operations can operate on files containing multiple manifests and every manifest can have a different result.

To support more granular checks we use the expect field that contains an array of Expectations. Every expectation is made of an optional match and a check statement.

This way it is possible to control the scope of a check.

Null match

If the match statement is null, the check statement applies to all manifests in the operation.

If no expectation matches a given manifest, the default expectation will be used, checking that no error occurred.

Apply

apply supports expect and has the following elements to be checked:

Name Purpose Type
$values Values provided when invoking chainsaw with --values flag object
$namespace Name of the current test namespace string
$client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object
$config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object
$test.id Current test id int
$step.id Current step id int
$operation.id Current operation id int
$operation.resourceId Current resource id int
$error The error message (if any) at the end of the operation string
@ The state of the resource (if any) at the end of the operation object

Command

command supports check and has the following elements to be checked:

Name Purpose Type
$values Values provided when invoking chainsaw with --values flag object
$namespace Name of the current test namespace string
$client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object
$config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object
$test.id Current test id int
$step.id Current step id int
$operation.id Current operation id int
$operation.resourceId Current resource id int
$error The error message (if any) at the end of the operation string
$stdout The content of the standard console output (if any) at the end of the operation string
$stderr The content of the standard console error output (if any) at the end of the operation string
@ Always null

Create

create supports expect and has the following elements to be checked:

Name Purpose Type
$values Values provided when invoking chainsaw with --values flag object
$namespace Name of the current test namespace string
$client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object
$config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object
$test.id Current test id int
$step.id Current step id int
$operation.id Current operation id int
$operation.resourceId Current resource id int
$error The error message (if any) at the end of the operation string
@ The state of the resource (if any) at the end of the operation object

Delete

delete supports check and has the following elements to be checked:

Name Purpose Type
$values Values provided when invoking chainsaw with --values flag object
$namespace Name of the current test namespace string
$client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object
$config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object
$test.id Current test id int
$step.id Current step id int
$operation.id Current operation id int
$operation.resourceId Current resource id int
$error The error message (if any) at the end of the operation string
@ The state of the resource (if any) at the end of the operation object

Patch

patch supports expect and has the following elements to be checked:

Name Purpose Type
$values Values provided when invoking chainsaw with --values flag object
$namespace Name of the current test namespace string
$client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object
$config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object
$test.id Current test id int
$step.id Current step id int
$operation.id Current operation id int
$operation.resourceId Current resource id int
$error The error message (if any) at the end of the operation string
@ The state of the resource (if any) at the end of the operation object

Script

script supports check and has the following elements to be checked:

Name Purpose Type
$values Values provided when invoking chainsaw with --values flag object
$namespace Name of the current test namespace string
$client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object
$config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object
$test.id Current test id int
$step.id Current step id int
$operation.id Current operation id int
$operation.resourceId Current resource id int
$error The error message (if any) at the end of the operation string
$stdout The content of the standard console output (if any) at the end of the operation string
$stderr The content of the standard console error output (if any) at the end of the operation string
@ Always null

Update

update supports expect and has the following elements to be checked:

Name Purpose Type
$values Values provided when invoking chainsaw with --values flag object
$namespace Name of the current test namespace string
$client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object
$config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object
$test.id Current test id int
$step.id Current step id int
$operation.id Current operation id int
$operation.resourceId Current resource id int
$error The error message (if any) at the end of the operation string
@ The state of the resource (if any) at the end of the operation object
\ No newline at end of file diff --git a/main/operations/command/index.html b/main/operations/command/index.html index 2827ff144..43a5e8ffe 100644 --- a/main/operations/command/index.html +++ b/main/operations/command/index.html @@ -1,4 +1,4 @@ - Command - Chainsaw
Skip to content

Command

The command operation provides a means to execute a specific command during the test step.

Configuration

Reference documentation

  • The full structure of the Command is documented here.
  • This operation supports bindings.
  • This operation supports outputs.

Usage examples

Below is an example of using command in a Test resource.

Example

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Command - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Command
 
The command operation provides a means to execute a specific command during the test step.
 
Configuration
 
 
Reference documentation
 
     
  • The full structure of the Command is documented here.
    •  
  • This operation supports bindings.
    •  
  • This operation supports outputs.
    •  
 
 
Usage examples
 
Below is an example of using command in a Test resource.
 
 
Example
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -22,4 +22,4 @@
       # - fail if the operation succeeded
       ($error != null): true
 # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/operations/create/index.html b/main/operations/create/index.html index 90e371a98..7474a30b8 100644 --- a/main/operations/create/index.html +++ b/main/operations/create/index.html @@ -1,4 +1,4 @@ - Create - Chainsaw
Skip to content

Create

The create operation lets you define resources that should be created in the Kubernetes cluster during the test step. These can be configurations, deployments, services, or any other Kubernetes resource.

Configuration

Reference documentation

  • The full structure of the Create is documented here.
  • This operation supports bindings.
  • This operation supports outputs.

Warning

If the resource to be created already exists in the cluster, the step will fail.

Usage examples

Below is an example of using create in a Test resource.

Using a specific file

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Create - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Create
 
The create operation lets you define resources that should be created in the Kubernetes cluster during the test step. These can be configurations, deployments, services, or any other Kubernetes resource.
 
Configuration
 
 
Reference documentation
 
     
  • The full structure of the Create is documented here.
    •  
  • This operation supports bindings.
    •  
  • This operation supports outputs.
    •  
 
 
 
Warning
 
If the resource to be created already exists in the cluster, the step will fail.
 
 
Usage examples
 
Below is an example of using create in a Test resource.
 
 
Using a specific file
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -84,4 +84,4 @@
         # - fail if the operation succeeded
         ($error != null): true
 # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/operations/delete/index.html b/main/operations/delete/index.html index d0214854c..10d9f248e 100644 --- a/main/operations/delete/index.html +++ b/main/operations/delete/index.html @@ -1,4 +1,4 @@ - Delete - Chainsaw
Skip to content

Delete

The delete operation allows you to specify resources that should be deleted from the Kubernetes cluster before a particular test step is executed.

Info

The propagation policy is forced to Background because some types default to Orphan (this is the case for unmanaged jobs for example) and we don't want to let dangling pods run in the cluster after cleanup.

Configuration

Reference documentation

  • The full structure of the Delete is documented here.
  • This operation supports bindings.

Usage examples

Below is an example of using delete in a Test resource.

Example

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Delete - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Delete
 
The delete operation allows you to specify resources that should be deleted from the Kubernetes cluster before a particular test step is executed.
 
 
Info
 
The propagation policy is forced to Background because some types default to Orphan (this is the case for unmanaged jobs for example) and we don't want to let dangling pods run in the cluster after cleanup.
 
 
Configuration
 
 
Reference documentation
 
     
  • The full structure of the Delete is documented here.
    •  
  • This operation supports bindings.
    •  
 
 
Usage examples
 
Below is an example of using delete in a Test resource.
 
 
Example
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -35,4 +35,4 @@
         # - fail if the operation succeeded
         ($error != null): true
 # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/operations/error/index.html b/main/operations/error/index.html index 6ccfa1ae1..f82226c1f 100644 --- a/main/operations/error/index.html +++ b/main/operations/error/index.html @@ -1,4 +1,4 @@ - Error - Chainsaw
Skip to content

Error

The error operation lets you define a set of expected errors for a test step. If any of these errors occur during the test, they are treated as expected outcomes. However, if an error that's not on this list occurs, it will be treated as a test failure.

Assertion trees

Errors in Chainsaw are based on assertion trees.

Assertion trees is a solution to declaratively represent complex conditions like partial array comparisons or complex operations against an incoming data structure.

Assertion trees are compatible with standard assertions that exist in tools like KUTTL but can do a lot more. Please see the assertion trees documentation in kyverno-json for details.

Configuration

Reference documentation

  • The full structure of the Error is documented here.
  • This operation supports bindings.

Usage examples

Below is an example of using error in a Test resource.

Expecting an error from a specific file

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Error - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Error
 
The error operation lets you define a set of expected errors for a test step. If any of these errors occur during the test, they are treated as expected outcomes. However, if an error that's not on this list occurs, it will be treated as a test failure.
 
 
Assertion trees
 
Errors in Chainsaw are based on assertion trees.
 
Assertion trees is a solution to declaratively represent complex conditions like partial array comparisons or complex operations against an incoming data structure.
 
Assertion trees are compatible with standard assertions that exist in tools like KUTTL but can do a lot more. Please see the assertion trees documentation in kyverno-json for details.
 
 
Configuration
 
 
Reference documentation
 
     
  • The full structure of the Error is documented here.
    •  
  • This operation supports bindings.
    •  
 
 
Usage examples
 
Below is an example of using error in a Test resource.
 
 
Expecting an error from a specific file
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -48,4 +48,4 @@
           spec:
             (replicas > 3): true
     # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/operations/index.html b/main/operations/index.html index b7c065c21..50ef75140 100644 --- a/main/operations/index.html +++ b/main/operations/index.html @@ -1 +1 @@ - Operations - Chainsaw
Skip to content

Operations

While tests are made of test steps, test steps can be considered made of operations.

Every operation in a test step runs sequentially.

Only one action per operation

Every operation consists of a single action. While it is syntactically possible to create an operation with multiple actions, Chainsaw will verify and reject tests if operations containing multiple actions are found.

The reasoning behind this intentional choice is that it becomes harder to understand in which order actions will be executed in case an operation consists of multiple actions. For this reason, operations consisting of multiple actions are disallowed.

Common fields

All operations share some configuration fields.

Reference documentation

The full structure of the Operation is documented here.

ContinueOnError

Determines whether a test step should continue or not in case the operation is not successful.

Even if the test continues executing, it will still be reported as failed.

Available operations

Non-resource assertions

It is possible to evaluate assertions that do not depend on resources.

See Non-resource assertions for details.

Operation checks

Some operations support checking the operation execution result against specific expectations.

See Operation checks for use case details and supported operations.

Resource templating

Chainsaw can apply transformations to the resources before they are processed by the operation.

See Resource templating for use case details and supported operations.

\ No newline at end of file + Operations - Chainsaw
Skip to content

Operations

While tests are made of test steps, test steps can be considered made of operations.

Every operation in a test step runs sequentially.

Only one action per operation

Every operation consists of a single action. While it is syntactically possible to create an operation with multiple actions, Chainsaw will verify and reject tests if operations containing multiple actions are found.

The reasoning behind this intentional choice is that it becomes harder to understand in which order actions will be executed in case an operation consists of multiple actions. For this reason, operations consisting of multiple actions are disallowed.

Common fields

All operations share some configuration fields.

Reference documentation

The full structure of the Operation is documented here.

ContinueOnError

Determines whether a test step should continue or not in case the operation is not successful.

Even if the test continues executing, it will still be reported as failed.

Available operations

Non-resource assertions

It is possible to evaluate assertions that do not depend on resources.

See Non-resource assertions for details.

Operation checks

Some operations support checking the operation execution result against specific expectations.

See Operation checks for use case details and supported operations.

Resource templating

Chainsaw can apply transformations to the resources before they are processed by the operation.

See Resource templating for use case details and supported operations.

\ No newline at end of file diff --git a/main/operations/non-resource-assert/index.html b/main/operations/non-resource-assert/index.html index 3f63c6560..944fcdd04 100644 --- a/main/operations/non-resource-assert/index.html +++ b/main/operations/non-resource-assert/index.html @@ -1,4 +1,4 @@ - Non-resource assertions - Chainsaw
Skip to content

Non-resource assertions

Under certain circumstances, it makes sense to evaluate assertions that do not depend on resources. For example, when asserting the number of nodes in a cluster is equal to a known value.

Usage examples

Below is an example of using assert in a Test resource.

Using a file

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Non-resource assertions - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Non-resource assertions
 
Under certain circumstances, it makes sense to evaluate assertions that do not depend on resources. For example, when asserting the number of nodes in a cluster is equal to a known value.
 
Usage examples
 
Below is an example of using assert in a Test resource.
 
 
Using a file
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: non-resource-assertion
@@ -13,4 +13,4 @@
         resource:
           (x_k8s_list($client, 'v1', 'Node')):
             (length(items)): 2
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/operations/patch/index.html b/main/operations/patch/index.html index 2171cc072..42a0d9db3 100644 --- a/main/operations/patch/index.html +++ b/main/operations/patch/index.html @@ -1,4 +1,4 @@ - Patch - Chainsaw
Skip to content

Patch

The patch operation lets you define resources that should be modified in the Kubernetes cluster during the test step. These can be configurations, deployments, services, or any other Kubernetes resource.

Configuration

Reference documentation

  • The full structure of the Patch is documented here.
  • This operation supports bindings.
  • This operation supports outputs.

Warning

If the resource to be modified does not exist in the cluster, the step will fail.

Usage examples

Below is an example of using patch in a Test resource.

Using a specific file

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Patch - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Patch
 
The patch operation lets you define resources that should be modified in the Kubernetes cluster during the test step. These can be configurations, deployments, services, or any other Kubernetes resource.
 
Configuration
 
 
Reference documentation
 
     
  • The full structure of the Patch is documented here.
    •  
  • This operation supports bindings.
    •  
  • This operation supports outputs.
    •  
 
 
 
Warning
 
If the resource to be modified does not exist in the cluster, the step will fail.
 
 
Usage examples
 
Below is an example of using patch in a Test resource.
 
 
Using a specific file
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -84,4 +84,4 @@
         # - fail if the operation succeeded
         ($error != null): true
 # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/operations/script/index.html b/main/operations/script/index.html index b84557bd1..b4f6418ce 100644 --- a/main/operations/script/index.html +++ b/main/operations/script/index.html @@ -1,4 +1,4 @@ - Script - Chainsaw
Skip to content

Script

The script operation provides a means to run a script during the test step.

Configuration

Reference documentation

  • The full structure of the Script is documented here.
  • This operation supports bindings.
  • This operation supports outputs.

Usage examples

Below is an example of using script in a Test resource.

Example

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Script - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Script
 
The script operation provides a means to run a script during the test step.
 
Configuration
 
 
Reference documentation
 
     
  • The full structure of the Script is documented here.
    •  
  • This operation supports bindings.
    •  
  • This operation supports outputs.
    •  
 
 
Usage examples
 
Below is an example of using script in a Test resource.
 
 
Example
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -20,4 +20,4 @@
       # - fail if the operation succeeded
       ($error != null): true
 # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/operations/sleep/index.html b/main/operations/sleep/index.html index bf44811fd..89b23a90d 100644 --- a/main/operations/sleep/index.html +++ b/main/operations/sleep/index.html @@ -1,4 +1,4 @@ - Sleep - Chainsaw
Skip to content

Sleep

The sleep operation provides a means to sleep for a configured duration.

Configuration

Reference documentation

The full structure of the Sleep is documented here.

Usage examples

Below is an example of using sleep in a Test resource.

Example

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Sleep - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Sleep
 
The sleep operation provides a means to sleep for a configured duration.
 
Configuration
 
 
Reference documentation
 
The full structure of the Sleep is documented here.
 
 
Usage examples
 
Below is an example of using sleep in a Test resource.
 
 
Example
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -9,4 +9,4 @@
     - sleep:
         duration: 30s
     # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/operations/templating/index.html b/main/operations/templating/index.html index 0f547637c..a503c341e 100644 --- a/main/operations/templating/index.html +++ b/main/operations/templating/index.html @@ -1,4 +1,4 @@ - Resource templating - Chainsaw
Skip to content

Resource templating

Experimental status

This is an experimental feature, and implementation could change slightly in the next versions.

Info

Templating was disabled by default in v0.1.* but is now enabled by default since v0.2.1.

Chainsaw can apply transformations to the resources before they are processed by the operation.

This is useful when a resource needs some runtime configuration.

Templating must be enabled at the configuration, test, step, or operation level for the templating process to kick in. Alternatively, templating can be enabled using the --template flag when invoking chainsaw from the command line.

Note

Unlike assertion trees, templating can only be specified in leave nodes of the YAML tree.

Supported operations

Resource templating is supported in the following operations:

Assert and Error

When templating assert or error operations, the content is already an assertion tree.

For this reason, only the elements used for looking up the resources to be processed by the operation will be considered for templating. That is, only apiVersion, kind, name, namespace and labels are considered for templating. Other fields are not, they are part of the assertion tree only.

assert and error example

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Resource templating - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Resource templating
 
 
Experimental status
 
This is an experimental feature, and implementation could change slightly in the next versions.
 
 
 
Info
 
Templating was disabled by default in v0.1.* but is now enabled by default since v0.2.1.
 
 
Chainsaw can apply transformations to the resources before they are processed by the operation.
 
This is useful when a resource needs some runtime configuration.
 
Templating must be enabled at the configuration, test, step, or operation level for the templating process to kick in. Alternatively, templating can be enabled using the --template flag when invoking chainsaw from the command line.
 
 
Note
 
Unlike assertion trees, templating can only be specified in leave nodes of the YAML tree.
 
 
Supported operations
 
Resource templating is supported in the following operations:
  
Assert and Error
 
When templating assert or error operations, the content is already an assertion tree.
 
For this reason, only the elements used for looking up the resources to be processed by the operation will be considered for templating. That is, only apiVersion, kind, name, namespace and labels are considered for templating. Other fields are not, they are part of the assertion tree only.
 
 
assert and error example
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: template
@@ -72,4 +72,4 @@
         apiVersion: v1
         kind: ConfigMap
         name: ($namespace)
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/operations/update/index.html b/main/operations/update/index.html index a785fb87a..dbeabe59a 100644 --- a/main/operations/update/index.html +++ b/main/operations/update/index.html @@ -1,4 +1,4 @@ - Update - Chainsaw
Skip to content

Update

The update operation lets you define resources that should be updated in the Kubernetes cluster during the test step. These can be configurations, deployments, services, or any other Kubernetes resource.

Configuration

Reference documentation

  • The full structure of the Update is documented here.
  • This operation supports bindings.
  • This operation supports outputs.

Warning

If the resource to be updated doesn't exist in the cluster, the step will fail.

Usage examples

Below is an example of using update in a Test resource.

Using a specific file

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Update - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Update
 
The update operation lets you define resources that should be updated in the Kubernetes cluster during the test step. These can be configurations, deployments, services, or any other Kubernetes resource.
 
Configuration
 
 
Reference documentation
 
     
  • The full structure of the Update is documented here.
    •  
  • This operation supports bindings.
    •  
  • This operation supports outputs.
    •  
 
 
 
Warning
 
If the resource to be updated doesn't exist in the cluster, the step will fail.
 
 
Usage examples
 
Below is an example of using update in a Test resource.
 
 
Using a specific file
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -84,4 +84,4 @@
         # - fail if the operation succeeded
         ($error != null): true
 # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/operations/wait/index.html b/main/operations/wait/index.html index ed4c02ff9..d5fbc90c5 100644 --- a/main/operations/wait/index.html +++ b/main/operations/wait/index.html @@ -1,4 +1,4 @@ - Wait - Chainsaw
Skip to content

Wait

The wait operation is a wrapper around kubectl wait. It allows to wait for deletion or conditions against resources.

Configuration

Reference documentation

The full structure of the Wait is documented here.

Deprecated syntax

You can specify the resource directly instead of using apiVersion and kind.

This is a deprecated syntax though and will be removed in a future version.

Clustered resources

When used with a clustered resource, the namespace is ignored and is not added to the corresponding kubectl command.

All resources

If you don't specify a name or a selector, the wait operation will consider all resources.

All namespaces

When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.

Usage examples

Below is an example of using wait in a Test resource.

Wait pod ready

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Wait - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Wait
 
The wait operation is a wrapper around kubectl wait. It allows to wait for deletion or conditions against resources.
 
Configuration
 
 
Reference documentation
 
The full structure of the Wait is documented here.
 
 
 
Deprecated syntax
 
You can specify the resource directly instead of using apiVersion and kind.
 
This is a deprecated syntax though and will be removed in a future version.
 
 
Clustered resources
 
When used with a clustered resource, the namespace is ignored and is not added to the corresponding kubectl command.
 
All resources
 
If you don't specify a name or a selector, the wait operation will consider all resources.
 
All namespaces
 
When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.
 
Usage examples
 
Below is an example of using wait in a Test resource.
 
 
Wait pod ready
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -156,4 +156,4 @@
         kind: Pod
         format: yaml
         # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/quick-start/advanced/cluster-wide-resources/index.html b/main/quick-start/advanced/cluster-wide-resources/index.html deleted file mode 100644 index 2bea419b6..000000000 --- a/main/quick-start/advanced/cluster-wide-resources/index.html +++ /dev/null @@ -1 +0,0 @@ - Cluster-wide resources - Chainsaw
Skip to content

Cluster-wide resources

This section is under construction and should be ready soon. Please check again in a few days.

If you are interested in contributing see the tracking issue.

\ No newline at end of file diff --git a/main/quick-start/advanced/concurrency/index.html b/main/quick-start/advanced/concurrency/index.html index ad0b9100a..cfaf2dc87 100644 --- a/main/quick-start/advanced/concurrency/index.html +++ b/main/quick-start/advanced/concurrency/index.html @@ -1,4 +1,4 @@ - Concurrency control - Chainsaw
Skip to content

Concurrency control

By default, Chainsaw will run tests in parallel.

The number of concurrent tests can be configured globally using a configuration file or with the --parallel flag.

Alternatively, the concurrent nature of a test can specified at the test level:

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Concurrency control - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Concurrency control
 
By default, Chainsaw will run tests in parallel.
 
The number of concurrent tests can be configured globally using a configuration file or with the --parallel flag.
 
Alternatively, the concurrent nature of a test can specified at the test level:
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -7,4 +7,4 @@
   # default value is `true`
   concurrent: true
   # ...
-
 
All non-concurrent tests are executed first, followed by the concurrent tests running in parallel.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

All non-concurrent tests are executed first, followed by the concurrent tests running in parallel.

\ No newline at end of file diff --git a/main/quick-start/advanced/crds/index.html b/main/quick-start/advanced/crds/index.html index b4ed3406d..d82803742 100644 --- a/main/quick-start/advanced/crds/index.html +++ b/main/quick-start/advanced/crds/index.html @@ -1,4 +1,4 @@ - Working with CRDs - Chainsaw
Skip to content

Working with CRDs

New Custom Resource Definitions are not immediately available for use in the Kubernetes API until the Kubernetes API has acknowledged them.

If a Custom Resource Definition is being defined inside of a test step, be sure to wait for the CustomResourceDefinition object to appear.

The test below applies a CRD and waits for it to become available:

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Working with CRDs - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Working with CRDs
 
New Custom Resource Definitions are not immediately available for use in the Kubernetes API until the Kubernetes API has acknowledged them.
 
If a Custom Resource Definition is being defined inside of a test step, be sure to wait for the CustomResourceDefinition object to appear.
 
The test below applies a CRD and waits for it to become available:
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -34,4 +34,4 @@
               singular: issue
             storedVersions:
             - v1alpha1
-
 
The CRD can be used in subsequent steps.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

The CRD can be used in subsequent steps.

\ No newline at end of file diff --git a/main/quick-start/advanced/index.html b/main/quick-start/advanced/index.html index e98ebc2c0..82eb2810a 100644 --- a/main/quick-start/advanced/index.html +++ b/main/quick-start/advanced/index.html @@ -1 +1 @@ - Advanced use cases - Chainsaw
Skip to content

Advanced use cases

This section is under construction and should be ready soon. Please check again in a few days.

If you are interested in contributing see the tracking issue.

\ No newline at end of file + Advanced use cases - Chainsaw
Skip to content

Advanced use cases

This section is under construction and should be ready soon. Please check again in a few days.

If you are interested in contributing see the tracking issue.

\ No newline at end of file diff --git a/main/quick-start/advanced/inline/index.html b/main/quick-start/advanced/inline/index.html index 19fb9a696..d28b4af97 100644 --- a/main/quick-start/advanced/inline/index.html +++ b/main/quick-start/advanced/inline/index.html @@ -1,4 +1,4 @@ - Inline resources - Chainsaw
Skip to content

Inline resources

When an operation needs to reference a resource, it can do so using a file path or directly specify the resource inline using the resource field.

The test below is equivalent to our first test:

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Inline resources - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Inline resources
 
When an operation needs to reference a resource, it can do so using a file path or directly specify the resource inline using the resource field.
 
The test below is equivalent to our first test:
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -21,4 +21,4 @@
             name: quick-start
           data:
             foo: bar
-
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/quick-start/advanced/kube-version/index.html b/main/quick-start/advanced/kube-version/index.html index 5fb3c1796..ae759b88b 100644 --- a/main/quick-start/advanced/kube-version/index.html +++ b/main/quick-start/advanced/kube-version/index.html @@ -1,4 +1,4 @@ - Check Kubernetes version - Chainsaw
Skip to content

Check Kubernetes version

The test below fetches the Kubernetes cluster version using x_k8s_server_version. It then uses the minor version retrieved to adapt an assertion based on the value in the $minorversion binding.

Tip

You can implement a ternary operator in JMESPath using an expression like this:

<condition> && <value-if-true> || <value-if-false>

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Check Kubernetes version - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Check Kubernetes version
 
The test below fetches the Kubernetes cluster version using x_k8s_server_version. It then uses the minor version retrieved to adapt an assertion based on the value in the $minorversion binding.
 
 
Tip
 
You can implement a ternary operator in JMESPath using an expression like this:
 
<condition> && <value-if-true> || <value-if-false>
 
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -32,4 +32,4 @@
               # Otherwise, due to a change in kubeadm, the group should be 'kubeadm:cluster-admins'.
               kyverno.io/created-by: (($minorversion < `29` && '{"groups":["system:masters","system:authenticated"],"username":"kubernetes-admin"}') || '{"groups":["kubeadm:cluster-admins","system:authenticated"],"username":"kubernetes-admin"}')
             name: pod01
-
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/quick-start/advanced/label-selectors/index.html b/main/quick-start/advanced/label-selectors/index.html index 040d06e5a..33a166c6b 100644 --- a/main/quick-start/advanced/label-selectors/index.html +++ b/main/quick-start/advanced/label-selectors/index.html @@ -1,4 +1,4 @@ - Use label selectors - Chainsaw
Skip to content

Use label selectors

Chainsaw can filter the tests to run using label selectors.

You can pass label selectors using the --selector flag when invoking the chainsaw test command.

Given the test below:

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Use label selectors - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Use label selectors
 
Chainsaw can filter the tests to run using label selectors.
 
You can pass label selectors using the --selector flag when invoking the chainsaw test command.
 
Given the test below:
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: basic
@@ -7,4 +7,4 @@
 spec:
   # ...
 
 
Invoking Chainsaw with the command below will take the test above into account:
 
chainsaw test --selector foo=bar
-
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/quick-start/advanced/multi-cluster/index.html b/main/quick-start/advanced/multi-cluster/index.html index 15e88a15c..861e21e78 100644 --- a/main/quick-start/advanced/multi-cluster/index.html +++ b/main/quick-start/advanced/multi-cluster/index.html @@ -1 +1 @@ - Multi-cluster setup - Chainsaw
Skip to content

Multi-cluster setup

This section is under construction and should be ready soon. Please check again in a few days.

If you are interested in contributing see the tracking issue.

\ No newline at end of file + Multi-cluster setup - Chainsaw
Skip to content

Multi-cluster setup

This section is under construction and should be ready soon. Please check again in a few days.

If you are interested in contributing see the tracking issue.

\ No newline at end of file diff --git a/main/quick-start/advanced/negative-testing/index.html b/main/quick-start/advanced/negative-testing/index.html index f6a1efed7..cc1145c60 100644 --- a/main/quick-start/advanced/negative-testing/index.html +++ b/main/quick-start/advanced/negative-testing/index.html @@ -1,4 +1,4 @@ - Negative testing - Chainsaw
Skip to content

Negative testing

Negative testing is the process of testing cases that are supposed to fail. That is, a test expects errors to happen and if the expected errors don't occur the test must fail.

Chainsaw supports negative testing by letting you decide what should be considered an error or not.

Tip

By default, Chainsaw will consider an operation failed if there was an error executing it (non-zero exit code in scripts and commands, error returned by the API server when calling into Kubernetes, etc...).

Script case

The test below expects an error and validates the returned error message:

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Negative testing - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Negative testing
 
Negative testing is the process of testing cases that are supposed to fail. That is, a test expects errors to happen and if the expected errors don't occur the test must fail.
 
Chainsaw supports negative testing by letting you decide what should be considered an error or not.
 
 
Tip
 
By default, Chainsaw will consider an operation failed if there was an error executing it (non-zero exit code in scripts and commands, error returned by the API server when calling into Kubernetes, etc...).
 
 
Script case
 
The test below expects an error and validates the returned error message:
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -43,4 +43,4 @@
               name: quick-start
           check:
             ($error != null): true
-
 
Using the match field, we can easily target failures related to specific resources.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Using the match field, we can easily target failures related to specific resources.

\ No newline at end of file diff --git a/main/quick-start/advanced/non-resource-assertions/index.html b/main/quick-start/advanced/non-resource-assertions/index.html index 4d3498803..7a97f6e55 100644 --- a/main/quick-start/advanced/non-resource-assertions/index.html +++ b/main/quick-start/advanced/non-resource-assertions/index.html @@ -1,4 +1,4 @@ - Non-resource assertions - Chainsaw
Skip to content

Non-resource assertions

Under certain circumstances, it makes sense to evaluate assertions that do not depend on resources. For example, when asserting the number of nodes in a cluster is equal to a known value.

The test below uses the x_k8s_list function to query the list of nodes in the cluster. It uses the results to compare the number of nodes found with a known number (1 in this case).

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Non-resource assertions - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Non-resource assertions
 
Under certain circumstances, it makes sense to evaluate assertions that do not depend on resources. For example, when asserting the number of nodes in a cluster is equal to a known value.
 
The test below uses the x_k8s_list function to query the list of nodes in the cluster. It uses the results to compare the number of nodes found with a known number (1 in this case).
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -9,4 +9,4 @@
         resource:
           (x_k8s_list($client, 'v1', 'Node')):
             (length(items)): 1
-
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/quick-start/advanced/test-output/index.html b/main/quick-start/advanced/test-output/index.html index 9579b8ed5..a78f8844d 100644 --- a/main/quick-start/advanced/test-output/index.html +++ b/main/quick-start/advanced/test-output/index.html @@ -1,4 +1,4 @@ - Test command output - Chainsaw
Skip to content

Test command output

Chainsaw can be used to easily check terminal output from CLIs and other commands. This is useful in that convoluted bash scripts involving chaining together tools like grep can be avoided or at least minimized to only complex use cases. Output to both stdout and stderr can be checked for a given string or precise contents.

Checking Output Contains

One basic use case for content checking is that the output simply contains a given string or piece of content. For example, you might want to run automated tests on a CLI binary you build to ensure that a given command produces output that contains some content you specify somewhere in the output. Let's use the following output from the kubectl version command to show these examples.

kubectl version
+ Test command output - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Test command output
 
Chainsaw can be used to easily check terminal output from CLIs and other commands. This is useful in that convoluted bash scripts involving chaining together tools like grep can be avoided or at least minimized to only complex use cases. Output to both stdout and stderr can be checked for a given string or precise contents.
 
Checking Output Contains
 
One basic use case for content checking is that the output simply contains a given string or piece of content. For example, you might want to run automated tests on a CLI binary you build to ensure that a given command produces output that contains some content you specify somewhere in the output. Let's use the following output from the kubectl version command to show these examples.
 
kubectl version
 
 Client Version: v1.28.2
 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
@@ -76,4 +76,4 @@
           ($error != null): true
           # This checks that the output is exactly as intended.
           ($stderr): "error: unknown command \"foo\" for \"kubectl\"\n\nDid you mean this?\n\ttop"
-
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/quick-start/assertion-trees/index.html b/main/quick-start/assertion-trees/index.html index fe6991f7e..10b61f16d 100644 --- a/main/quick-start/assertion-trees/index.html +++ b/main/quick-start/assertion-trees/index.html @@ -1,78 +1,132 @@ - Use assertions - Chainsaw
Skip to content

Use assertions

Whether verifying the scaling behavior of a database cluster or ensuring data consistency across instances, Chainsaw's assertion model provides the precision and flexibility needed for comprehensive validation.

Chainsaw allows declaring complex assertions with a simple and no-code approach, allowing assertions based on comparisons beyond simple equality, working with arrays, and other scenarios that could not be achieved before.

Tip

Under the hood, Chainsaw uses kyverno-json assertion trees. Refer to the assertion trees documentation for more details on the supported syntax.

Beyond simple equality

The assertion below will check that the number of replicas for a deployment is greater than 3 AND less than 6.

Chainsaw doesn't need to know the exact expected number of replicas. The (replicas > 3 && replicas < 6) expression will be evaluated until the result is true or the operation timeout expires (making the assertion fail).

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Use assertions - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Use assertions
 
Chainsaw allows declaring complex assertions with a simple and no-code approach, allowing assertions based on comparisons beyond simple equality, working with arrays, and other scenarios that could not be achieved before.
 
 
Tip
 
Under the hood, Chainsaw uses kyverno-json assertion trees. Refer to the assertion trees documentation for more details on the supported syntax.
 
 
Basic assertion
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
 spec:
   steps:
   - try:
-    # ...
-    - assert:
-        resource:
-          apiVersion: v1
-          kind: Deployment
-          metadata:
-            name: foo
+    - assert:
+        resource:
+          apiVersion: apps/v1
+          kind: Deployment
+          metadata:
+            name: coredns
+            namespace: kube-system
           spec:
-            (replicas > `3` && replicas < `6`): true
-    # ...
-
 
Working with arrays
 
Chainsaw query language makes it easy to assert on arrays. You can filter and transform arrays to select what you want to assert.
 
Filtering
 
In the example below we are creating a resource, then we assert that a condition with type == 'Ready' exists and has a field matching status: 'True':
 
apiVersion: chainsaw.kyverno.io/v1alpha1
+            replicas: 2
+
 
When asking Chainsaw to execute the assertion above, it will look for a deployment named coredns in the kube-system namespace and will compare the existing resource with the (partial) resource definition contained in the assertion.
 
In this specific case, if the field spec.replicas is set to 2 in the existing resource, the assertion will be considered valid. If it is not equal to 2 the assertion will be considered failed.
 
This is the most basic assertion Chainsaw can evaluate.
 
Slightly less basic assertion
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
 spec:
   steps:
   - try:
-    - apply:
+    - assert:
         resource:
-          apiVersion: tempo.grafana.com/v1alpha1
-          kind: TempoStack
+          apiVersion: apps/v1
+          kind: Deployment
           metadata:
-            name: simplest
-          spec:
-            storage:
-              secret:
-                name: minio
-                type: s3
-            # ...
-    - assert:
-        resource:
-          apiVersion: tempo.grafana.com/v1alpha1
-          kind: TempoStack
-          metadata:
-            name: simplest
-          status:
-            # filter conditions array to keep elements where `type == 'Ready'`
-            # and assert there's a single element matching the filter
-            # and that this element status is `True`
-            (conditions[?type == 'Ready']):
-            - status: 'True'
-
 
Comprehensive reporting
 
Chainsaw offers detailed resource diffs upon assertion failures.
 
In the example below, the assertion failure message (metadata.annotations.foo: Invalid value: "null": Expected value: "bar") is augmented with a resource diff.
 
It provides a clear view of discrepancies between expected and actual resources and gives more context around the specific failure (we can easily identify the owner of the offending pod for example).
 
| 09:55:50 | deployment | step-1   | ASSERT    | RUN   | v1/Pod @ chainsaw-rare-liger/*
-| 09:56:20 | deployment | step-1   | ASSERT    | ERROR | v1/Pod @ chainsaw-rare-liger/*
-    === ERROR
-    ---------------------------------------------------
-    v1/Pod/chainsaw-rare-liger/example-5477b4ff8c-tnhd9
-    ---------------------------------------------------
-    * metadata.annotations.foo: Invalid value: "null": Expected value: "bar"
-
-    --- expected
-    +++ actual
-    @@ -1,10 +1,16 @@
-      apiVersion: v1
-      kind: Pod
-      metadata:
-    -  annotations:
-    -    foo: bar
-        labels:
-          app: nginx
-    +  name: example-5477b4ff8c-tnhd9
-        namespace: chainsaw-rare-liger
-    +  ownerReferences:
-    +  - apiVersion: apps/v1
-    +    blockOwnerDeletion: true
-    +    controller: true
-    +    kind: ReplicaSet
-    +    name: example-5477b4ff8c
-    +    uid: 118abe16-ec42-4894-83db-64479c4aac6f
-      spec: {}
-| 09:56:20 | deployment | step-1   | TRY       | DONE  |
-
 
Next step
 
To continue our exploration of the main Chainsaw features, let's look at bindings and resource templating next.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+            labels:
+              k8s-app: kube-dns
+            namespace: kube-system
+          spec:
+            replicas: 2
+

This time we are not providing a resource name.

Chainsaw will look up all deployments with the k8s-app: kube-dns label in the kube-system namespace. The assertion will be considered valid if at least one deployment matches the (partial) resource definition contained in the assertion. If none match, the assertion will be considered failed.

Apart from the resource lookup process being a little bit more interesting, this kind of assertion is essentially the same as the previous one. Chainsaw is basically making a decision by comparing an actual and expected resource.

Beyond simple equality

The assertion below will check that the number of replicas for a deployment is greater than 3 AND less than 6.

Chainsaw doesn't need to know the exact expected number of replicas. The (replicas > 3 && replicas < 6) expression will be evaluated until the result is true or the operation timeout expires (making the assertion fail).

apiVersion: chainsaw.kyverno.io/v1alpha1
+kind: Test
+metadata:
+  name: example
+spec:
+  steps:
+  - try:
+    # ...
+    - assert:
+        resource:
+          apiVersion: v1
+          kind: Deployment
+          metadata:
+            name: foo
+          spec:
+            (replicas > `3` && replicas < `6`): true
+    # ...
+

Tip

To indicate that a key or value in the YAML document is an expression, simply place the element between parentheses:

  • this is an expression -> interpreted as a string
  • (this is an expression) -> interpreted as a JMESPath expression

Working with arrays

Chainsaw query language makes it easy to assert on arrays. You can filter and transform arrays to select what you want to assert.

Filtering

In the example below we are creating a resource, then we assert that a condition with type == 'Ready' exists and has a field matching status: 'True':

apiVersion: chainsaw.kyverno.io/v1alpha1
+kind: Test
+metadata:
+  name: example
+spec:
+  steps:
+  - try:
+    - apply:
+        resource:
+          apiVersion: tempo.grafana.com/v1alpha1
+          kind: TempoStack
+          metadata:
+            name: simplest
+          spec:
+            storage:
+              secret:
+                name: minio
+                type: s3
+            # ...
+    - assert:
+        resource:
+          apiVersion: tempo.grafana.com/v1alpha1
+          kind: TempoStack
+          metadata:
+            name: simplest
+          status:
+            # filter conditions array to keep elements where `type == 'Ready'`
+            # and assert there's a single element matching the filter
+            # and that this element status is `True`
+            (conditions[?type == 'Ready']):
+            - status: 'True'
+

Iterating

Being able to filter arrays allows selecting the elements to be processed.

On top of that, Chainsaw allows iterating over array elements to validate each item separately.

apiVersion: chainsaw.kyverno.io/v1alpha1
+kind: Test
+metadata:
+  name: example
+spec:
+  steps:
+  - try:
+    - assert:
+        resource:
+          apiVersion: apps/v1
+          kind: Deployment
+          metadata:
+            labels:
+              k8s-app: kube-dns
+            namespace: kube-system
+          spec:
+            template:
+              spec:
+                # the `~` modifier tells Chainsaw to iterate over the array elements
+                ~.(containers):
+                  securityContext: {}
+

This assertion uses the ~ modifier and Chainsaw will evaluate descendants once per element in the array.

Comprehensive reporting

Chainsaw offers detailed resource diffs upon assertion failures.

In the example below, the assertion failure message (metadata.annotations.foo: Invalid value: "null": Expected value: "bar") is augmented with a resource diff.

It provides a clear view of discrepancies between expected and actual resources and gives more context around the specific failure (we can easily identify the owner of the offending pod for example).

| 09:55:50 | deployment | step-1   | ASSERT    | RUN   | v1/Pod @ chainsaw-rare-liger/*
+| 09:56:20 | deployment | step-1   | ASSERT    | ERROR | v1/Pod @ chainsaw-rare-liger/*
+    === ERROR
+    ---------------------------------------------------
+    v1/Pod/chainsaw-rare-liger/example-5477b4ff8c-tnhd9
+    ---------------------------------------------------
+    * metadata.annotations.foo: Invalid value: "null": Expected value: "bar"
+
+    --- expected
+    +++ actual
+    @@ -1,10 +1,16 @@
+      apiVersion: v1
+      kind: Pod
+      metadata:
+    -  annotations:
+    -    foo: bar
+        labels:
+          app: nginx
+    +  name: example-5477b4ff8c-tnhd9
+        namespace: chainsaw-rare-liger
+    +  ownerReferences:
+    +  - apiVersion: apps/v1
+    +    blockOwnerDeletion: true
+    +    controller: true
+    +    kind: ReplicaSet
+    +    name: example-5477b4ff8c
+    +    uid: 118abe16-ec42-4894-83db-64479c4aac6f
+      spec: {}
+| 09:56:20 | deployment | step-1   | TRY       | DONE  |
+

Next step

To continue our exploration of the main Chainsaw features, let's look at bindings and resource templating next.

\ No newline at end of file diff --git a/main/quick-start/bindings/index.html b/main/quick-start/bindings/index.html index 89c466ea5..815bd108c 100644 --- a/main/quick-start/bindings/index.html +++ b/main/quick-start/bindings/index.html @@ -1,4 +1,4 @@ - Use bindings - Chainsaw
Skip to content

Use bindings

You can think of bindings as a side context where you can store and retrieve data based on keys.

This is particularly useful when some data is only known at runtime. For example, to pass data from one operation to another, to implement resource templating, to fetch data from an external system, etc.

Chainsaw offers some built-in bindings you can directly use in your tests but you can also create your own bindings if needed.

Inheritance

Bindings can be configured at the test, step or operation level.

All bindings configured at a given level are automatically inherited in child levels.

JMESPath

Chainsaw uses the JMESPath language, and bindings are implemented using lexical scoping.

Immutability

Bindings are immutable. This means two bindings can have the same name without overwriting each other.

When a binding is registered it potentially hides other bindings with the same name.

When this binding goes out of scope, previously registered bindings with the same name become visible again.

Built-in bindings

The $namespace binding is a good example of a built-in binding provided by Chainsaw. It contains the name of the ephemeral namespace used to execute a test (by default Chainsaw will create an ephemeral namespace for each test).

In the operation below, we are assigning the value of the $namespace binding to an environment variable, and echo it in a script:

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Use bindings - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Use bindings
 
You can think of bindings as a side context where you can store and retrieve data based on keys.
 
This is particularly useful when some data is only known at runtime. For example, to pass data from one operation to another, to implement resource templating, to fetch data from an external system, etc.
 
Chainsaw offers some built-in bindings you can directly use in your tests but you can also create your own bindings if needed.
 
Inheritance
 
Bindings can be configured at the test, step or operation level.
 
All bindings configured at a given level are automatically inherited in child levels.
 
 
JMESPath
 
Chainsaw uses the JMESPath language, and bindings are implemented using lexical scoping.
 
 
Immutability
 
Bindings are immutable. This means two bindings can have the same name without overwriting each other.
 
When a binding is registered it potentially hides other bindings with the same name.
 
When this binding goes out of scope, previously registered bindings with the same name become visible again.
 
Built-in bindings
 
The $namespace binding is a good example of a built-in binding provided by Chainsaw. It contains the name of the ephemeral namespace used to execute a test (by default Chainsaw will create an ephemeral namespace for each test).
 
In the operation below, we are assigning the value of the $namespace binding to an environment variable, and echo it in a script:
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -38,4 +38,4 @@
         - name: GREETINGS
           value: (join(' ', [$hello, $chainsaw, 'is', $awesome]))
         content: echo $GREETINGS
-
 
Next step
 
Let's see how bindings can be useful with resource templating.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Next step

Let's see how bindings can be useful with resource templating.

\ No newline at end of file diff --git a/main/quick-start/cleanup/index.html b/main/quick-start/cleanup/index.html index cc1d4be8a..671f788df 100644 --- a/main/quick-start/cleanup/index.html +++ b/main/quick-start/cleanup/index.html @@ -1,4 +1,4 @@ - Control your cleanup - Chainsaw
Skip to content

Control your cleanup

Unless configured differently, by default Chainsaw will automatically remove the resources it created after a test finishes.

Cleanup happens in reverse order of creation (created last, cleaned up first). This is important, especially when the controller being tested makes use of finalizers.

Overriding cleanup timeout

Note that Chainsaw performs a blocking deletion, that is, it will wait until the resource is not present anymore in the cluster before proceeding with the next resource cleanup.

Timeout

A global cleanup timeout can be defined at the configuration level or using command line flags.

It can also be overridden on a per-test or per-step basis but not at the operation level.

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Control your cleanup - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Control your cleanup
 
Unless configured differently, by default Chainsaw will automatically remove the resources it created after a test finishes.
 
Cleanup happens in reverse order of creation (created last, cleaned up first). This is important, especially when the controller being tested makes use of finalizers.
 
 
Overriding cleanup timeout
 
Note that Chainsaw performs a blocking deletion, that is, it will wait until the resource is not present anymore in the cluster before proceeding with the next resource cleanup.
 
 
Timeout
 
A global cleanup timeout can be defined at the configuration level or using command line flags.
 
It can also be overridden on a per-test or per-step basis but not at the operation level.
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -52,4 +52,4 @@
     - script:
         content: |
           rm -f ./dynamic
-
 
Next step
 
At this point, we covered the main Chainsaw features.
 
Look at the next steps section to find out what to do next.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Next step

At this point, we covered the main Chainsaw features.

Look at the next steps section to find out what to do next.

\ No newline at end of file diff --git a/main/quick-start/completion/index.html b/main/quick-start/completion/index.html index 62c58a7f9..767e497af 100644 --- a/main/quick-start/completion/index.html +++ b/main/quick-start/completion/index.html @@ -1 +1 @@ - Shell completion - Chainsaw
Skip to content

Shell completion

Once installed, use chainsaw completion command to generate and register the autocompletion script for the specified shell.

Supported shells are:

\ No newline at end of file + Shell completion - Chainsaw
Skip to content

Shell completion

Once installed, use chainsaw completion command to generate and register the autocompletion script for the specified shell.

Supported shells are:

\ No newline at end of file diff --git a/main/quick-start/first-test/index.html b/main/quick-start/first-test/index.html index 244f29917..5b35f7ca1 100644 --- a/main/quick-start/first-test/index.html +++ b/main/quick-start/first-test/index.html @@ -1,4 +1,4 @@ - Create a Test - Chainsaw
Skip to content

Create a Test

To create a chainsaw test all you need to do is to create one (or more) YAML file(s).

The recommended approach is to create one folder per test, with a chainsaw-test.yaml file containing one (or more) test definition(s). The test definition can reference other files in the same folder or anywhere else on the file system as needed.

Tip

While chainsaw supports other syntaxes, we strongly recommend the explicit approach.

What is a test?

To put it simply, a test can be represented as an ordered sequence of test steps.

In turn, a test step can be represented as an ordered sequence of operations.

Let's write our first test

For this quick start, we will create a (very simple) Test with one step and two operations:

  1. Create a ConfigMap from a manifest
  2. Verify the ConfigMap was created and contains the expected data

Follow the instructions below to create the folder and files defining our first test.

Create a test folder

# create test folder
+ Create a test - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Create a test
 
To create a chainsaw test all you need to do is to create one (or more) YAML file(s).
 
The recommended approach is to create one folder per test, with a chainsaw-test.yaml file containing one (or more) test definition(s). The test definition can reference other files in the same folder or anywhere else on the file system as needed.
 
 
Tip
 
While chainsaw supports other syntaxes, we strongly recommend the explicit approach.
 
 
What is a test?
 
To put it simply, a test can be represented as an ordered sequence of test steps.
 
In turn, a test step can be represented as an ordered sequence of operations.
 
Let's write our first test
 
For this quick start, we will create a (very simple) Test with one step and two operations:
 
     
  1. Create a ConfigMap from a manifest
    2.  
  2. Verify the ConfigMap was created and contains the expected data
    3.  
 
Follow the instructions below to create the folder and files defining our first test.
 
Create a test folder
 
# create test folder
 mkdir chainsaw-quick-start
 
 # enter test folder
@@ -30,4 +30,4 @@
         # file is relative to the test folder
         file: configmap.yaml
 EOF
-
 
Next step
 
Now we have created our first test, you can continue to the next section to execute it.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Next step

Now we have created our first test, you can continue to the next section to execute it.

\ No newline at end of file diff --git a/main/quick-start/index.html b/main/quick-start/index.html index e94d66c5f..f9e646913 100644 --- a/main/quick-start/index.html +++ b/main/quick-start/index.html @@ -1 +1 @@ - Getting started - Chainsaw
Skip to content

Getting started

Chainsaw is a tool primarily developed to run end-to-end tests in Kubernetes clusters.

It is meant to test Kubernetes operators work as expected by running a sequence of steps and asserting various conditions.

Why we made it?

While developing Kyverno we need to run end-to-end tests to make sure our admission controller works as expected.

A typical Kyverno end-to-end test

Kyverno can validate, mutate and generate resources based on policies installed in a cluster and a typical test is:

  1. Create a policy
  2. Create a resource
  3. Check that Kyverno acted as expected
  4. Cleanup and move to the next test

How to use it?

Chainsaw is built with CI tools in mind - you only really need to download and execute it in your build script.

However, installing it on your local machine is entirely possible.

\ No newline at end of file + Getting started - Chainsaw
Skip to content

Getting started

Chainsaw is a tool primarily developed to run end-to-end tests in Kubernetes clusters.

It is meant to test Kubernetes operators work as expected by running a sequence of steps and asserting various conditions.

Why we made it?

While developing Kyverno we need to run end-to-end tests to make sure our admission controller works as expected.

A typical Kyverno end-to-end test

Kyverno can validate, mutate and generate resources based on policies installed in a cluster and a typical test is:

  1. Create a policy
  2. Create a resource
  3. Check that Kyverno acted as expected
  4. Cleanup and move to the next test

How to use it?

Chainsaw is built with CI tools in mind - you only really need to download and execute it in your build script.

However, installing it on your local machine is entirely possible.

\ No newline at end of file diff --git a/main/quick-start/install/index.html b/main/quick-start/install/index.html index 710522cd7..0440b3b82 100644 --- a/main/quick-start/install/index.html +++ b/main/quick-start/install/index.html @@ -1,4 +1,4 @@ - Installation - Chainsaw
Skip to content

Installation

You can install the pre-compiled binary (in several ways), compile from sources, or run with Docker.

We also provide a GitHub action to easily install Chainsaw in your workflows.

Install the pre-compiled binary

Homebrew tap

add tap:

brew tap kyverno/chainsaw https://github.com/kyverno/chainsaw
+ Installation - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Installation
 
You can install the pre-compiled binary (in several ways), compile from sources, or run with Docker.
 
We also provide a GitHub action to easily install Chainsaw in your workflows.
 
Install the pre-compiled binary
 
Homebrew tap
 
add tap:
 
brew tap kyverno/chainsaw https://github.com/kyverno/chainsaw
 
 
install chainsaw:
 
brew install kyverno/chainsaw/chainsaw
 
 
 
Don't forget to specify the tap name
 
Homebrew core already has a tool named chainsaw.
 
Be sure that you specify the tap name when installing to install the right tool.
 
 
Manually
 
Download the pre-compiled binaries for your system from the releases page and copy them to the desired location.
 
Install using go install
 
You can install with go install with:
 
go install github.com/kyverno/chainsaw@latest
 
 
Run with Docker
 
Chainsaw is also available as a Docker image which you can pull and run:
 
docker pull ghcr.io/kyverno/chainsaw:<version>
@@ -20,4 +20,4 @@
   pkgs.kyverno-chainsaw
 ];
 
 
Using nix-shell
 
A nix-shell will temporarily modify your $PATH environment variable. This can be used to try a piece of software before deciding to permanently install it. Use the following command to install kyverno-chainsaw :
 
nix-shell -p kyverno-chainsaw
-
 
GitHub action
 
A GitHub action is available to install Chainsaw in your workflows. See the GitHub action dedicated documentation.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

GitHub action

A GitHub action is available to install Chainsaw in your workflows. See the GitHub action dedicated documentation.

\ No newline at end of file diff --git a/main/quick-start/next-steps/index.html b/main/quick-start/next-steps/index.html index a7a1dc306..dfd95d468 100644 --- a/main/quick-start/next-steps/index.html +++ b/main/quick-start/next-steps/index.html @@ -1 +1 @@ - Next steps - Chainsaw
Skip to content

Next steps

We covered the main features of Chainsaw in this Getting started sections.

While this should help you understand Chainsaw better, there are a lot of other things Chainsaw can do for you.

Tip

If there's anything you would like to be improved, please reach out, we will be happy to discuss and improve as much as we can.

To continue exploring the capabilities of Chainsaw:

Additional resources

Resources, blog posts and videos talking about Chainsaw:

Chainsaw review video

If you haven't watched the video below yet, we strongly recommend watching it to discover Chainsaw!

\ No newline at end of file + Next steps - Chainsaw
Skip to content

Next steps

We covered the main features of Chainsaw in this Getting started sections.

While this should help you understand Chainsaw better, there are a lot of other things Chainsaw can do for you.

Tip

If there's anything you would like to be improved, please reach out, we will be happy to discuss and improve as much as we can.

To continue exploring the capabilities of Chainsaw:

Additional resources

Resources, blog posts and videos talking about Chainsaw:

Chainsaw review video

If you haven't watched the video below yet, we strongly recommend watching it to discover Chainsaw!

\ No newline at end of file diff --git a/main/quick-start/operation-outputs/index.html b/main/quick-start/operation-outputs/index.html index ff9fd67ab..95b9e7bf7 100644 --- a/main/quick-start/operation-outputs/index.html +++ b/main/quick-start/operation-outputs/index.html @@ -1,4 +1,4 @@ - Use operation outputs - Chainsaw
Skip to content

Use operation outputs

Operation outputs can be useful for communicating and reusing computation results across operations.

Lifetime of outputs

Once an output has been added to the bindings context, this binding will be available to all following operations in the same step.

Currently, outputs do not cross the step boundaries.

Matching

An output supports an optional match field. The match is used to conditionally create a binding.

In the case of applying a file, for example, the file may contain multiple resources. The match can be used to select the resource to use for creating the binding.

Load an existing resource

The example below invokes a kubectl command to get a configmap from the cluster in json format.

The json output is then parsed and added to the $cm binding and the next operation performs an assertion on it by reading the binding instead of querying the cluster.

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Use operation outputs - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Use operation outputs
 
Operation outputs can be useful for communicating and reusing computation results across operations.
 
Lifetime of outputs
 
Once an output has been added to the bindings context, this binding will be available to all following operations in the same step.
 
Currently, outputs do not cross the step boundaries.
 
Matching
 
An output supports an optional match field. The match is used to conditionally create a binding.
 
In the case of applying a file, for example, the file may contain multiple resources. The match can be used to select the resource to use for creating the binding.
 
Load an existing resource
 
The example below invokes a kubectl command to get a configmap from the cluster in json format.
 
The json output is then parsed and added to the $cm binding and the next operation performs an assertion on it by reading the binding instead of querying the cluster.
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -37,4 +37,4 @@
         - name: UID
           value: ($cm.metadata.uid)
         content: echo $UID
-
 
Next step
 
In the next section, we will look at the three main elements of a test step, the try, catch and finally blocks.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Next step

In the next section, we will look at the three main elements of a test step, the try, catch and finally blocks.

\ No newline at end of file diff --git a/main/quick-start/resource-templating/index.html b/main/quick-start/resource-templating/index.html index 6329b845d..03d8c8c52 100644 --- a/main/quick-start/resource-templating/index.html +++ b/main/quick-start/resource-templating/index.html @@ -1,4 +1,4 @@ - Use resource templating - Chainsaw
Skip to content

Use resource templating

Chainsaw simplifies dynamic resource configuration with native resource templating support.

Sometimes things we need to create resources or assertions are only known at runtime.

In the past, users have created all sorts of hacks using tools like envsubst for dynamic substitution of env-variables. Those workarounds usually lack flexibility and introduce new problems like hiding the real resources from Chainsaw, preventing it from cleaning resources properly.

Tip

Resource templating is heavily based on bindings and uses JMESPath language.

Leverage bindings

In the template below, we are using the $namespace binding at two different places, effectively injecting the ephemeral namespace name in the name and the data.foo fields:

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Use resource templating - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Use resource templating
 
Chainsaw simplifies dynamic resource configuration with native resource templating support.
 
Sometimes things we need to create resources or assertions are only known at runtime.
 
In the past, users have created all sorts of hacks using tools like envsubst for dynamic substitution of env-variables. Those workarounds usually lack flexibility and introduce new problems like hiding the real resources from Chainsaw, preventing it from cleaning resources properly.
 
 
Tip
 
Resource templating is heavily based on bindings and uses JMESPath language.
 
 
Leverage bindings
 
In the template below, we are using the $namespace binding at two different places, effectively injecting the ephemeral namespace name in the name and the data.foo fields:
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -26,4 +26,4 @@
           name: (join('-', [$namespace, 'cm']))
         data:
           foo: bar
-
 
Next step
 
Combining bindings and templates with operation outputs allows even more flexibility to pass arbitrary data from one operation to another.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Next step

Combining bindings and templates with operation outputs allows even more flexibility to pass arbitrary data from one operation to another.

\ No newline at end of file diff --git a/main/quick-start/run-tests/index.html b/main/quick-start/run-tests/index.html index 419b66d9d..6c6b2629f 100644 --- a/main/quick-start/run-tests/index.html +++ b/main/quick-start/run-tests/index.html @@ -1,4 +1,4 @@ - Run tests - Chainsaw
Skip to content

Run tests

After installing chainsaw and writing tests, the next natural step is to run Chainsaw to execute the tests.

Create a local cluster

To use Chainsaw you will need a Kubernetes cluster, Chainsaw won't create one for you.

Not a cluster management tool

We consider this is not the responsibility of Chainsaw to manage clusters. There are plenty of solutions to create and manage local clusters that will do that better than Chainsaw.

The command below will create a local cluster using kind. Use the tool of your choice or directly jump to the next section if you already have a KUBECONFIG configured and pointing to a valid cluster.

# create cluster
+ Run tests - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Run tests
 
After installing chainsaw and writing tests, the next natural step is to run Chainsaw to execute the tests.
 
Create a local cluster
 
To use Chainsaw you will need a Kubernetes cluster, Chainsaw won't create one for you.
 
 
Not a cluster management tool
 
We consider this is not the responsibility of Chainsaw to manage clusters. There are plenty of solutions to create and manage local clusters that will do that better than Chainsaw.
 
 
The command below will create a local cluster using kind. Use the tool of your choice or directly jump to the next section if you already have a KUBECONFIG configured and pointing to a valid cluster.
 
# create cluster
 kind create cluster --image "kindest/node:v1.29.4"
 
 
Run Chainsaw
 
Now you can run the chainsaw test command.
 
> chainsaw test
 
@@ -51,4 +51,4 @@
 - Failed  tests 0
 - Skipped tests 0
 Done.
-
 
 
Tip
 
Chainsaw expects a path to the test folder and will discover tests by analyzing files recursively. When no path is provided Chainsaw will use the current path by default (.).
 
 
Next step
 
The test above demonstrates the most basic usage of Chainsaw. In the next sections, we will look at the main features that make Chainsaw a very unique tool.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Tip

Chainsaw expects a path to the test folder and will discover tests by analyzing files recursively. When no path is provided Chainsaw will use the current path by default (.).

Next step

The test above demonstrates the most basic usage of Chainsaw. In the next sections, we will look at the main features that make Chainsaw a very unique tool.

\ No newline at end of file diff --git a/main/quick-start/timeouts/index.html b/main/quick-start/timeouts/index.html index f91d02c6c..e125d42b2 100644 --- a/main/quick-start/timeouts/index.html +++ b/main/quick-start/timeouts/index.html @@ -1,4 +1,4 @@ - Control your timeouts - Chainsaw
Skip to content

Control your timeouts

Timeouts in Chainsaw are specified per type of operation. This is handy because the timeout varies greatly depending on the nature of an operation.

For example, applying a manifest in a cluster is expected to be reasonably fast, while validating a resource can be a long operation.

Inheritance

Timeouts can be configured globally and at the test, step or individual operation level.

All timeouts configured at a given level are automatically inherited in child levels. When looking up a timeout, the most specific one takes precedence over the others.

Info

To learn more about timeouts and how to configure global values, see the timeouts configuration page.

At the test level

When a timeout is configured at the test level it will apply to all operations and steps in the test, unless overridden at a more specific level.

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Control your timeouts - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Control your timeouts
 
Timeouts in Chainsaw are specified per type of operation. This is handy because the timeout varies greatly depending on the nature of an operation.
 
For example, applying a manifest in a cluster is expected to be reasonably fast, while validating a resource can be a long operation.
 
Inheritance
 
Timeouts can be configured globally and at the test, step or individual operation level.
 
All timeouts configured at a given level are automatically inherited in child levels. When looking up a timeout, the most specific one takes precedence over the others.
 
 
Info
 
To learn more about timeouts and how to configure global values, see the timeouts configuration page.
 
 
At the test level
 
When a timeout is configured at the test level it will apply to all operations and steps in the test, unless overridden at a more specific level.
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -105,4 +105,4 @@
           status:
             (conditions[?type == 'Ready']):
             - status: 'True'
-
 
Next step
 
In the next section, we will see how Chainsaw manages cleanup.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Next step

In the next section, we will see how Chainsaw manages cleanup.

\ No newline at end of file diff --git a/main/quick-start/try-catch/index.html b/main/quick-start/try-catch/index.html index 82cf4ea51..610f52830 100644 --- a/main/quick-start/try-catch/index.html +++ b/main/quick-start/try-catch/index.html @@ -1,4 +1,4 @@ - Use try, catch and finally - Chainsaw
Skip to content

Use try, catch and finally

A test step is made of 3 main blocks used to determine the actions Chainsaw will perform when executing the step, depending on the test outcome.

  • The try block (required)
  • The catch block (optional)
  • The finally block (optional)

Operations defined in the try block are executed first, then:

  • If an operation fails to execute, Chainsaw won't execute the remaining operations and will execute all operations defined in the catch block instead (if any).
  • If all operations succeed, Chainsaw will NOT execute operations defined in the catch block (if any).
  • Regardless of the step outcome (success or failure), Chainsaw will execute all operations defined in the finally block (if any).

Note

Note that all operations coming from the catch or finally blocks are executed. If one operation fails, Chainsaw will mark the test as failed and continue executing with the next operation.

Cleanup

At the end of a test, Chainsaw automatically cleans up the resources created during the test (cleanup is done in the opposite order of creation).

All operations from the catch and finally blocks are executed before the cleanup process kicks in. This order allows analyzing the resources that potentially caused the step failure before they are deleted.

Catch

Operations in a catch block are executed only when the step is considered failed.

This is particularly useful to collect additional information to help understand what caused the failure.

In the example below, the test contains a catch block to collect events in the cluster when an operation fails in the step.

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Use try, catch and finally - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Use try, catch and finally
 
A test step is made of 3 main blocks used to determine the actions Chainsaw will perform when executing the step, depending on the test outcome.
 
     
  • The try block (required)
    •  
  • The catch block (optional)
    •  
  • The finally block (optional)
    •  
 
Operations defined in the try block are executed first, then:
 
     
  • If an operation fails to execute, Chainsaw won't execute the remaining operations and will execute all operations defined in the catch block instead (if any).
    •  
  • If all operations succeed, Chainsaw will NOT execute operations defined in the catch block (if any).
    •  
  • Regardless of the step outcome (success or failure), Chainsaw will execute all operations defined in the finally block (if any).
    •  
 
 
Note
 
Note that all operations coming from the catch or finally blocks are executed. If one operation fails, Chainsaw will mark the test as failed and continue executing with the next operation.
 
 
Cleanup
 
At the end of a test, Chainsaw automatically cleans up the resources created during the test (cleanup is done in the opposite order of creation).
 
All operations from the catch and finally blocks are executed before the cleanup process kicks in. This order allows analyzing the resources that potentially caused the step failure before they are deleted.
 
Catch
 
Operations in a catch block are executed only when the step is considered failed.
 
This is particularly useful to collect additional information to help understand what caused the failure.
 
In the example below, the test contains a catch block to collect events in the cluster when an operation fails in the step.
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -38,4 +38,4 @@
     - script:
         content: |
           rm -f ./dynamic
-
 
Next step
 
Every operation in a test must be executed in a timely fashion. In the next section, we will see how you can control your timeouts.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Next step

Every operation in a test must be executed in a timely fashion. In the next section, we will see how you can control your timeouts.

\ No newline at end of file diff --git a/main/reference/index.html b/main/reference/index.html index b56d8b4d8..07746c3bc 100644 --- a/main/reference/index.html +++ b/main/reference/index.html @@ -1 +1 @@ - Reference documentation - Chainsaw
Skip to content

Reference documentation

\ No newline at end of file + Reference documentation - Chainsaw
Skip to content

Reference documentation

\ No newline at end of file diff --git a/main/search/search_index.json b/main/search/search_index.json index d62962085..719003b64 100644 --- a/main/search/search_index.json +++ b/main/search/search_index.json @@ -1 +1 @@ -{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"gh-action/","title":"GitHub action","text":"

A GitHub action is available to easily install Chainsaw in your workflows.

The GitHub action is available at kyverno/action-install-chainsaw or in the marketplace.

"},{"location":"gh-action/#usage","title":"Usage","text":"

This action currently supports GitHub-provided Linux, macOS and Windows runners (self-hosted runners may not work).

Add the following entry to your Github workflow YAML file:

uses: kyverno/action-install-chainsaw@v0.1.0\nwith:\n  release: v0.1.0 # optional\n

Example using a pinned version:

jobs:\n  example:\n    runs-on: ubuntu-latest\n\n    permissions: {}\n\n    name: Install Chainsaw\n    steps:\n      - name: Install Chainsaw\n        uses: kyverno/action-install-chainsaw@v0.1.0\n        with:\n          release: v0.0.9\n      - name: Check install\n        run: chainsaw version\n

Example using the default version:

jobs:\n  example:\n    runs-on: ubuntu-latest\n\n    permissions: {}\n\n    name: Install Chainsaw\n    steps:\n      - name: Install Chainsaw\n        uses: kyverno/action-install-chainsaw@v0.1.0\n      - name: Check install\n        run: chainsaw version\n

Example using cosign verification:

jobs:\n  example:\n    runs-on: ubuntu-latest\n\n    permissions: {}\n\n    name: Install Chainsaw\n    steps:\n      - name: Install Cosign\n        uses: sigstore/cosign-installer@v3.1.1\n      - name: Install Chainsaw\n        uses: kyverno/action-install-chainsaw@v0.1.0\n        with:\n          verify: true\n      - name: Check install\n        run: chainsaw version\n

If you want to install Chainsaw from its main version by using go install under the hood, you can set release as main. Once you did that, Chainsaw will be installed via go install which means that please ensure that go is installed.

Example of installing Chainsaw via go install:

jobs:\n  example:\n    runs-on: ubuntu-latest\n\n    permissions: {}\n\n    name: Install Chainsaw via go install\n    steps:\n      - name: Install go\n        uses: actions/setup-go@v4\n        with:\n          go-version: '1.21'\n      - name: Install Chainsaw\n        uses: kyverno/action-install-chainsaw@v0.1.0\n        with:\n          release: main\n      - name: Check install\n        run: chainsaw version\n
"},{"location":"gh-action/#optional-inputs","title":"Optional Inputs","text":"

The following optional inputs:

Input Description release chainsaw version to use instead of the default. install-dir directory to place the chainsaw binary into instead of the default ($HOME/.chainsaw). use-sudo set to true if install-dir location requires sudo privs. Defaults to false. verify set to true to enable cosign verification of the downloaded archive."},{"location":"json-schemas/","title":"JSON schemas","text":"

JSON schemas for Chainsaw resources are available to enable validation and autocompletion in your IDE:

"},{"location":"json-schemas/#vs-code","title":"VS code","text":"

In VS code, simply add a comment on top of your YAML resources.

"},{"location":"json-schemas/#test","title":"Test","text":"
# yaml-language-server: $schema=https://raw.githubusercontent.com/kyverno/chainsaw/main/.schemas/json/test-chainsaw-v1alpha1.json\napiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: basic\nspec:\n  steps:\n  - try:\n    - apply:\n        file: configmap.yaml\n    - assert:\n        file: configmap-assert.yaml\n
"},{"location":"json-schemas/#configuration","title":"Configuration","text":"
# yaml-language-server: $schema=https://raw.githubusercontent.com/kyverno/chainsaw/main/.schemas/json/configuration-chainsaw-v1alpha1.json\napiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Configuration\nmetadata:\n  name: congiguration\nspec:\n  fullName: true\n  failFast: true\n  forceTerminationGracePeriod: 5s\n
"},{"location":"json-schemas/#applying-crds","title":"Applying CRDs","text":"

An alternative is to apply the Chainsaw CRDs in a kubernetes cluster and let the Kubernetes extension do the rest.

CRD definitions are provided in our GitHub repository.

The command below will apply the Chainsaw CRDs to the configured cluster:

kubectl apply -f https://raw.githubusercontent.com/kyverno/chainsaw/main/config/crds/chainsaw.kyverno.io_configurations.yaml\nkubectl apply -f https://raw.githubusercontent.com/kyverno/chainsaw/main/config/crds/chainsaw.kyverno.io_tests.yaml\n
"},{"location":"json-schemas/#exporting-schemas","title":"Exporting schemas","text":"

Chainsaw can export JSON schemas locally.

chainsaw export schemas <local path>\n

See Chainsaw export schemas reference for more details.

"},{"location":"apis/chainsaw.v1alpha1/","title":"chainsaw (v1alpha1)","text":"

Package v1alpha1 contains API Schema definitions for the v1alpha1 API group.

"},{"location":"apis/chainsaw.v1alpha1/#resource-types","title":"Resource Types","text":""},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Configuration","title":"Configuration","text":"

Configuration is the resource that contains the configuration used to run tests.

Field Type Required Inline Description apiVersion string chainsaw.kyverno.io/v1alpha1 kind string Configuration metadata meta/v1.ObjectMeta

Standard object's metadata.

spec ConfigurationSpec

Configuration spec.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Test","title":"Test","text":"

Test is the resource that contains a test definition.

Field Type Required Inline Description apiVersion string chainsaw.kyverno.io/v1alpha1 kind string Test metadata meta/v1.ObjectMeta

Standard object's metadata.

spec TestSpec

Test spec.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Apply","title":"Apply","text":"

Appears in:

Apply represents a set of configurations or resources that should be applied during testing.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

outputs []Output

Outputs defines output bindings.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

FileRefOrResource FileRefOrResource

FileRefOrResource provides a reference to the resources to be applied.

template bool

Template determines whether resources should be considered for templating.

dryRun bool

DryRun determines whether the file should be applied in dry run mode.

expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Assert","title":"Assert","text":"

Appears in:

Assert represents a test condition that is expected to hold true during the testing process.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

FileRefOrCheck FileRefOrCheck

FileRefOrAssert provides a reference to the assertion.

template bool

Template determines whether resources should be considered for templating.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Binding","title":"Binding","text":"

Appears in:

Binding represents a key/value set as a binding in an executing test.

Field Type Required Inline Description name string

Name the name of the binding.

value policy/v1alpha1.Any

Value value of the binding.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Catch","title":"Catch","text":"

Appears in:

Catch defines actions to be executed on failure.

Field Type Required Inline Description description string

Description contains a description of the operation.

podLogs PodLogs

PodLogs determines the pod logs collector to execute.

events Events

Events determines the events collector to execute.

describe Describe

Describe determines the resource describe collector to execute.

wait Wait

Wait determines the resource wait collector to execute.

get Get

Get determines the resource get collector to execute.

delete Delete

Delete represents a deletion operation.

command Command

Command defines a command to run.

script Script

Script defines a script to run.

sleep Sleep

Sleep defines zzzz.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Cluster","title":"Cluster","text":"

Appears in:

Cluster defines cluster config and context.

Field Type Required Inline Description kubeconfig string

Kubeconfig is the path to the referenced file.

context string

Context is the name of the context to use.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Command","title":"Command","text":"

Appears in:

Command describes a command to run as a part of a test step.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

outputs []Output

Outputs defines output bindings.

env []Binding

Env defines additional environment variables.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

entrypoint string

Entrypoint is the command entry point to run.

args []string

Args is the command arguments.

skipLogOutput bool

SkipLogOutput removes the output from the command. Useful for sensitive logs or to reduce noise.

check policy/v1alpha1.Any

Check is an assertion tree to validate the operation outcome.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Condition","title":"Condition","text":"

Appears in:

Condition represents parameters for waiting on a specific condition of a resource.

Field Type Required Inline Description name string

Name defines the specific condition to wait for, e.g., \"Available\", \"Ready\".

value string

Value defines the specific condition status to wait for, e.g., \"True\", \"False\".

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-ConfigurationSpec","title":"ConfigurationSpec","text":"

Appears in:

ConfigurationSpec contains the configuration used to run tests.

Field Type Required Inline Description timeouts Timeouts

Global timeouts configuration. Applies to all tests/test steps if not overridden.

skipDelete bool

If set, do not delete the resources after running the tests (implies SkipClusterDelete).

template bool

Template determines whether resources should be considered for templating.

failFast bool

FailFast determines whether the test should stop upon encountering the first failure.

parallel int

The maximum number of tests to run at once.

reportFormat ReportFormatType

ReportFormat determines test report format (JSON reportPath string

ReportPath defines the path.

reportName string

ReportName defines the name of report to create. It defaults to \"chainsaw-report\".

namespace string

Namespace defines the namespace to use for tests. If not specified, every test will execute in a random ephemeral namespace unless the namespace is overridden in a the test spec.

namespaceTemplate policy/v1alpha1.Any

NamespaceTemplate defines a template to create the test namespace.

fullName bool

FullName makes use of the full test case folder path instead of the folder name.

excludeTestRegex string

ExcludeTestRegex is used to exclude tests based on a regular expression.

includeTestRegex string

IncludeTestRegex is used to include tests based on a regular expression.

repeatCount int

RepeatCount indicates how many times the tests should be executed.

testFile string

TestFile is the name of the file containing the test to run. If no extension is provided, chainsaw will try with .yaml first and .yml if needed.

forceTerminationGracePeriod meta/v1.Duration

ForceTerminationGracePeriod forces the termination grace period on pods, statefulsets, daemonsets and deployments.

delayBeforeCleanup meta/v1.Duration

DelayBeforeCleanup adds a delay between the time a test ends and the time cleanup starts.

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

catch []Catch

Catch defines what the tests steps will execute when an error happens. This will be combined with catch handlers defined at the test and step levels.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Create","title":"Create","text":"

Appears in:

Create represents a set of resources that should be created. If a resource already exists in the cluster it will fail.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

outputs []Output

Outputs defines output bindings.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

FileRefOrResource FileRefOrResource

FileRefOrResource provides a reference to the file containing the resources to be created.

template bool

Template determines whether resources should be considered for templating.

dryRun bool

DryRun determines whether the file should be applied in dry run mode.

expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Delete","title":"Delete","text":"

Appears in:

Delete is a reference to an object that should be deleted

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

template bool

Template determines whether resources should be considered for templating.

ref ObjectReference

ObjectReference determines objects to be deleted.

expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Deletion","title":"Deletion","text":"

Appears in:

Deletion represents parameters for waiting on a resource's deletion.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Describe","title":"Describe","text":"

Appears in:

Describe defines how to describe resources.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

ResourceReference ResourceReference

ResourceReference referenced resource type.

ObjectLabelsSelector ObjectLabelsSelector

ObjectLabelsSelector determines the selection process of referenced objects.

showEvents bool

Show Events indicates whether to include related events.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Error","title":"Error","text":"

Appears in:

Error represents an anticipated error condition that may arise during testing. Instead of treating such an error as a test failure, it acknowledges it as expected.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

FileRefOrCheck FileRefOrCheck

FileRefOrAssert provides a reference to the expected error.

template bool

Template determines whether resources should be considered for templating.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Events","title":"Events","text":"

Appears in:

Events defines how to collect events.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

ObjectLabelsSelector ObjectLabelsSelector

ObjectLabelsSelector determines the selection process of referenced objects.

format Format

Format determines the output format (json or yaml).

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Expectation","title":"Expectation","text":"

Appears in:

Expectation represents a check to be applied on the result of an operation with a match filter to determine if the verification should be considered.

Field Type Required Inline Description match policy/v1alpha1.Any

Match defines the matching statement.

check policy/v1alpha1.Any

Check defines the verification statement.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-FileRef","title":"FileRef","text":"

Appears in:

FileRef represents a file reference.

Field Type Required Inline Description file string

File is the path to the referenced file. This can be a direct path to a file or an expression that matches multiple files, such as \"manifest/*.yaml\" for all YAML files within the \"manifest\" directory.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-FileRefOrCheck","title":"FileRefOrCheck","text":"

Appears in:

FileRefOrCheck represents a file reference or resource.

Field Type Required Inline Description FileRef FileRef

FileRef provides a reference to the file containing the resources to be applied.

resource policy/v1alpha1.Any

Check provides a check used in assertions.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-FileRefOrResource","title":"FileRefOrResource","text":"

Appears in:

FileRefOrResource represents a file reference or resource.

Field Type Required Inline Description FileRef FileRef

FileRef provides a reference to the file containing the resources to be applied.

resource meta/v1/unstructured.Unstructured

Resource provides a resource to be applied.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Finally","title":"Finally","text":"

Appears in:

Finally defines actions to be executed at the end of a test.

Field Type Required Inline Description description string

Description contains a description of the operation.

podLogs PodLogs

PodLogs determines the pod logs collector to execute.

events Events

Events determines the events collector to execute.

describe Describe

Describe determines the resource describe collector to execute.

wait Wait

Wait determines the resource wait collector to execute.

get Get

Get determines the resource get collector to execute.

delete Delete

Delete represents a deletion operation.

command Command

Command defines a command to run.

script Script

Script defines a script to run.

sleep Sleep

Sleep defines zzzz.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-For","title":"For","text":"

Appears in:

For specifies the condition to wait for.

Field Type Required Inline Description deletion Deletion

Deletion specifies parameters for waiting on a resource's deletion.

condition Condition

Condition specifies the condition to wait for.

jsonPath JsonPath

JsonPath specifies the json path condition to wait for.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Format","title":"Format","text":"

(Alias of string)

Appears in:

Format determines the output format (json or yaml).

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Get","title":"Get","text":"

Appears in:

Get defines how to get resources.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

ResourceReference ResourceReference

ResourceReference referenced resource type.

ObjectLabelsSelector ObjectLabelsSelector

ObjectLabelsSelector determines the selection process of referenced objects.

format Format

Format determines the output format (json or yaml).

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-JsonPath","title":"JsonPath","text":"

Appears in:

JsonPath represents parameters for waiting on a json path of a resource.

Field Type Required Inline Description path string

Path defines the json path to wait for, e.g. '{.status.phase}'.

value string

Value defines the expected value to wait for, e.g., \"Running\".

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-ObjectLabelsSelector","title":"ObjectLabelsSelector","text":"

Appears in:

ObjectLabelsSelector represents a strategy to select objects. For a single object name and namespace are used to identify the object. For multiple objects use selector.

Field Type Required Inline Description namespace string

Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

name string

Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

selector string

Selector defines labels selector.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-ObjectReference","title":"ObjectReference","text":"

Appears in:

ObjectReference represents one or more objects with a specific apiVersion and kind. For a single object name and namespace are used to identify the object. For multiple objects use labels.

Field Type Required Inline Description ObjectType ObjectType

ObjectType determines the type of referenced objects.

ObjectSelector ObjectSelector

ObjectSelector determines the selection process of referenced objects.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-ObjectSelector","title":"ObjectSelector","text":"

Appears in:

ObjectSelector represents a strategy to select objects. For a single object name and namespace are used to identify the object. For multiple objects use labels.

Field Type Required Inline Description namespace string

Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

name string

Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

labels map[string]string

Label selector to match objects to delete

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-ObjectType","title":"ObjectType","text":"

Appears in:

ObjectType represents a specific apiVersion and kind.

Field Type Required Inline Description apiVersion string

API version of the referent.

kind string

Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Operation","title":"Operation","text":"

Appears in:

Operation defines a single operation, only one action is permitted for a given operation.

Field Type Required Inline Description description string

Description contains a description of the operation.

continueOnError bool

ContinueOnError determines whether a test should continue or not in case the operation was not successful. Even if the test continues executing, it will still be reported as failed.

apply Apply

Apply represents resources that should be applied for this test step. This can include things like configuration settings or any other resources that need to be available during the test.

assert Assert

Assert represents an assertion to be made. It checks whether the conditions specified in the assertion hold true.

command Command

Command defines a command to run.

create Create

Create represents a creation operation.

delete Delete

Delete represents a deletion operation.

error Error

Error represents the expected errors for this test step. If any of these errors occur, the test will consider them as expected; otherwise, they will be treated as test failures.

patch Patch

Patch represents a patch operation.

script Script

Script defines a script to run.

sleep Sleep

Sleep defines zzzz.

update Update

Update represents an update operation.

wait Wait

Wait determines the resource wait collector to execute.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Output","title":"Output","text":"

Appears in:

Output represents an output binding with a match to determine if the binding must be considered or not.

Field Type Required Inline Description Binding Binding

Binding determines the binding to create when the match succeeds.

match policy/v1alpha1.Any

Match defines the matching statement.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Patch","title":"Patch","text":"

Appears in:

Patch represents a set of resources that should be patched. If a resource doesn't exist yet in the cluster it will fail.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

outputs []Output

Outputs defines output bindings.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

FileRefOrResource FileRefOrResource

FileRefOrResource provides a reference to the file containing the resources to be patched.

template bool

Template determines whether resources should be considered for templating.

dryRun bool

DryRun determines whether the file should be applied in dry run mode.

expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-PodLogs","title":"PodLogs","text":"

Appears in:

PodLogs defines how to collect pod logs.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

ObjectLabelsSelector ObjectLabelsSelector

ObjectLabelsSelector determines the selection process of referenced objects.

container string

Container in pod to get logs from else --all-containers is used.

tail int

Tail is the number of last lines to collect from pods. If omitted or zero, then the default is 10 if you use a selector, or -1 (all) if you use a pod name. This matches default behavior of kubectl logs.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-ReportFormatType","title":"ReportFormatType","text":"

(Alias of string)

Appears in:

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-ResourceReference","title":"ResourceReference","text":"

Appears in:

ResourceReference represents a resource (API), it can be represented with a resource or a kind. Optionally an apiVersion can be specified.

Field Type Required Inline Description apiVersion string

API version of the referent.

kind string

Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

resource string

Resource name of the referent.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Script","title":"Script","text":"

Appears in:

Script describes a script to run as a part of a test step.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

outputs []Output

Outputs defines output bindings.

env []Binding

Env defines additional environment variables.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

content string

Content defines a shell script (run with \"sh -c ...\").

skipLogOutput bool

SkipLogOutput removes the output from the command. Useful for sensitive logs or to reduce noise.

check policy/v1alpha1.Any

Check is an assertion tree to validate the operation outcome.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Sleep","title":"Sleep","text":"

Appears in:

Sleep represents a duration while nothing happens.

Field Type Required Inline Description duration meta/v1.Duration

Duration is the delay used for sleeping.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-TestSpec","title":"TestSpec","text":"

Appears in:

TestSpec contains the test spec.

Field Type Required Inline Description description string

Description contains a description of the test.

timeouts Timeouts

Timeouts for the test. Overrides the global timeouts set in the Configuration on a per operation basis.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

skip bool

Skip determines whether the test should skipped.

concurrent bool

Concurrent determines whether the test should run concurrently with other tests.

skipDelete bool

SkipDelete determines whether the resources created by the test should be deleted after the test is executed.

template bool

Template determines whether resources should be considered for templating.

namespace string

Namespace determines whether the test should run in a random ephemeral namespace or not.

namespaceTemplate policy/v1alpha1.Any

NamespaceTemplate defines a template to create the test namespace.

bindings []Binding

Bindings defines additional binding key/values.

steps []TestStep

Steps defining the test.

catch []Catch

Catch defines what the steps will execute when an error happens. This will be combined with catch handlers defined at the step level.

forceTerminationGracePeriod meta/v1.Duration

ForceTerminationGracePeriod forces the termination grace period on pods, statefulsets, daemonsets and deployments.

delayBeforeCleanup meta/v1.Duration

DelayBeforeCleanup adds a delay between the time a test ends and the time cleanup starts.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-TestStep","title":"TestStep","text":"

Appears in:

TestStep contains the test step definition used in a test spec.

Field Type Required Inline Description name string

Name of the step.

TestStepSpec TestStepSpec

TestStepSpec of the step.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-TestStepSpec","title":"TestStepSpec","text":"

Appears in:

TestStepSpec defines the desired state and behavior for each test step.

Field Type Required Inline Description description string

Description contains a description of the test step.

timeouts Timeouts

Timeouts for the test step. Overrides the global timeouts set in the Configuration and the timeouts eventually set in the Test.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

skipDelete bool

SkipDelete determines whether the resources created by the step should be deleted after the test step is executed.

template bool

Template determines whether resources should be considered for templating.

bindings []Binding

Bindings defines additional binding key/values.

try []Operation

Try defines what the step will try to execute.

catch []Catch

Catch defines what the step will execute when an error happens.

finally []Finally

Finally defines what the step will execute after the step is terminated.

cleanup []Finally

Cleanup defines what will be executed after the test is terminated.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Timeouts","title":"Timeouts","text":"

Appears in:

Timeouts contains timeouts per operation.

Field Type Required Inline Description apply meta/v1.Duration

Apply defines the timeout for the apply operation

assert meta/v1.Duration

Assert defines the timeout for the assert operation

cleanup meta/v1.Duration

Cleanup defines the timeout for the cleanup operation

delete meta/v1.Duration

Delete defines the timeout for the delete operation

error meta/v1.Duration

Error defines the timeout for the error operation

exec meta/v1.Duration

Exec defines the timeout for exec operations

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Update","title":"Update","text":"

Appears in:

Update represents a set of resources that should be updated. If a resource does not exist in the cluster it will fail.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

outputs []Output

Outputs defines output bindings.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

FileRefOrResource FileRefOrResource

FileRefOrResource provides a reference to the file containing the resources to be created.

template bool

Template determines whether resources should be considered for templating.

dryRun bool

DryRun determines whether the file should be applied in dry run mode.

expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Wait","title":"Wait","text":"

Appears in:

Wait specifies how to perform wait operations on resources.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Specifies how long to wait for the condition to be met before timing out.

cluster string

Cluster defines the target cluster where the wait operation will be performed (default cluster will be used if not specified).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

ResourceReference ResourceReference

ResourceReference referenced resource type.

ObjectLabelsSelector ObjectLabelsSelector

ObjectLabelsSelector determines the selection process of referenced objects.

for For

For specifies the condition to wait for.

format Format

Format determines the output format (json or yaml).

"},{"location":"apis/chainsaw.v1alpha2/","title":"chainsaw (v1alpha2)","text":"

Package v1alpha2 contains API Schema definitions for the v1alpha2 API group.

"},{"location":"apis/chainsaw.v1alpha2/#resource-types","title":"Resource Types","text":""},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-Configuration","title":"Configuration","text":"

Configuration is the resource that contains the configuration used to run tests.

Field Type Required Inline Description apiVersion string chainsaw.kyverno.io/v1alpha2 kind string Configuration metadata meta/v1.ObjectMeta

Standard object's metadata.

spec ConfigurationSpec

Configuration spec.

"},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-Cleanup","title":"Cleanup","text":"

Appears in:

Cleanup contains the cleanup configuration.

Field Type Required Inline Description skipDelete bool

If set, do not delete the resources after running a test.

delayBeforeCleanup meta/v1.Duration

DelayBeforeCleanup adds a delay between the time a test ends and the time cleanup starts.

"},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-ConfigurationSpec","title":"ConfigurationSpec","text":"

Appears in:

ConfigurationSpec contains the configuration used to run tests.

Field Type Required Inline Description catch []Catch

Catch defines what the tests steps will execute when an error happens. This will be combined with catch handlers defined at the test and step levels.

cleanup Cleanup

Cleanup contains cleanup configuration.

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

discovery Discovery

Discovery contains tests discovery configuration.

execution Execution

Execution contains tests execution configuration.

namespace Namespace

Namespace contains properties for the namespace to use for tests.

report Report

Report contains properties for the report.

templating Templating

Templating contains the templating config.

timeouts Timeouts

Global timeouts configuration. Applies to all tests/test steps if not overridden.

"},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-Discovery","title":"Discovery","text":"

Appears in:

Discovery contains the tests discovery configuration.

Field Type Required Inline Description excludeTestRegex string

ExcludeTestRegex is used to exclude tests based on a regular expression.

includeTestRegex string

IncludeTestRegex is used to include tests based on a regular expression.

testFile string

TestFile is the name of the file containing the test to run. If no extension is provided, chainsaw will try with .yaml first and .yml if needed.

fullName bool

FullName makes use of the full test case folder path instead of the folder name.

"},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-Execution","title":"Execution","text":"

Appears in:

Execution contains the runner configuration.

Field Type Required Inline Description failFast bool

FailFast determines whether the test should stop upon encountering the first failure.

parallel int

The maximum number of tests to run at once.

repeatCount int

RepeatCount indicates how many times the tests should be executed.

forceTerminationGracePeriod meta/v1.Duration

ForceTerminationGracePeriod forces the termination grace period on pods, statefulsets, daemonsets and deployments.

"},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-Namespace","title":"Namespace","text":"

Appears in:

Namespace contains info about the namespace used for testing.

Field Type Required Inline Description name string

Name defines the namespace to use for tests. If not specified, every test will execute in a random ephemeral namespace unless the namespace is overridden in a the test spec.

template policy/v1alpha1.Any

Template defines a template to create the test namespace.

"},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-Report","title":"Report","text":"

Appears in:

Report contains info about the report.

Field Type Required Inline Description format ReportFormatType

ReportFormat determines test report format (JSON path string

ReportPath defines the path.

name string

ReportName defines the name of report to create. It defaults to \"chainsaw-report\".

"},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-ReportFormatType","title":"ReportFormatType","text":"

(Alias of string)

Appears in:

"},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-Templating","title":"Templating","text":"

Appears in:

Templating contains the templating configuration.

Field Type Required Inline Description enabled bool

Enabled determines whether resources should be considered for templating.

"},{"location":"bindings/","title":"Bindings","text":"

Chainsaw has a concept of bindings which can be seen as an execution context.

Bindings are used in assertion trees and resource templating, as well as when using the --values flag when invoking chainsaw.

While some bindings are built-in and provided by chainsaw, it's possible to define custom bindings at the test, step, or individual operation level. Those bindings can in turn be used to create custom environment variables in command and script operations.

"},{"location":"bindings/#immutability","title":"Immutability","text":"

It's important to note that bindings are immutable, it's not possible to overwrite a binding and two bindings with the same name can exist without overwriting each other.

When a binding value is resolved, chainsaw will walk the binding chain upwards until it finds a binding with the expected name. Effectively, the last one registered in the chain will be used.

"},{"location":"bindings/#templating","title":"Templating","text":"

A binding name supports templating.

The name of a binding can therefore be dynamic and depend on values or other bindings.

"},{"location":"bindings/#usage","title":"Usage","text":"

The example below defines custom bindings at the test level.

Test level bindings

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  # bindings defined at the test level are available to all steps and operations\n  bindings:\n  - name: hello\n    value: hello\n  - name: chainsaw\n    value: chainsaw\n  # `greetings` depends on `$hello` and `$chainsaw` bindings defined above\n  - name: greetings\n    value: (join(' ', [$hello, $chainsaw]))\n  steps:\n  - try:\n    - script:\n        # custom environment variables, defined using custom bindings\n        env:\n        - name: GREETINGS\n          value: ($greetings)\n        content: echo $GREETINGS\n        check:\n          ($error): ~\n          ($stdout): hello chainsaw\n

The example below is similar to the previous one but also defines custom bindings at the step level.

Step level bindings

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  # bindings defined at the test level are available to all steps and operations\n  bindings:\n  - name: hello\n    value: hello\n  - name: chainsaw\n    value: chainsaw\n  steps:\n  - bindings:\n    # `greetings` depends on `$hello` and `$chainsaw` bindings defined at higher levels\n    - name: greetings\n      value: (join(' ', [$hello, $chainsaw]))\n    try:\n    - script:\n        # custom environment variables, defined using custom bindings\n        env:\n        - name: GREETINGS\n          value: ($greetings)\n        content: echo $GREETINGS\n        check:\n          ($error): ~\n          ($stdout): hello chainsaw\n

Finally, custom bindings can also be defined at the operation level.

Operation level bindings

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  # bindings defined at the test level are available to all steps and operations\n  bindings:\n  - name: hello\n    value: hello\n  steps:\n  - bindings:\n    - name: chainsaw\n      value: chainsaw\n    try:\n    - script:\n        bindings:\n        # `greetings` depends on `$hello` and `$chainsaw` bindings defined at the higher levels\n        - name: greetings\n          value: (join(' ', [$hello, $chainsaw]))\n        # custom environment variables, defined using custom bindings\n        env:\n        - name: GREETINGS\n          value: ($greetings)\n        content: echo $GREETINGS\n        check:\n          ($error): ~\n          ($stdout): hello chainsaw\n
"},{"location":"bindings/#outputs","title":"Outputs","text":"

Under certain conditions, bindings can also be used to add computed results to the context.

See Outputs for details.

"},{"location":"bindings/outputs/","title":"Outputs","text":"

Binding outputs can be useful to communicate and reuse computation results between operations.

"},{"location":"bindings/outputs/#supported-operations","title":"Supported operations","text":"

Currently, only script and command operations support outputs.

"},{"location":"bindings/outputs/#lifetime-of-outputs","title":"Lifetime of outputs","text":"

Once an output has been added in the form of a binding, this binding will be available to all following operations in the same step.

Currently, outputs do not cross the step boundaries.

"},{"location":"bindings/outputs/#matching","title":"Matching","text":"

An output supports an optional match field. The match is used to conditionally create a binding.

In the case of applying a file, for example, the file may contain multiple resources. The match can be used to select the resource to use for creating the binding.

"},{"location":"bindings/outputs/#examples","title":"Examples","text":"

The example below defines invokes a kubectl command to get a configmap from the cluster in json format.

The json output is then parsed and added to the $cm binding and the next operation performs an assertion on it by reading the binding instead of querying the cluster.

Output in script

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  # ...\n  - try:\n    - script:\n        content: kubectl get cm quick-start -n $NAMESPACE -o json\n        outputs:\n        - match:\n            (json_parse($stdout)):\n              apiVersion: v1\n              kind: ConfigMap\n          name: cm\n          value: (json_parse($stdout))\n    - assert:\n        resource:\n          ($cm):\n            metadata:\n              (uid != null): true\n  # ...\n

Output in command

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  # ...\n  - try:\n    - command:\n        entrypoint: kubectl\n        args:\n        - get\n        - cm\n        - quick-start\n        - -n\n        - $NAMESPACE\n        - -o\n        - json\n        outputs:\n        - match:\n            (json_parse($stdout)):\n              apiVersion: v1\n              kind: ConfigMap\n          name: cm\n          value: (json_parse($stdout))\n    - assert:\n        resource:\n          ($cm):\n            metadata:\n              (uid != null): true\n  # ...\n
"},{"location":"collectors/","title":"Collectors","text":""},{"location":"collectors/#purpose","title":"Purpose","text":"

The purpose of collectors is to collect certain information about the outcome of a step should it fail (in the case of catch) or at the end of the step (in the case of finally).

The ultimate goal of collectors is to gather information about the failure of a step and therefore help understand what caused it to fail.

A test step can have an arbitrary number of collectors.

"},{"location":"collectors/#available-collectors","title":"Available collectors","text":""},{"location":"collectors/#templating","title":"Templating","text":"

All string fields in collectors support templating.

"},{"location":"collectors/describe/","title":"Describe","text":"

Describing resources present in the cluster can help understand what happened and troubleshoot test failures.

"},{"location":"collectors/describe/#configuration","title":"Configuration","text":"

Reference documentation

Deprecated syntax

You can specify the resource directly instead of using apiVersion and kind.

This is a deprecated syntax though and will be removed in a future version.

"},{"location":"collectors/describe/#clustered-resources","title":"Clustered resources","text":"

When used with a clustered resource, the namespace is ignored and is not added to the corresponding kubectl command.

"},{"location":"collectors/describe/#all-namespaces","title":"All namespaces","text":"

When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.

"},{"location":"collectors/describe/#usage-examples","title":"Usage examples","text":""},{"location":"collectors/describe/#describe-pods","title":"Describe pods","text":"

If a name is specified, Chainsaw will describe resources that have a name starting with the provided name in the test namespace (or in the cluster if it is a clustered-level resource).

Describe pods in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n    # ...\n    finally:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n    # ...\n

If a namespace is specified, Chainsaw will describe resources in the specified namespace.

Describe pods in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        namespace: foo\n    # ...\n    finally:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        namespace: foo\n    # ...\n
"},{"location":"collectors/describe/#label-selector","title":"Label selector","text":"

An optional label selector can be configured to refine the resources to be described.

Describe pods using a label selector in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        selector: app=my-app\n    # ...\n    finally:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        selector: app=my-app\n    # ...\n

If a namespace is specified, Chainsaw will describe resources in the specified namespace.

Describe pods using a label selector in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        selector: app=my-app\n        namespace: foo\n    # ...\n    finally:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        selector: app=my-app\n        namespace: foo\n    # ...\n
"},{"location":"collectors/describe/#show-events","title":"Show events","text":"

The showEvents field can be used to enable or disable showing events when describing resources.

Default

By default, showEventsis true.

Do not show events

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        namespace: foo\n        showEvents: false\n    # ...\n    finally:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        namespace: foo\n        showEvents: false\n    # ...\n
"},{"location":"collectors/events/","title":"Events","text":"

Collecting namespace events can help understand what happened inside the cluster.

"},{"location":"collectors/events/#configuration","title":"Configuration","text":"

Reference documentation

"},{"location":"collectors/events/#all-namespaces","title":"All namespaces","text":"

When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.

"},{"location":"collectors/events/#usage-examples","title":"Usage examples","text":""},{"location":"collectors/events/#single-event","title":"Single event","text":"

If a name is specified, Chainsaw will retrieve the specified event in the test namespace.

Collect event in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - events:\n        name: my-event\n    # ...\n    finally:\n    - events:\n        name: my-event\n    # ...\n

If a namespace is specified, Chainsaw will retrieve the specified event in the specified namespace.

Collect event in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - events:\n        name: my-event\n        namespace: foo\n    # ...\n    finally:\n    - events:\n        name: my-event\n        namespace: foo\n    # ...\n
"},{"location":"collectors/events/#all-events","title":"All events","text":"

If no name and namespace are specified, Chainsaw will retrieve all events in the test namespace.

Collect all events in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - events: {}\n    # ...\n    finally:\n    - events: {}\n    # ...\n

On the other hand, if a namespace is specified, Chainsaw will retrieve all events in the specified namespace.

Collect all events in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - events:\n        namespace: foo\n    # ...\n    finally:\n    - events:\n        namespace: foo\n    # ...\n
"},{"location":"collectors/events/#label-selector","title":"Label selector","text":"

An optional label selector can be configured to refine the events to be retrieved.

Collect events using a label selector in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - events:\n        selector: app=my-app\n    # ...\n    finally:\n    - events:\n        selector: app=my-app\n    # ...\n

If a namespace is specified, Chainsaw will retrieve events using the specified namespace.

Collect events using a label selector in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - events:\n        selector: app=my-app\n        namespace: foo\n    # ...\n    finally:\n    - events:\n        selector: app=my-app\n        namespace: foo\n    # ...\n
"},{"location":"collectors/events/#format","title":"Format","text":"

An optional format can be specified. Supported formats are json and yaml.

If format is not specified, results will be returned in text format.

Use json format

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - events:\n        format: json\n    # ...\n    finally:\n    - events:\n        format: json\n    # ...\n

Use yaml format

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - events:\n        format: yaml\n    # ...\n    finally:\n    - events:\n        format: yaml\n    # ...\n
"},{"location":"collectors/get/","title":"Get","text":"

The get collector is used to list and print resources in the cluster.

"},{"location":"collectors/get/#configuration","title":"Configuration","text":"

Reference documentation

Deprecated syntax

You can specify the resource directly instead of using apiVersion and kind.

This is a deprecated syntax though and will be removed in a future version.

"},{"location":"collectors/get/#clustered-resources","title":"Clustered resources","text":"

When used with a clustered resource, the namespace is ignored and is not added to the corresponding kubectl command.

"},{"location":"collectors/get/#all-namespaces","title":"All namespaces","text":"

When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.

"},{"location":"collectors/get/#usage-examples","title":"Usage examples","text":""},{"location":"collectors/get/#single-resource","title":"Single resource","text":"

If a name is specified, Chainsaw will retrieve the specified resource in the test namespace.

Get pod in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n    # ...\n    finally:\n    - get:\n        resource: pods\n        name: my-pod\n    # ...\n

If a namespace is specified, Chainsaw will retrieve the specified resource in the specified namespace.

Collect pod in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n        namespace: foo\n    # ...\n    finally:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n        namespace: foo\n    # ...\n
"},{"location":"collectors/get/#all-resources","title":"All resources","text":"

If no name and namespace are specified, Chainsaw will retrieve all resources in the test namespace.

Collect all resources in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - get:\n        apiVersion: v1\n        kind: Pod\n    # ...\n    finally:\n    - get:\n        apiVersion: v1\n        kind: Pod\n    # ...\n

On the other hand, if a namespace is specified, Chainsaw will retrieve all resources in the specified namespace.

Collect all resources in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        namespace: foo\n    # ...\n    finally:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        namespace: foo\n    # ...\n
"},{"location":"collectors/get/#label-selector","title":"Label selector","text":"

An optional label selector can be configured to refine the resources to be retrieved.

Collect resources using a label selector in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        selector: app=my-app\n    # ...\n    finally:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        selector: app=my-app\n    # ...\n

If a namespace is specified, Chainsaw will retrieve resources using the specified namespace.

Collect resources using a label selector in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        selector: app=my-app\n        namespace: foo\n    # ...\n    finally:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        selector: app=my-app\n        namespace: foo\n    # ...\n
"},{"location":"collectors/get/#format","title":"Format","text":"

An optional format can be specified. Supported formats are json and yaml.

If format is not specified, results will be returned in text format.

Use json format

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        format: json\n    # ...\n    finally:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        format: json\n    # ...\n

Use yaml format

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        format: yaml\n    # ...\n    finally:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        format: yaml\n    # ...\n
"},{"location":"collectors/pod-logs/","title":"Pod logs","text":"

Collecting pod logs can help understand what happened inside one or more pods.

"},{"location":"collectors/pod-logs/#configuration","title":"Configuration","text":"

Reference documentation

"},{"location":"collectors/pod-logs/#all-namespaces","title":"All namespaces","text":"

When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.

"},{"location":"collectors/pod-logs/#usage-examples","title":"Usage examples","text":""},{"location":"collectors/pod-logs/#single-pod","title":"Single pod","text":"

If a pod name is specified, Chainsaw will retrieve logs from this specific pod in the test namespace.

Collect pod logs in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - podLogs:\n        name: my-pod\n    # ...\n    finally:\n    - podLogs:\n        name: my-pod\n    # ...\n

If a namespace is specified, Chainsaw will retrieve logs from this specific pod in the specified namespace.

Collect pod logs in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - podLogs:\n        name: my-pod\n        namespace: foo\n    # ...\n    finally:\n    - podLogs:\n        name: my-pod\n        namespace: foo\n    # ...\n
"},{"location":"collectors/pod-logs/#all-pods","title":"All pods","text":"

If no pod name and namespace is specified, Chainsaw will retrieve logs from all pods in the test namespace.

Collect all pod logs in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - podLogs: {}\n    # ...\n    finally:\n    - podLogs: {}\n    # ...\n

On the other hand, if a namespace is specified, Chainsaw will retrieve logs from all pods in the specified namespace.

Collect all pod logs in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - podLogs:\n        namespace: foo\n    # ...\n    finally:\n    - podLogs:\n        namespace: foo\n    # ...\n
"},{"location":"collectors/pod-logs/#label-selector","title":"Label selector","text":"

An optional label selector can be configured to refine the pods to retrieve logs from.

Collect pod logs using a label selector in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - podLogs:\n        selector: app=my-app\n    # ...\n    finally:\n    - podLogs:\n        selector: app=my-app\n    # ...\n

If a namespace is specified, Chainsaw will retrieve pod logs using the specified namespace.

Collect pod logs using a label selector in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - podLogs:\n        selector: app=my-app\n        namespace: foo\n    # ...\n    finally:\n    - podLogs:\n        selector: app=my-app\n        namespace: foo\n    # ...\n
"},{"location":"collectors/pod-logs/#tail","title":"Tail","text":"

The tail field can be used to limit the amount of log lines retrieved when querying pod logs.

Default

By default, tail will be 10 when a label selector is used, and all if a pod name is specified.

Tail example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - podLogs:\n        selector: app=my-app\n        namespace: foo\n        tail: 30\n    # ...\n    finally:\n    - podLogs:\n        selector: app=my-app\n        namespace: foo\n        tail: 30\n    # ...\n
"},{"location":"collectors/pod-logs/#container","title":"Container","text":"

The container field can be used to retrieve logs from a specific container in the pod.

Default

By default logs from all containers will be fetched.

Container example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - podLogs:\n        selector: app=my-app\n        namespace: foo\n        container: nginx\n    # ...\n    finally:\n    - podLogs:\n        selector: app=my-app\n        namespace: foo\n        container: nginx\n    # ...\n
"},{"location":"commands/chainsaw/","title":"Chainsaw","text":""},{"location":"commands/chainsaw/#chainsaw","title":"chainsaw","text":"

Stronger tool for e2e testing

chainsaw [flags]\n
"},{"location":"commands/chainsaw/#options","title":"Options","text":"
  -h, --help   help for chainsaw\n
"},{"location":"commands/chainsaw/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_assert/","title":"Chainsaw assert","text":""},{"location":"commands/chainsaw_assert/#chainsaw-assert","title":"chainsaw assert","text":"

Evaluate assertion

chainsaw assert [flags] [FILE]\n
"},{"location":"commands/chainsaw_assert/#options","title":"Options","text":"
      --clustered                           Defines if the resource is clustered (only applies when resource is loaded from a file)\n  -f, --file string                         Path to the file to assert or '-' to read from stdin\n  -h, --help                                help for assert\n      --kube-as string                      Username to impersonate for the operation\n      --kube-as-group stringArray           Group to impersonate for the operation, this flag can be repeated to specify multiple groups.\n      --kube-as-uid string                  UID to impersonate for the operation\n      --kube-certificate-authority string   Path to a cert file for the certificate authority\n      --kube-client-certificate string      Path to a client certificate file for TLS\n      --kube-client-key string              Path to a client key file for TLS\n      --kube-cluster string                 The name of the kubeconfig cluster to use\n      --kube-context string                 The name of the kubeconfig context to use\n      --kube-disable-compression            If true, opt-out of response compression for all requests to the server\n      --kube-insecure-skip-tls-verify       If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure\n  -n, --kube-namespace string               If present, the namespace scope for this CLI request\n      --kube-password string                Password for basic authentication to the API server\n      --kube-proxy-url string               If provided, this URL will be used to connect via proxy\n      --kube-request-timeout string         The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default \"0\")\n      --kube-server string                  The address and port of the Kubernetes API server\n      --kube-tls-server-name string         If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.\n      --kube-token string                   Bearer token for authentication to the API server\n      --kube-user string                    The name of the kubeconfig user to use\n      --kube-username string                Username for basic authentication to the API server\n      --namespace string                    Namespace to use (default \"default\")\n      --no-color                            Removes output colors\n  -r, --resource string                     Path to the file containing the resource\n      --timeout duration                    The assert timeout to use (default 30s)\n
"},{"location":"commands/chainsaw_assert/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_build/","title":"Chainsaw build","text":""},{"location":"commands/chainsaw_build/#chainsaw-build","title":"chainsaw build","text":"

Build commands

chainsaw build [flags]\n
"},{"location":"commands/chainsaw_build/#options","title":"Options","text":"
  -h, --help   help for build\n
"},{"location":"commands/chainsaw_build/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_build_docs/","title":"Chainsaw build docs","text":""},{"location":"commands/chainsaw_build_docs/#chainsaw-build-docs","title":"chainsaw build docs","text":"

Build tests documentation

chainsaw build docs [flags]\n
"},{"location":"commands/chainsaw_build_docs/#options","title":"Options","text":"
      --catalog string         Path to the built test catalog file\n  -h, --help                   help for docs\n      --readme-file string     Name of the built docs file (default \"README.md\")\n      --test-dir stringArray   Directories containing test cases to run\n      --test-file string       Name of the test file (default \"chainsaw-test\")\n
"},{"location":"commands/chainsaw_build_docs/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_completion/","title":"Chainsaw completion","text":""},{"location":"commands/chainsaw_completion/#chainsaw-completion","title":"chainsaw completion","text":"

Generate the autocompletion script for the specified shell

"},{"location":"commands/chainsaw_completion/#synopsis","title":"Synopsis","text":"

Generate the autocompletion script for chainsaw for the specified shell. See each sub-command's help for details on how to use the generated script.

"},{"location":"commands/chainsaw_completion/#options","title":"Options","text":"
  -h, --help   help for completion\n
"},{"location":"commands/chainsaw_completion/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_completion_bash/","title":"Chainsaw completion bash","text":""},{"location":"commands/chainsaw_completion_bash/#chainsaw-completion-bash","title":"chainsaw completion bash","text":"

Generate the autocompletion script for bash

"},{"location":"commands/chainsaw_completion_bash/#synopsis","title":"Synopsis","text":"

Generate the autocompletion script for the bash shell.

This script depends on the 'bash-completion' package. If it is not installed already, you can install it via your OS's package manager.

To load completions in your current shell session:

source <(chainsaw completion bash)\n

To load completions for every new session, execute once:

"},{"location":"commands/chainsaw_completion_bash/#linux","title":"Linux:","text":"
chainsaw completion bash > /etc/bash_completion.d/chainsaw\n
"},{"location":"commands/chainsaw_completion_bash/#macos","title":"macOS:","text":"
chainsaw completion bash > $(brew --prefix)/etc/bash_completion.d/chainsaw\n

You will need to start a new shell for this setup to take effect.

chainsaw completion bash\n
"},{"location":"commands/chainsaw_completion_bash/#options","title":"Options","text":"
  -h, --help              help for bash\n      --no-descriptions   disable completion descriptions\n
"},{"location":"commands/chainsaw_completion_bash/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_completion_fish/","title":"Chainsaw completion fish","text":""},{"location":"commands/chainsaw_completion_fish/#chainsaw-completion-fish","title":"chainsaw completion fish","text":"

Generate the autocompletion script for fish

"},{"location":"commands/chainsaw_completion_fish/#synopsis","title":"Synopsis","text":"

Generate the autocompletion script for the fish shell.

To load completions in your current shell session:

chainsaw completion fish | source\n

To load completions for every new session, execute once:

chainsaw completion fish > ~/.config/fish/completions/chainsaw.fish\n

You will need to start a new shell for this setup to take effect.

chainsaw completion fish [flags]\n
"},{"location":"commands/chainsaw_completion_fish/#options","title":"Options","text":"
  -h, --help              help for fish\n      --no-descriptions   disable completion descriptions\n
"},{"location":"commands/chainsaw_completion_fish/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_completion_powershell/","title":"Chainsaw completion powershell","text":""},{"location":"commands/chainsaw_completion_powershell/#chainsaw-completion-powershell","title":"chainsaw completion powershell","text":"

Generate the autocompletion script for powershell

"},{"location":"commands/chainsaw_completion_powershell/#synopsis","title":"Synopsis","text":"

Generate the autocompletion script for powershell.

To load completions in your current shell session:

chainsaw completion powershell | Out-String | Invoke-Expression\n

To load completions for every new session, add the output of the above command to your powershell profile.

chainsaw completion powershell [flags]\n
"},{"location":"commands/chainsaw_completion_powershell/#options","title":"Options","text":"
  -h, --help              help for powershell\n      --no-descriptions   disable completion descriptions\n
"},{"location":"commands/chainsaw_completion_powershell/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_completion_zsh/","title":"Chainsaw completion zsh","text":""},{"location":"commands/chainsaw_completion_zsh/#chainsaw-completion-zsh","title":"chainsaw completion zsh","text":"

Generate the autocompletion script for zsh

"},{"location":"commands/chainsaw_completion_zsh/#synopsis","title":"Synopsis","text":"

Generate the autocompletion script for the zsh shell.

If shell completion is not already enabled in your environment you will need to enable it. You can execute the following once:

echo \"autoload -U compinit; compinit\" >> ~/.zshrc\n

To load completions in your current shell session:

source <(chainsaw completion zsh)\n

To load completions for every new session, execute once:

"},{"location":"commands/chainsaw_completion_zsh/#linux","title":"Linux:","text":"
chainsaw completion zsh > \"${fpath[1]}/_chainsaw\"\n
"},{"location":"commands/chainsaw_completion_zsh/#macos","title":"macOS:","text":"
chainsaw completion zsh > $(brew --prefix)/share/zsh/site-functions/_chainsaw\n

You will need to start a new shell for this setup to take effect.

chainsaw completion zsh [flags]\n
"},{"location":"commands/chainsaw_completion_zsh/#options","title":"Options","text":"
  -h, --help              help for zsh\n      --no-descriptions   disable completion descriptions\n
"},{"location":"commands/chainsaw_completion_zsh/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_create/","title":"Chainsaw create","text":""},{"location":"commands/chainsaw_create/#chainsaw-create","title":"chainsaw create","text":"

Create Chainsaw resources

chainsaw create [flags]\n
"},{"location":"commands/chainsaw_create/#options","title":"Options","text":"
  -h, --help   help for create\n
"},{"location":"commands/chainsaw_create/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_create_test/","title":"Chainsaw create test","text":""},{"location":"commands/chainsaw_create_test/#chainsaw-create-test","title":"chainsaw create test","text":"

Create a Chainsaw test

chainsaw create test [flags]\n
"},{"location":"commands/chainsaw_create_test/#options","title":"Options","text":"
      --description   If set, adds description when applicable (default true)\n      --force         If set, existing test will be deleted if needed\n  -h, --help          help for test\n      --save          If set, created test will be saved\n
"},{"location":"commands/chainsaw_create_test/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_docs/","title":"Chainsaw docs","text":""},{"location":"commands/chainsaw_docs/#chainsaw-docs","title":"chainsaw docs","text":"

Generate reference documentation

chainsaw docs [flags]\n
"},{"location":"commands/chainsaw_docs/#options","title":"Options","text":"
      --autogenTag      Determines if the generated docs should contain a timestamp (default true)\n  -h, --help            help for docs\n  -o, --output string   Output path (default \".\")\n      --website         Website version\n
"},{"location":"commands/chainsaw_docs/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_export/","title":"Chainsaw export","text":""},{"location":"commands/chainsaw_export/#chainsaw-export","title":"chainsaw export","text":"

Export commands

chainsaw export [flags]\n
"},{"location":"commands/chainsaw_export/#options","title":"Options","text":"
  -h, --help   help for export\n
"},{"location":"commands/chainsaw_export/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_export_schemas/","title":"Chainsaw export schemas","text":""},{"location":"commands/chainsaw_export_schemas/#chainsaw-export-schemas","title":"chainsaw export schemas","text":"

Export JSON schemas

chainsaw export schemas [flags]\n
"},{"location":"commands/chainsaw_export_schemas/#options","title":"Options","text":"
  -h, --help   help for schemas\n
"},{"location":"commands/chainsaw_export_schemas/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_lint/","title":"Chainsaw lint","text":""},{"location":"commands/chainsaw_lint/#chainsaw-lint","title":"chainsaw lint","text":"

Lint a file or read from standard input

"},{"location":"commands/chainsaw_lint/#synopsis","title":"Synopsis","text":"

Use chainsaw lint to lint a specific file or read from standard input for either test or configuration.

chainsaw lint [test|configuration] [flags]\n
"},{"location":"commands/chainsaw_lint/#options","title":"Options","text":"
  -f, --file string   Specify the file to lint or '-' for standard input\n  -h, --help          help for lint\n
"},{"location":"commands/chainsaw_lint/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_migrate/","title":"Chainsaw migrate","text":""},{"location":"commands/chainsaw_migrate/#chainsaw-migrate","title":"chainsaw migrate","text":"

Migrate resources to Chainsaw

chainsaw migrate [flags]\n
"},{"location":"commands/chainsaw_migrate/#options","title":"Options","text":"
  -h, --help   help for migrate\n
"},{"location":"commands/chainsaw_migrate/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_migrate_kuttl/","title":"Chainsaw migrate kuttl","text":""},{"location":"commands/chainsaw_migrate_kuttl/#chainsaw-migrate-kuttl","title":"chainsaw migrate kuttl","text":"

Migrate KUTTL resources to Chainsaw

chainsaw migrate kuttl [flags]\n
"},{"location":"commands/chainsaw_migrate_kuttl/#options","title":"Options","text":"
  -h, --help   help for kuttl\n
"},{"location":"commands/chainsaw_migrate_kuttl/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_migrate_kuttl_config/","title":"Chainsaw migrate kuttl config","text":""},{"location":"commands/chainsaw_migrate_kuttl_config/#chainsaw-migrate-kuttl-config","title":"chainsaw migrate kuttl config","text":"

Migrate KUTTL config to Chainsaw

chainsaw migrate kuttl config [flags]\n
"},{"location":"commands/chainsaw_migrate_kuttl_config/#options","title":"Options","text":"
      --cleanup   If set, delete converted files\n  -h, --help      help for config\n      --save      If set, converted files will be saved\n
"},{"location":"commands/chainsaw_migrate_kuttl_config/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_migrate_kuttl_tests/","title":"Chainsaw migrate kuttl tests","text":""},{"location":"commands/chainsaw_migrate_kuttl_tests/#chainsaw-migrate-kuttl-tests","title":"chainsaw migrate kuttl tests","text":"

Migrate KUTTL tests to Chainsaw

chainsaw migrate kuttl tests [flags]\n
"},{"location":"commands/chainsaw_migrate_kuttl_tests/#options","title":"Options","text":"
      --cleanup   If set, delete converted files\n  -h, --help      help for tests\n      --save      If set, converted files will be saved\n
"},{"location":"commands/chainsaw_migrate_kuttl_tests/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_test/","title":"Chainsaw test","text":""},{"location":"commands/chainsaw_test/#chainsaw-test","title":"chainsaw test","text":"

Run tests

chainsaw test [flags]... [test directories]...\n
"},{"location":"commands/chainsaw_test/#options","title":"Options","text":"
      --apply-timeout duration                    The apply timeout to use as default for configuration (default 5s)\n      --assert-timeout duration                   The assert timeout to use as default for configuration (default 30s)\n      --cleanup-delay duration                    Adds a delay between the time a test ends and the time cleanup starts\n      --cleanup-timeout duration                  The cleanup timeout to use as default for configuration (default 30s)\n      --cluster strings                           Register cluster (format <cluster name>=<kubeconfig path>:[context name])\n      --config string                             Chainsaw configuration file\n      --delete-timeout duration                   The delete timeout to use as default for configuration (default 15s)\n      --error-timeout duration                    The error timeout to use as default for configuration (default 30s)\n      --exclude-test-regex string                 Regular expression to exclude tests\n      --exec-timeout duration                     The exec timeout to use as default for configuration (default 5s)\n      --fail-fast                                 Stop the test upon encountering the first failure\n      --force-termination-grace-period duration   If specified, overrides termination grace periods in applicable resources\n      --full-name                                 Use full test case folder path instead of folder name\n  -h, --help                                      help for test\n      --include-test-regex string                 Regular expression to include tests\n      --kube-as string                            Username to impersonate for the operation\n      --kube-as-group stringArray                 Group to impersonate for the operation, this flag can be repeated to specify multiple groups.\n      --kube-as-uid string                        UID to impersonate for the operation\n      --kube-certificate-authority string         Path to a cert file for the certificate authority\n      --kube-client-certificate string            Path to a client certificate file for TLS\n      --kube-client-key string                    Path to a client key file for TLS\n      --kube-cluster string                       The name of the kubeconfig cluster to use\n      --kube-context string                       The name of the kubeconfig context to use\n      --kube-disable-compression                  If true, opt-out of response compression for all requests to the server\n      --kube-insecure-skip-tls-verify             If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure\n  -n, --kube-namespace string                     If present, the namespace scope for this CLI request\n      --kube-password string                      Password for basic authentication to the API server\n      --kube-proxy-url string                     If provided, this URL will be used to connect via proxy\n      --kube-request-timeout string               The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default \"0\")\n      --kube-server string                        The address and port of the Kubernetes API server\n      --kube-tls-server-name string               If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.\n      --kube-token string                         Bearer token for authentication to the API server\n      --kube-user string                          The name of the kubeconfig user to use\n      --kube-username string                      Username for basic authentication to the API server\n      --namespace string                          Namespace to use for tests\n      --no-cluster                                Runs without cluster\n      --no-color                                  Removes output colors\n      --parallel int                              The maximum number of tests to run at once\n      --pause-on-failure                          Pause test execution failure (implies no concurrency)\n      --repeat-count int                          Number of times to repeat each test (default 1)\n      --report-format string                      Test report format (JSON|XML|nil)\n      --report-name string                        The name of the report to create (default \"chainsaw-report\")\n      --report-path string                        The path of the report to create\n      --selector strings                          Selector (label query) to filter on\n      --skip-delete                               If set, do not delete the resources after running the tests\n      --template                                  If set, resources will be considered for templating (default true)\n      --test-dir strings                          Directories containing test cases to run\n      --test-file string                          Name of the test file (default \"chainsaw-test\")\n      --values strings                            Values passed to the tests\n
"},{"location":"commands/chainsaw_test/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_version/","title":"Chainsaw version","text":""},{"location":"commands/chainsaw_version/#chainsaw-version","title":"chainsaw version","text":"

Print the version informations

chainsaw version [flags]\n
"},{"location":"commands/chainsaw_version/#options","title":"Options","text":"
  -h, --help   help for version\n
"},{"location":"commands/chainsaw_version/#see-also","title":"SEE ALSO","text":""},{"location":"community/","title":"Community","text":"

Chainsaw has a growing community and we would definitely love to see you join and contribute.

Everyone is welcome to make suggestions, report bugs, open feature requests, contribute code or docs, participate in discussions, write blogs or anything that can benefit the project.

Chainsaw is built and maintained under the Kyverno umbrella but decisions are Community driven Everyone's voice matters

"},{"location":"community/#slack-channel","title":"Slack channel","text":"

Join our slack channel #kyverno-chainsaw to meet with users, contributors and maintainers.

"},{"location":"community/#community-meetings","title":"Community Meetings","text":"

To attend our community meetings, join the Chainsaw group. You will then be sent a meeting invite and will have access to the agenda and meeting notes. Any member may suggest topics for discussion.

This is a public, weekly for Kyverno-Chainsaw maintainers to make announcements and provide project updates, and request input and feedback. This forum allows community members to raise agenda items of any sort, including but not limited to any PRs or issues on which they are working.

Weekly every Thursday at 2:00 PM UTC

"},{"location":"community/#roadmap","title":"RoadMap","text":"

For detailed information on our planned features and upcoming updates, please view our Roadmap.

"},{"location":"community/#contributing","title":"Contributing","text":"

Please read the contributing guide for details around:

  1. Code of Conduct
  2. Code Culture
  3. Details on how to contribute
"},{"location":"community/#adopters","title":"Adopters","text":"

If you are using Chainsaw and want to share it publicly we always appreciate a bit of support. Pull requests to the ADOPTERS LIST will put a smile on our faces

"},{"location":"configuration/","title":"Configuring Chainsaw","text":"

Chainsaw is a comprehensive tool designed to facilitate end-to-end testing in Kubernetes.

This documentation will focus on providing a breakdown of its configuration structure and how to use it.

Chainsaw can be configured in two different and complementary ways:

Precedence

If both are specified, command-line flags will take precedence over configuration coming from a configuration file.

"},{"location":"configuration/#specific-configuration-options","title":"Specific configuration options","text":"

Please pay attention to the configuration options below, they may or may not be relevant in your case but can be useful in certain cases:

"},{"location":"configuration/cleanup-delay/","title":"Delay before cleanup","text":"

At the end of each test, Chainsaw will delete the resources it created during the test.

When testing operators, it can be useful to wait a little bit before starting the cleanup process to make sure the operator/controller has the necessary time to update the internal state.

For this reason, Chainsaw provides the delayBeforeCleanup configuration option and the corresponding --delay-before-cleanup flag.

"},{"location":"configuration/cleanup-delay/#configuration","title":"Configuration","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Configuration\nmetadata:\n  name: custom-config\nspec:\n  # ...\n  delayBeforeCleanup: 5s\n  # ...\n
"},{"location":"configuration/cleanup-delay/#flag","title":"Flag","text":"
chainsaw test --delay-before-cleanup 5s ...\n
"},{"location":"configuration/file/","title":"Configuration file","text":"

Chainsaw is described as a Stronger tool for e2e testing.

With its versatile configuration options, you can customize the testing process to fit your needs.

Chainsaw prioritizes its configuration in the following order:

  1. User-specified configuration

    If you explicitly provide a configuration file using a command-line flag

  2. Default configuration file

    If no configuration is specified, Chainsaw will look for a default file named .chainsaw.yaml in the current working directory

  3. Internal default configuration

    In the absence of both the above, Chainsaw will use a default configuration file embedded in the Chainsaw binary

"},{"location":"configuration/file/#example","title":"Example","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Configuration\nmetadata:\n  name: custom-config\nspec:\n  timeouts:\n    apply: 45s\n    assert: 20s\n    cleanup: 45s\n    delete: 25s\n    error: 10s\n    exec: 45s\n  skipDelete: false\n  failFast: true\n  parallel: 4\n  # ...\n
"},{"location":"configuration/file/#how-to-specify-a-configuration","title":"How to specify a configuration","text":"

To use a custom configuration file:

chainsaw test --config path/to/your/config.yaml\n

Defaults

If you don't specify any configuration, Chainsaw will look for the default configuration file .chainsaw.yaml in the current working directory.

If that file is not found, it will fall back to its internal default configuration.

"},{"location":"configuration/file/#default-configuration","title":"Default configuration","text":"

The default configuration below is used by Chainsaw when no configuration file was provided and the default file .chainsaw.yaml does not exist.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Configuration\nmetadata:\n  name: default\nspec: {}\n
"},{"location":"configuration/file/#reference-documentation","title":"Reference documentation","text":"

See Configuration API reference for more details.

"},{"location":"configuration/flags/","title":"Command line flags","text":"

Even after a configuration file is loaded, you can override specific settings using command-line flags.

"},{"location":"configuration/flags/#example","title":"Example","text":"
chainsaw test                         \\\n  path/to/test/dir                    \\\n  --config path/to/your/config.yaml   \\\n  --assert-timeout 45s                \\\n  --skip-delete false                 \\\n  --fail-fast true                    \\\n  --parallel 4                        \\\n  ...\n

In this example, Chainsaw will load a configuration file but the timeout configuration and other settings will be overridden by the values set in the flags, regardless of the value in the loaded configuration file.

"},{"location":"configuration/flags/#usage","title":"Usage","text":"

The command below will run tests using the configuration from my-config.yaml, taking tests from /path/to/tests, and running a maximum of 10 tests simultaneously.

chainsaw test /path/to/tests --config my-config.yaml --parallel 10\n
"},{"location":"configuration/flags/#reference-documentation","title":"Reference documentation","text":"

See Chainsaw test command reference for more details.

"},{"location":"configuration/grace/","title":"Termination graceful period","text":"

Some Kubernetes resources can take time before being stopped. For example, deleting a Pod can take time if the underlying container doesn't quit quickly enough.

For this reason, Chainsaw provides the forceTerminationGracePeriod configuration option and the corresponding --force-termination-grace-period flag. If set, Chainsaw will override the terminationGracePeriodSeconds when working with the following resource kinds:

"},{"location":"configuration/grace/#configuration","title":"Configuration","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Configuration\nmetadata:\n  name: custom-config\nspec:\n  # ...\n  forceTerminationGracePeriod: 5s\n  # ...\n
"},{"location":"configuration/grace/#flag","title":"Flag","text":"
chainsaw test --force-termination-grace-period 5s ...\n
"},{"location":"configuration/multi-cluster/","title":"Multi cluster","text":"

Chainsaw supports testing against multiple clusters.

To use a specific cluster in a test (or test step) you will need to register the cluster either using the config file or using command line flags.

Since v0.2.1 you can also register clusters dynamically at the test, step and operation levels. This is particularly useful when a cluster is created in a test step and used in subsequent steps.

"},{"location":"configuration/multi-cluster/#configuration","title":"Configuration","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Configuration\nmetadata:\n  name: custom-config\nspec:\n  # ...\n  clusters:\n    cluster-1:\n      kubeconfig: /path/to/kubeconfig-1\n      context: context-name\n    cluster-2:\n      kubeconfig: /path/to/kubeconfig-2\n      context: context-name\n  # ...\n
"},{"location":"configuration/multi-cluster/#flag","title":"Flag","text":"

The --cluster flag can appear multiple times and is expected to come in the following format --cluster cluster-name=/path/to/kubeconfig[:context-name].

chainsaw test                                               \\\n    --cluster cluster-1=/path/to/kubeconfig-1               \\\n    --cluster cluster-2=/path/to/kubeconfig-2:context-name\n
"},{"location":"configuration/multi-cluster/#dynamic-cluster-registration","title":"Dynamic cluster registration","text":"

The test below illustrates dynamic cluster registration:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: dynamic-clusters\nspec:\n  steps:\n  - try:\n    - script:\n        timeout: 1m\n        content: |\n          kind create cluster --name dynamic --kubeconfig ./dynamic\n    finally:\n    - script:\n        content: |\n          kind delete cluster --name dynamic\n    - script:\n        content: |\n          rm -f ./dynamic\n  - clusters:\n      dynamic:\n        kubeconfig: ./dynamic\n    cluster: dynamic\n    try:\n    - apply:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: quick-start\n            namespace: default\n          data:\n            foo: bar\n    - assert:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: quick-start\n            namespace: default\n          data:\n            foo: bar\n

In the test above, a cluster is created in the first step of the test then registered and used in the second step.

"},{"location":"configuration/multi-cluster/#usage","title":"Usage","text":"

Once registered, a cluster can be specified at the test, step or operation level. If specified at multiple levels, the most granular one is selected, effectively overriding the cluster specified at higher levels.

Specifying cluster at the test level

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  # all steps and operations will be executed against the\n  # cluster specified below (unless overridden)\n  cluster: cluster-1\n  steps:\n  - try:\n    # ...\n    - assert:\n        resource:\n          apiVersion: v1\n          kind: Deployment\n          metadata:\n            name: foo\n          spec:\n            (replicas > 3): true\n    # ...\n

Specifying cluster at the step level

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - # all operations will be executed against the\n    # cluster specified below (unless overridden)\n    cluster: cluster-1\n    try:\n    # ...\n    - assert:\n        resource:\n          apiVersion: v1\n          kind: Deployment\n          metadata:\n            name: foo\n          spec:\n            (replicas > 3): true\n    # ...\n

Specifying cluster at the operation level

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - assert:\n        # operation will be executed against the\n        # cluster specified below\n        cluster: cluster-1\n        resource:\n          apiVersion: v1\n          kind: Deployment\n          metadata:\n            name: foo\n          spec:\n            (replicas > 3): true\n    # ...\n
"},{"location":"configuration/multi-cluster/#client-binding","title":"$client binding","text":"

When a cluster is specified (whatever the test, step or operation level), the $client binding visible in operation expressions is always the Kubernetes client corresponding to the configured cluster.

In the example below, $client binding is a client to the configured cluster-1.

$client binding

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - assert:\n        # operation will be executed against the\n        # cluster specified below\n        cluster: cluster-1\n        resource:\n          (x_k8s_list($client, 'v1', 'Node')):\n            (length(items)): 1\n    # ...\n
"},{"location":"configuration/multi-cluster/#kubeconfig-in-scripts-and-commands","title":"KUBECONFIG in scripts and commands","text":"

When a cluster is specified (whatever the test, step or operation level), a temporary KUBECONFIG is automatically created and the KUBECONFIG is set to this temporary file for every command and script operation.

This way command and script operations are executed in the context of the configured cluster.

Temporary KUBECONFIG

The default KUBECONFIG you usually observe in your local shell will be different in script and command operations.

Temporary KUBECONFIG

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - script:\n        # operation will be executed against the\n        # cluster specified below\n        cluster: cluster-1\n        # the kubectl command below will use a \n        # KUBECONFIG environment variable pointing\n        # to a temporary kubeconfig file configured\n        # for the specified cluster\n        content: kubectl get pods\n    # ...\n
"},{"location":"configuration/no-cluster/","title":"Running without a cluster","text":"

Chainsaw can be run without connection to a Kubernetes cluster. In this case, chainsaw will not try to create an ephemeral namespace and all operations requiring a Kubernetes cluster will fail.

To run chainsaw in this mode pass the --no-cluster flag.

"},{"location":"configuration/no-cluster/#example","title":"Example","text":"
# run chainsaw without connection to a Kubernetes cluster\nchainsaw test --no-cluster\n
"},{"location":"configuration/reports/","title":"Reports","text":"

Chainsaw can generate JUnit reports in XML or JSON format.

To produce a test report, configure the report format, report path and report name in the configuration or using CLI flags.

"},{"location":"configuration/reports/#configuration","title":"Configuration","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Configuration\nmetadata:\n  name: custom-config\nspec:\n  # ...\n  reportFormat: JSON\n  reportName: chainsaw-report\n  reportPath: /home/chainsaw\n  # ...\n
"},{"location":"configuration/reports/#flag","title":"Flag","text":"
chainsaw test --report-format JSON --report-name chainsaw-report --report-path /path/to/save/report ...\n

Note: The reportPath can be specified as either a relative or an absolute path.

"},{"location":"configuration/selector/","title":"Label selectors","text":"

Chainsaw can filter the tests to run using label selectors.

You can pass label selectors using the --selector flag when invoking the chainsaw test command.

"},{"location":"configuration/selector/#example","title":"Example","text":"

Given the test below:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: basic\n  labels:\n    foo: bar\nspec:\n  # ...\n

Invoking Chainsaw with the command below will take the test above into account:

chainsaw test --selector foo=bar\n
"},{"location":"configuration/templating/","title":"Resource templating","text":"

Experimental status

This is an experimental feature, and implementation could change slightly in the next versions.

Info

Templating was disabled by default in v0.1.* but is now enabled by default since v0.2.1.

Chainsaw can apply transformations to the resources before they are processed by the operation.

This is useful when a resource needs some runtime configuration.

Templating must be enabled at the configuration, test, step, or operation level for the templating process to kick in. Alternatively, templating can be enabled using the --template flag when invoking chainsaw from the command line.

Note

Unlike assertion trees, templating can only be specified in leave nodes of the YAML tree.

"},{"location":"configuration/templating/#example","title":"Example","text":"

The test below will create, assert, and delete a ConfigMap with a dynamic name configured at runtime using the $namespace binding.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: values\nspec:\n  steps:\n  - try:\n    - apply:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: ($namespace)\n    - assert:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: ($namespace)\n    - delete:\n        ref:\n          apiVersion: v1\n          kind: ConfigMap\n          name: ($namespace)\n
"},{"location":"configuration/timeouts/","title":"Timeouts","text":"

Timeouts in Chainsaw are specified per type of operation. This is required because the timeout varies greatly depending on the nature of an operation.

For example, applying a manifest in a cluster is expected to be reasonably fast, while validating a resource can be a long operation.

Chainsaw supports separately configuring the timeouts below:

Overriding timeouts

Each timeout can be overridden at the test level, test step level, or individual operation level.

Timeouts defined in the Configuration are used in operations when not overridden.

"},{"location":"configuration/timeouts/#configuration","title":"Configuration","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Configuration\nmetadata:\n  name: custom-config\nspec:\n  # ...\n  timeouts:\n    apply: 45s\n    assert: 20s\n    cleanup: 45s\n    delete: 25s\n    error: 10s\n    exec: 45s\n  # ...\n
"},{"location":"configuration/timeouts/#flags","title":"Flags","text":"
chainsaw test                     \\\n    --apply-timeout 45s             \\\n    --assert-timeout 45s            \\\n    --cleanup-timeout 45s           \\\n    --delete-timeout 45s            \\\n    --error-timeout 45s             \\\n    --exec-timeout 45s              \\\n    ...\n
"},{"location":"configuration/values/","title":"Passing data to tests","text":"

Chainsaw can pass arbitrary values when running tests using the --values flag. Values will be available to tests under the $values binding.

This is useful when a test needs to be configured externally.

"},{"location":"configuration/values/#example","title":"Example","text":"

The test below expects the $value.foo to be provided when chainsaw is invoked.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: values\nspec:\n  steps:\n  - try:\n    - assert:\n        resource:\n          ($values.foo): bar\n

Now you can invoke chainsaw like this:

# pass object { \"foo\": \"bar\"\u00a0} as values to the executed tests\n# `--values -` means values are read from standard input\necho \"foo: bar\" | chainsaw test --values -\n\n# read values from a file\nchainsaw test --values ./values.yaml\n\n# pass values using heredoc\nchainsaw test --values - <<EOF\nfoo: bar\nEOF\n
"},{"location":"examples/","title":"Examples","text":""},{"location":"examples/#setup","title":"Setup","text":"

To use Chainsaw you will need a Kubernetes cluster, Chainsaw won't create one for you.

In these examples, we will use kind but feel free to use the tool of your choice.

Not a cluster management tool

We consider this is not the responsibility of Chainsaw to manage clusters.

There are plenty of solutions to create and manage local clusters that will do that better than Chainsaw.

"},{"location":"examples/#create-a-kind-cluster","title":"Create a kind cluster","text":"

Please refer to the kind install docs to install it locally.

Once kind creating a local cluster is as simple as running:

# define kind image\nexport KIND_IMAGE=\"kindest/node:v1.28.0\"\n\n# create cluster\nkind create cluster --image $KIND_IMAGE\n
"},{"location":"examples/#install-chainsaw","title":"Install Chainsaw","text":"

Refer to install docs to install Chainsaw.

"},{"location":"examples/array-assertions/","title":"Array assertions","text":"

This example demonstrates how to perform complex assertions on arrays.

"},{"location":"examples/array-assertions/#setup","title":"Setup","text":"

See Setup docs

"},{"location":"examples/array-assertions/#manifests","title":"Manifests","text":""},{"location":"examples/array-assertions/#resourcesyaml","title":"resources.yaml","text":"
apiVersion: v1\nkind: Pod\nmetadata:\n  name: example\nspec:\n  containers:\n  - name: container-1\n    image: nginx-1\n    env:\n    - name: ENV_1\n      value: value-1\n  - name: container-2\n    image: nginx-2\n    env:\n    - name: ENV_2\n      value: value-2\n  - name: container-3\n    image: nginx-3\n    env:\n    - name: ENV_3\n      value: value-3\n
"},{"location":"examples/array-assertions/#assertionsyaml","title":"assertions.yaml","text":"
apiVersion: v1\nkind: Pod\nmetadata:\n  name: example\nspec:\n  # iterate over all containers having `name: container-1`\n  ~.(containers[?name == 'container-1']):\n    image: nginx-1\n  # iterate over all containers, bind `$index` to the element index\n  ~index.(containers):\n    image: (join('-', ['nginx', to_string($index + `1`)]))\n  # nested iteration\n  ~index2.(containers):\n    ~.(env):\n      name: (join('_', ['ENV', to_string($index2 + `1`)]))\n      value: (join('-', ['value', to_string($index2 + `1`)]))\n
"},{"location":"examples/array-assertions/#test","title":"Test","text":""},{"location":"examples/array-assertions/#chainsaw-testyaml","title":"chainsaw-test.yaml","text":"
# yaml-language-server: $schema=https://raw.githubusercontent.com/kyverno/chainsaw/main/.schemas/json/test-chainsaw-v1alpha1.json\napiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: array-assertions\nspec:\n  steps:\n  - try:\n    - apply:\n        file: resources.yaml\n    - assert:\n        file: assertions.yaml\n
"},{"location":"examples/array-assertions/#execute","title":"Execute","text":"
chainsaw test\n
"},{"location":"jp/functions/","title":"Functions","text":"

Experimental functions

Experimental functions are denoted by the x_ prefix.

These are functions that are subject to signature change in a future version.

"},{"location":"jp/functions/#built-in-functions","title":"built-in functions","text":"Name Signature abs abs(number) avg avg(array[number]) ceil ceil(number) contains contains(array|string, any) ends_with ends_with(string, string) find_first find_first(string, string, number, number) find_last find_last(string, string, number, number) floor floor(number) from_items from_items(array[array]) group_by group_by(array, expref) items items(object) join join(string, array[string]) keys keys(object) length length(string|array|object) lower lower(string) map map(expref, array) max max(array[number]|array[string]) max_by max_by(array, expref) merge merge(object) min min(array[number]|array[string]) min_by min_by(array, expref) not_null not_null(any) pad_left pad_left(string, number, string) pad_right pad_right(string, number, string) replace replace(string, string, string, number) reverse reverse(array|string) sort sort(array[string]|array[number]) sort_by sort_by(array, expref) split split(string, string, number) starts_with starts_with(string, string) sum sum(array[number]) to_array to_array(any) to_number to_number(any) to_string to_string(any) trim trim(string, string) trim_left trim_left(string, string) trim_right trim_right(string, string) type type(any) upper upper(string) values values(object) zip zip(array, array)"},{"location":"jp/functions/#kyverno-json-functions","title":"kyverno-json functions","text":"Name Signature at at(array, any) concat concat(string, string) json_parse json_parse(string) wildcard wildcard(string, string)"},{"location":"jp/functions/#kyverno-functions","title":"kyverno functions","text":"Name Signature compare compare(string, string) equal_fold equal_fold(string, string) replace replace(string, string, string, number) replace_all replace_all(string, string, string) to_upper to_upper(string) to_lower to_lower(string) trim trim(string, string) trim_prefix trim_prefix(string, string) split split(string, string) regex_replace_all regex_replace_all(string, string|number, string|number) regex_replace_all_literal regex_replace_all_literal(string, string|number, string|number) regex_match regex_match(string, string|number) pattern_match pattern_match(string, string|number) label_match label_match(object, object) to_boolean to_boolean(string) add add(any, any) sum sum(array) subtract subtract(any, any) multiply multiply(any, any) divide divide(any, any) modulo modulo(any, any) round round(number, number) base64_decode base64_decode(string) base64_encode base64_encode(string) time_since time_since(string, string, string) time_now time_now() time_now_utc time_now_utc() path_canonicalize path_canonicalize(string) truncate truncate(string, number) semver_compare semver_compare(string, string) parse_json parse_json(string) parse_yaml parse_yaml(string) lookup lookup(object|array, string|number) items items(object|array, string, string) object_from_lists object_from_lists(array, array) random random(string) x509_decode x509_decode(string) time_to_cron time_to_cron(string) time_add time_add(string, string) time_parse time_parse(string, string) time_utc time_utc(string) time_diff time_diff(string, string) time_before time_before(string, string) time_after time_after(string, string) time_between time_between(string, string, string) time_truncate time_truncate(string, string)"},{"location":"jp/functions/#chainsaw-functions","title":"chainsaw functions","text":"Name Signature env env(string) x_k8s_get x_k8s_get(any, string, string, string, string) x_k8s_list x_k8s_list(any, string, string, string) x_k8s_exists x_k8s_exists(any, string, string, string, string) x_k8s_resource_exists x_k8s_resource_exists(any, string, string) x_k8s_server_version x_k8s_server_version(any)"},{"location":"more/events/","title":"Working with events","text":"

Kubernetes events are regular Kubernetes objects and can be asserted on just like any other object:

apiVersion: v1\nkind: Event\nreason: Started\nsource:\n  component: kubelet\ninvolvedObject:\n  apiVersion: v1\n  kind: Pod\n  name: my-pod\n
"},{"location":"more/kuttl-migration/","title":"Migration from KUTTL","text":""},{"location":"more/kuttl-migration/#overview","title":"Overview","text":"

The chainsaw migrate kuttl tests and chainsaw migrate kuttl config commands are designed for the migration of KUTTL tests to Chainsaw.

Reference documentation

You can view the full command documentation here.

"},{"location":"more/kuttl-migration/#examples","title":"Examples","text":""},{"location":"more/kuttl-migration/#migrate-tests","title":"Migrate tests","text":"

The command below will migrate KUTTL tests to Chainsaw and overwrite original files with converted ones.

chainsaw migrate kuttl tests path/to/kuttl/tests --save --cleanup\n

This will generate a chainsaw-test.yaml for every KUTTL test discovered.

"},{"location":"more/kuttl-migration/#migrate-configuration","title":"Migrate configuration","text":"

The command below will migrate a KUTTL test suite file to the corresponding Chainsaw Configuration.

chainsaw migrate kuttl config path/to/kuttl/testsuite --save --cleanup\n

This will generate a .chainsaw.yaml configuration file.

"},{"location":"more/lint/","title":"Lint tests","text":""},{"location":"more/lint/#overview","title":"Overview","text":"

Chainsaw comes with a lint command to detect ill-formated tests.

Reference documentation

You can view the full command documentation here.

"},{"location":"more/lint/#usage","title":"Usage","text":"

To build the docs of a test, Chainsaw provides the chainsaw lint test -f path/to/chainsaw-test.yaml command.

chainsaw lint test -f - <<EOF\napiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: assertion-tree\nspec:\n  steps:\n  - try:\n    - assert:\n        file: assert.yaml\nEOF\n
Processing input...\nThe document is valid\n
"},{"location":"more/test-docs/","title":"Building test docs","text":""},{"location":"more/test-docs/#overview","title":"Overview","text":"

Chainsaw makes it simple to build the documentation of your tests.

As test suites grow, it becomes important to document what a test does and how it is supposed to work.

Going through the implementation of a test to understand its purpose is not an efficient strategy.

Reference documentation

You can view the full command documentation here.

"},{"location":"more/test-docs/#usage","title":"Usage","text":"

To build the docs of a test, Chainsaw provides the chainsaw build docs command.

chainsaw build docs --test-dir path/to/chainsaw/tests\n

This will automatically discover tests and document steps and operations in try, catch and finally statements.

"},{"location":"more/test-docs/#the-description-field","title":"The description field","text":"

Additionally, you can set the description field in:

Chainsaw will output them nicely in the built docs.

"},{"location":"more/test-docs/#example","title":"Example","text":"

See below for an example test and the corresponding built docs.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: basic\nspec:\n  description: This is a very simple test that creates a configmap and checks the content is as expected.\n  steps:\n  - description: This steps applies the configmap in the cluster and checks the configmap content.\n    try:\n    - description: Create the configmap.\n      apply:\n        file: configmap.yaml\n    - description: Check the configmap content.\n      assert:\n        file: configmap-assert.yaml\n
"},{"location":"more/test-docs/#test-basic","title":"Test: basic","text":"

This is a very simple test that creates a configmap and checks the content is as expected.

"},{"location":"more/test-docs/#steps","title":"Steps","text":"# Name Try Catch Finally 1 step-1 2 0 0"},{"location":"more/test-docs/#step-step-1","title":"Step: step-1","text":"

This step applies the configmap in the cluster and checks the configmap content.

"},{"location":"more/test-docs/#try","title":"Try","text":"# Operation Description 1 apply Create the configmap. 2 assert Check the configmap content."},{"location":"operations/","title":"Operations","text":"

While tests are made of test steps, test steps can be considered made of operations.

Every operation in a test step runs sequentially.

Only one action per operation

Every operation consists of a single action. While it is syntactically possible to create an operation with multiple actions, Chainsaw will verify and reject tests if operations containing multiple actions are found.

The reasoning behind this intentional choice is that it becomes harder to understand in which order actions will be executed in case an operation consists of multiple actions. For this reason, operations consisting of multiple actions are disallowed.

"},{"location":"operations/#common-fields","title":"Common fields","text":"

All operations share some configuration fields.

Reference documentation

The full structure of the Operation is documented here.

"},{"location":"operations/#continueonerror","title":"ContinueOnError","text":"

Determines whether a test step should continue or not in case the operation is not successful.

Even if the test continues executing, it will still be reported as failed.

"},{"location":"operations/#available-operations","title":"Available operations","text":""},{"location":"operations/#non-resource-assertions","title":"Non-resource assertions","text":"

It is possible to evaluate assertions that do not depend on resources.

See Non-resource assertions for details.

"},{"location":"operations/#operation-checks","title":"Operation checks","text":"

Some operations support checking the operation execution result against specific expectations.

See Operation checks for use case details and supported operations.

"},{"location":"operations/#resource-templating","title":"Resource templating","text":"

Chainsaw can apply transformations to the resources before they are processed by the operation.

See Resource templating for use case details and supported operations.

"},{"location":"operations/apply/","title":"Apply","text":"

The apply operation lets you define resources that should be applied to the Kubernetes cluster during the test step.

These can be configurations, deployments, services, or any other Kubernetes resource.

"},{"location":"operations/apply/#configuration","title":"Configuration","text":"

Reference documentation

"},{"location":"operations/apply/#usage-examples","title":"Usage examples","text":"

Below is an example of using apply in a Test resource.

Using a specific file

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - apply:\n        file: my-configmap.yaml\n    # ...\n

Using file path expressions

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example-multi\nspec:\n  steps:\n  - try:\n    # ...\n    - apply:\n        file: \"configs/*.yaml\"\n    # ...\n

Using an URL

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - apply:\n        file: https://raw.githubusercontent.com/kyverno/chainsaw/main/testdata/step/configmap.yaml\n    # ...\n

Using an inline resource

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - apply:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: chainsaw-quick-start\n          data:\n            foo: bar\n    # ...\n
"},{"location":"operations/apply/#operation-check","title":"Operation check","text":"

Below is an example of using an operation check.

With check

# ...\n- apply:\n    file: my-configmap.yaml\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: ConfigMap\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n

With check

# ...\n- apply:\n    resource:\n      apiVersion: v1\n      kind: ConfigMap\n      metadata:\n        name: chainsaw-quick-start\n      data:\n        foo: bar\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: ConfigMap\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n
"},{"location":"operations/assert/","title":"Assert","text":"

The assert operation allows you to specify conditions that should hold true for a successful test.

For example, after applying resources, you might want to ensure that a particular pod is running or a service is accessible.

Assertion trees

Assertions in Chainsaw are based on assertion trees.

Assertion trees are a solution to declaratively represent complex conditions like partial array comparisons or complex operations against an incoming data structure.

Assertion trees are compatible with standard assertions that exist in tools like KUTTL but can do a lot more. Please see the assertion trees documentation in kyverno-json for details.

"},{"location":"operations/assert/#configuration","title":"Configuration","text":"

Reference documentation

"},{"location":"operations/assert/#usage-examples","title":"Usage examples","text":"

Below is an example of using assert in a Test resource.

Using a specific file for assertions

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - assert:\n        file: ../resources/deployment-assert.yaml\n    # ...\n

Using file path expressions for assertions

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example-multi\nspec:\n  steps:\n  - try:\n    # ...\n    - assert:\n        file: \"../assertions/*.yaml\"\n    # ...\n

Using an URL

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - assert:\n        file: https://raw.githubusercontent.com/kyverno/chainsaw/main/testdata/resource/valid.yaml\n    # ...\n

Using an inline assertion tree

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - assert:\n        resource:\n          apiVersion: v1\n          kind: Deployment\n          metadata:\n            name: foo\n          spec:\n            (replicas > 3): true\n    # ...\n
"},{"location":"operations/check/","title":"Operation checks","text":"

Considering an operation's success or failure is not always as simple as checking an error code.

To support those kinds of use cases, some operations support an additional check field to evaluate the operation result against an assertion tree.

Assertion trees

Assertions in Chainsaw are based on assertion trees.

Assertion trees are a solution to declaratively represent complex conditions like partial array comparisons or complex operations against an incoming data structure.

Assertion trees are compatible with standard assertions that exist in tools like KUTTL but can do a lot more. Please see the assertion trees documentation in kyverno-json for details.

Checked model

Different operations have a different model passed through the assertion tree.

The object passed to the assertion tree is the output object of the operation. Additional data like error or standard logs are passed using bindings ($error, $stdout, $stderr)

"},{"location":"operations/check/#expect-vs-check","title":"Expect vs Check","text":"

While a simple check is enough to determine the result of a single operation, we needed a more advanced construct to cover apply and create operations. Those operations can operate on files containing multiple manifests and every manifest can have a different result.

To support more granular checks we use the expect field that contains an array of Expectations. Every expectation is made of an optional match and a check statement.

This way it is possible to control the scope of a check.

Null match

If the match statement is null, the check statement applies to all manifests in the operation.

If no expectation matches a given manifest, the default expectation will be used, checking that no error occurred.

"},{"location":"operations/check/#apply","title":"Apply","text":"

apply supports expect and has the following elements to be checked:

Name Purpose Type $values Values provided when invoking chainsaw with --values flag object $namespace Name of the current test namespace string $client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object $config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object $test.id Current test id int $step.id Current step id int $operation.id Current operation id int $operation.resourceId Current resource id int $error The error message (if any) at the end of the operation string @ The state of the resource (if any) at the end of the operation object"},{"location":"operations/check/#command","title":"Command","text":"

command supports check and has the following elements to be checked:

Name Purpose Type $values Values provided when invoking chainsaw with --values flag object $namespace Name of the current test namespace string $client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object $config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object $test.id Current test id int $step.id Current step id int $operation.id Current operation id int $operation.resourceId Current resource id int $error The error message (if any) at the end of the operation string $stdout The content of the standard console output (if any) at the end of the operation string $stderr The content of the standard console error output (if any) at the end of the operation string @ Always null"},{"location":"operations/check/#create","title":"Create","text":"

create supports expect and has the following elements to be checked:

Name Purpose Type $values Values provided when invoking chainsaw with --values flag object $namespace Name of the current test namespace string $client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object $config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object $test.id Current test id int $step.id Current step id int $operation.id Current operation id int $operation.resourceId Current resource id int $error The error message (if any) at the end of the operation string @ The state of the resource (if any) at the end of the operation object"},{"location":"operations/check/#delete","title":"Delete","text":"

delete supports check and has the following elements to be checked:

Name Purpose Type $values Values provided when invoking chainsaw with --values flag object $namespace Name of the current test namespace string $client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object $config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object $test.id Current test id int $step.id Current step id int $operation.id Current operation id int $operation.resourceId Current resource id int $error The error message (if any) at the end of the operation string @ The state of the resource (if any) at the end of the operation object"},{"location":"operations/check/#patch","title":"Patch","text":"

patch supports expect and has the following elements to be checked:

Name Purpose Type $values Values provided when invoking chainsaw with --values flag object $namespace Name of the current test namespace string $client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object $config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object $test.id Current test id int $step.id Current step id int $operation.id Current operation id int $operation.resourceId Current resource id int $error The error message (if any) at the end of the operation string @ The state of the resource (if any) at the end of the operation object"},{"location":"operations/check/#script","title":"Script","text":"

script supports check and has the following elements to be checked:

Name Purpose Type $values Values provided when invoking chainsaw with --values flag object $namespace Name of the current test namespace string $client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object $config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object $test.id Current test id int $step.id Current step id int $operation.id Current operation id int $operation.resourceId Current resource id int $error The error message (if any) at the end of the operation string $stdout The content of the standard console output (if any) at the end of the operation string $stderr The content of the standard console error output (if any) at the end of the operation string @ Always null"},{"location":"operations/check/#update","title":"Update","text":"

update supports expect and has the following elements to be checked:

Name Purpose Type $values Values provided when invoking chainsaw with --values flag object $namespace Name of the current test namespace string $client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object $config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object $test.id Current test id int $step.id Current step id int $operation.id Current operation id int $operation.resourceId Current resource id int $error The error message (if any) at the end of the operation string @ The state of the resource (if any) at the end of the operation object"},{"location":"operations/command/","title":"Command","text":"

The command operation provides a means to execute a specific command during the test step.

"},{"location":"operations/command/#configuration","title":"Configuration","text":"

Reference documentation

"},{"location":"operations/command/#usage-examples","title":"Usage examples","text":"

Below is an example of using command in a Test resource.

Example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - command:\n        entrypoint: echo\n        args:\n        - hello chainsaw\n    # ...\n

When defining shell command args in YAML format, it's crucial to consider potential differences in behavior, as Chainsaw may interpret them differently compared to regular shell or bash environments, due to quote removal.

"},{"location":"operations/command/#operation-check","title":"Operation check","text":"

Below is an example of using an operation check.

With check

# ...\n- command:\n    entrypoint: echo\n    args:\n    - hello chainsaw\n    check:\n      # an error is expected, this will:\n      # - succeed if the operation failed\n      # - fail if the operation succeeded\n      ($error != null): true\n# ...\n
"},{"location":"operations/create/","title":"Create","text":"

The create operation lets you define resources that should be created in the Kubernetes cluster during the test step. These can be configurations, deployments, services, or any other Kubernetes resource.

"},{"location":"operations/create/#configuration","title":"Configuration","text":"

Reference documentation

Warning

If the resource to be created already exists in the cluster, the step will fail.

"},{"location":"operations/create/#usage-examples","title":"Usage examples","text":"

Below is an example of using create in a Test resource.

Using a specific file

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - create:\n        file: my-configmap.yaml\n    # ...\n

Using file path expressions

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example-multi\nspec:\n  steps:\n  - try:\n    # ...\n    - create:\n        file: \"configs/*.yaml\"\n    # ...\n

Using an URL

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - create:\n        file: https://raw.githubusercontent.com/kyverno/chainsaw/main/testdata/resource/valid.yaml\n    # ...\n

Using an inline resource

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - create:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: chainsaw-quick-start\n          data:\n            foo: bar\n    # ...\n
"},{"location":"operations/create/#operation-check","title":"Operation check","text":"

Below is an example of using an operation check.

With check

# ...\n- create:\n    file: my-configmap.yaml\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: ConfigMap\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n

With check

# ...\n- create:\n    resource:\n      apiVersion: v1\n      kind: ConfigMap\n      metadata:\n        name: chainsaw-quick-start\n      data:\n        foo: bar\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: ConfigMap\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n
"},{"location":"operations/delete/","title":"Delete","text":"

The delete operation allows you to specify resources that should be deleted from the Kubernetes cluster before a particular test step is executed.

Info

The propagation policy is forced to Background because some types default to Orphan (this is the case for unmanaged jobs for example) and we don't want to let dangling pods run in the cluster after cleanup.

"},{"location":"operations/delete/#configuration","title":"Configuration","text":"

Reference documentation

"},{"location":"operations/delete/#usage-examples","title":"Usage examples","text":"

Below is an example of using delete in a Test resource.

Example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - delete:\n        ref:\n          apiVersion: v1\n          kind: Pod\n          namespace: default\n          name: my-test-pod\n    # ...\n
"},{"location":"operations/delete/#operation-check","title":"Operation check","text":"

Below is an example of using an operation check.

With check

# ...\n- delete:\n    ref:\n      apiVersion: v1\n      kind: Pod\n      namespace: default\n      name: my-test-pod\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: Pod\n        metadata:\n          namespace: default\n          name: my-test-pod\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n
"},{"location":"operations/error/","title":"Error","text":"

The error operation lets you define a set of expected errors for a test step. If any of these errors occur during the test, they are treated as expected outcomes. However, if an error that's not on this list occurs, it will be treated as a test failure.

Assertion trees

Errors in Chainsaw are based on assertion trees.

Assertion trees is a solution to declaratively represent complex conditions like partial array comparisons or complex operations against an incoming data structure.

Assertion trees are compatible with standard assertions that exist in tools like KUTTL but can do a lot more. Please see the assertion trees documentation in kyverno-json for details.

"},{"location":"operations/error/#configuration","title":"Configuration","text":"

Reference documentation

"},{"location":"operations/error/#usage-examples","title":"Usage examples","text":"

Below is an example of using error in a Test resource.

Expecting an error from a specific file

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - error:\n        file: ../resources/deployment-error.yaml\n    # ...\n

Using file path expressions to expect errors from multiple files

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example-multi\nspec:\n  steps:\n  - try:\n    # ...\n    - error:\n        file: \"../errors/*.yaml\"\n    # ...\n

Using an URL

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - error:\n        file: https://raw.githubusercontent.com/user/repo/branch/path/to/deployment-error.yaml\n    # ...\n

Using an inline assertion tree

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - error:\n        resource:\n          apiVersion: v1\n          kind: Deployment\n          metadata:\n            name: foo\n          spec:\n            (replicas > 3): true\n    # ...\n
"},{"location":"operations/non-resource-assert/","title":"Non-resource assertions","text":"

Under certain circumstances, it makes sense to evaluate assertions that do not depend on resources. For example, when asserting the number of nodes in a cluster is equal to a known value.

"},{"location":"operations/non-resource-assert/#usage-examples","title":"Usage examples","text":"

Below is an example of using assert in a Test resource.

Using a file

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: non-resource-assertion\nspec:\n  steps:\n  - try:\n    - assert:\n        resource:\n          (x_k8s_list($client, 'v1', 'Node')):\n            (length(items)): 1\n    - error:\n        resource:\n          (x_k8s_list($client, 'v1', 'Node')):\n            (length(items)): 2\n
"},{"location":"operations/patch/","title":"Patch","text":"

The patch operation lets you define resources that should be modified in the Kubernetes cluster during the test step. These can be configurations, deployments, services, or any other Kubernetes resource.

"},{"location":"operations/patch/#configuration","title":"Configuration","text":"

Reference documentation

Warning

If the resource to be modified does not exist in the cluster, the step will fail.

"},{"location":"operations/patch/#usage-examples","title":"Usage examples","text":"

Below is an example of using patch in a Test resource.

Using a specific file

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - patch:\n        file: my-configmap.yaml\n    # ...\n

Using file path expressions

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example-multi\nspec:\n  steps:\n  - try:\n    # ...\n    - patch:\n        file: \"configs/*.yaml\"\n    # ...\n

Using an URL

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - patch:\n        file: https://raw.githubusercontent.com/kyverno/chainsaw/main/testdata/resource/valid.yaml\n    # ...\n

Using an inline resource

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - patch:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: chainsaw-quick-start\n          data:\n            foo: bar\n    # ...\n
"},{"location":"operations/patch/#operation-check","title":"Operation check","text":"

Below is an example of using an operation check.

With check

# ...\n- patch:\n    file: my-configmap.yaml\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: ConfigMap\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n

With check

# ...\n- patch:\n    resource:\n      apiVersion: v1\n      kind: ConfigMap\n      metadata:\n        name: chainsaw-quick-start\n      data:\n        foo: bar\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: ConfigMap\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n
"},{"location":"operations/script/","title":"Script","text":"

The script operation provides a means to run a script during the test step.

"},{"location":"operations/script/#configuration","title":"Configuration","text":"

Reference documentation

"},{"location":"operations/script/#usage-examples","title":"Usage examples","text":"

Below is an example of using script in a Test resource.

Example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - script:\n        content: |\n          echo \"hello chainsaw\"\n    # ...\n
"},{"location":"operations/script/#operation-check","title":"Operation check","text":"

Below is an example of using an operation check.

With check

# ...\n- script:\n    content: |\n      echo \"hello chainsaw\"\n    check:\n      # an error is expected, this will:\n      # - succeed if the operation failed\n      # - fail if the operation succeeded\n      ($error != null): true\n# ...\n
"},{"location":"operations/sleep/","title":"Sleep","text":"

The sleep operation provides a means to sleep for a configured duration.

"},{"location":"operations/sleep/#configuration","title":"Configuration","text":"

Reference documentation

The full structure of the Sleep is documented here.

"},{"location":"operations/sleep/#usage-examples","title":"Usage examples","text":"

Below is an example of using sleep in a Test resource.

Example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - sleep:\n        duration: 30s\n    # ...\n
"},{"location":"operations/templating/","title":"Resource templating","text":"

Experimental status

This is an experimental feature, and implementation could change slightly in the next versions.

Info

Templating was disabled by default in v0.1.* but is now enabled by default since v0.2.1.

Chainsaw can apply transformations to the resources before they are processed by the operation.

This is useful when a resource needs some runtime configuration.

Templating must be enabled at the configuration, test, step, or operation level for the templating process to kick in. Alternatively, templating can be enabled using the --template flag when invoking chainsaw from the command line.

Note

Unlike assertion trees, templating can only be specified in leave nodes of the YAML tree.

"},{"location":"operations/templating/#supported-operations","title":"Supported operations","text":"

Resource templating is supported in the following operations:

"},{"location":"operations/templating/#assert-and-error","title":"Assert and Error","text":"

When templating assert or error operations, the content is already an assertion tree.

For this reason, only the elements used for looking up the resources to be processed by the operation will be considered for templating. That is, only apiVersion, kind, name, namespace and labels are considered for templating. Other fields are not, they are part of the assertion tree only.

assert and error example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: template\nspec:\n  template: true\n  steps:\n  - assert:\n      resource:\n        # apiVersion, kind, name, namespace and labels are considered for templating\n        apiVersion: v1\n        kind: ConfigMap\n        metadata:\n          name: ($namespace)\n        # other fields are not (they are part of the assertion tree)\n        data:\n          foo: ($namespace)\n  - error:\n      resource:\n        # apiVersion, kind, name, namespace and labels are considered for templating\n        apiVersion: v1\n        kind: ConfigMap\n        metadata:\n          name: ($namespace)\n        # other fields are not (they are part of the assertion tree)\n        data:\n          bar: ($namespace)\n
"},{"location":"operations/templating/#apply-create-and-patch","title":"Apply, Create and Patch","text":"

When templating apply, create or patch operations, the whole content is considered for templating.

apply and create example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: template\nspec:\n  template: true\n  steps:\n  - apply:\n      resource:\n        # the whole content is considered for templating\n        apiVersion: v1\n        kind: ConfigMap\n        metadata:\n          name: ($namespace)\n        data:\n          foo: ($namespace)\n  - create:\n      resource:\n        # the whole content is considered for templating\n        apiVersion: v1\n        kind: ConfigMap\n        metadata:\n          name: ($namespace)\n        data:\n          foo: ($namespace)\n  - patch:\n      resource:\n        # the whole content is considered for templating\n        apiVersion: v1\n        kind: ConfigMap\n        metadata:\n          name: ($namespace)\n        data:\n          foo: ($namespace)\n
"},{"location":"operations/templating/#delete","title":"Delete","text":"

When templating delete operations, the whole content is considered for templating.

apply and create example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: template\nspec:\n  template: true\n  steps:\n  - delete:\n      ref:\n        # the whole content is considered for templating\n        apiVersion: v1\n        kind: ConfigMap\n        name: ($namespace)\n
"},{"location":"operations/update/","title":"Update","text":"

The update operation lets you define resources that should be updated in the Kubernetes cluster during the test step. These can be configurations, deployments, services, or any other Kubernetes resource.

"},{"location":"operations/update/#configuration","title":"Configuration","text":"

Reference documentation

Warning

If the resource to be updated doesn't exist in the cluster, the step will fail.

"},{"location":"operations/update/#usage-examples","title":"Usage examples","text":"

Below is an example of using update in a Test resource.

Using a specific file

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - update:\n        file: my-configmap.yaml\n    # ...\n

Using file path expressions

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example-multi\nspec:\n  steps:\n  - try:\n    # ...\n    - update:\n        file: \"configs/*.yaml\"\n    # ...\n

Using an URL

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - update:\n        file: https://raw.githubusercontent.com/kyverno/chainsaw/main/testdata/resource/valid.yaml\n    # ...\n

Using an inline resource

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - update:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: chainsaw-quick-start\n          data:\n            foo: bar\n    # ...\n
"},{"location":"operations/update/#operation-check","title":"Operation check","text":"

Below is an example of using an operation check.

With check

# ...\n- update:\n    file: my-configmap.yaml\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: ConfigMap\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n

With check

# ...\n- update:\n    resource:\n      apiVersion: v1\n      kind: ConfigMap\n      metadata:\n        name: chainsaw-quick-start\n      data:\n        foo: bar\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: ConfigMap\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n
"},{"location":"operations/wait/","title":"Wait","text":"

The wait operation is a wrapper around kubectl wait. It allows to wait for deletion or conditions against resources.

"},{"location":"operations/wait/#configuration","title":"Configuration","text":"

Reference documentation

The full structure of the Wait is documented here.

Deprecated syntax

You can specify the resource directly instead of using apiVersion and kind.

This is a deprecated syntax though and will be removed in a future version.

"},{"location":"operations/wait/#clustered-resources","title":"Clustered resources","text":"

When used with a clustered resource, the namespace is ignored and is not added to the corresponding kubectl command.

"},{"location":"operations/wait/#all-resources","title":"All resources","text":"

If you don't specify a name or a selector, the wait operation will consider all resources.

"},{"location":"operations/wait/#all-namespaces","title":"All namespaces","text":"

When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.

"},{"location":"operations/wait/#usage-examples","title":"Usage examples","text":"

Below is an example of using wait in a Test resource.

Wait pod ready

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n        timeout: 1m\n        for:\n          condition:\n            name: Ready\n            value: 'true'\n    # ...\n

Wait pod ready in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n        namespace: my-ns\n        timeout: 1m\n        for:\n          condition:\n            name: Ready\n            value: 'true'\n    # ...\n

Wait pods ready using a label selector

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        selector: app=foo\n        timeout: 1m\n        for:\n          condition:\n            name: Ready\n            value: 'true'\n    # ...\n

Wait pod deleted

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n        timeout: 1m\n        for:\n          deletion: {}\n    # ...\n

Wait pod deleted in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n        namespace: my-ns\n        timeout: 1m\n        for:\n          deletion: {}\n    # ...\n

Wait pods deleted using a label selector

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        selector: app=foo\n        timeout: 1m\n        for:\n          deletion: {}\n    # ...\n
"},{"location":"operations/wait/#format","title":"Format","text":"

An optional format can be specified. Supported formats are json and yaml.

If format is not specified, results will be returned in text format.

Use json format

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        format: json\n        # ...\n    catch:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        format: json\n        # ...\n    finally:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        format: json\n        # ...\n

Use yaml format

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        format: yaml\n        # ...\n    catch:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        format: yaml\n        # ...\n    finally:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        format: yaml\n        # ...\n
"},{"location":"quick-start/","title":"Getting started","text":"

Chainsaw is a tool primarily developed to run end-to-end tests in Kubernetes clusters.

It is meant to test Kubernetes operators work as expected by running a sequence of steps and asserting various conditions.

"},{"location":"quick-start/#why-we-made-it","title":"Why we made it?","text":"

While developing Kyverno we need to run end-to-end tests to make sure our admission controller works as expected.

A typical Kyverno end-to-end test

Kyverno can validate, mutate and generate resources based on policies installed in a cluster and a typical test is:

  1. Create a policy
  2. Create a resource
  3. Check that Kyverno acted as expected
  4. Cleanup and move to the next test
"},{"location":"quick-start/#how-to-use-it","title":"How to use it?","text":"

Chainsaw is built with CI tools in mind - you only really need to download and execute it in your build script.

However, installing it on your local machine is entirely possible.

"},{"location":"quick-start/assertion-trees/","title":"Use assertions","text":"

Whether verifying the scaling behavior of a database cluster or ensuring data consistency across instances, Chainsaw's assertion model provides the precision and flexibility needed for comprehensive validation.

Chainsaw allows declaring complex assertions with a simple and no-code approach, allowing assertions based on comparisons beyond simple equality, working with arrays, and other scenarios that could not be achieved before.

Tip

Under the hood, Chainsaw uses kyverno-json assertion trees. Refer to the assertion trees documentation for more details on the supported syntax.

"},{"location":"quick-start/assertion-trees/#beyond-simple-equality","title":"Beyond simple equality","text":"

The assertion below will check that the number of replicas for a deployment is greater than 3 AND less than 6.

Chainsaw doesn't need to know the exact expected number of replicas. The (replicas > 3 && replicas < 6) expression will be evaluated until the result is true or the operation timeout expires (making the assertion fail).

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - assert:\n        resource:\n          apiVersion: v1\n          kind: Deployment\n          metadata:\n            name: foo\n          spec:\n            (replicas > `3` && replicas < `6`): true\n    # ...\n
"},{"location":"quick-start/assertion-trees/#working-with-arrays","title":"Working with arrays","text":"

Chainsaw query language makes it easy to assert on arrays. You can filter and transform arrays to select what you want to assert.

"},{"location":"quick-start/assertion-trees/#filtering","title":"Filtering","text":"

In the example below we are creating a resource, then we assert that a condition with type == 'Ready' exists and has a field matching status: 'True':

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        resource:\n          apiVersion: tempo.grafana.com/v1alpha1\n          kind: TempoStack\n          metadata:\n            name: simplest\n          spec:\n            storage:\n              secret:\n                name: minio\n                type: s3\n            # ...\n    - assert:\n        resource:\n          apiVersion: tempo.grafana.com/v1alpha1\n          kind: TempoStack\n          metadata:\n            name: simplest\n          status:\n            # filter conditions array to keep elements where `type == 'Ready'`\n            # and assert there's a single element matching the filter\n            # and that this element status is `True`\n            (conditions[?type == 'Ready']):\n            - status: 'True'\n
"},{"location":"quick-start/assertion-trees/#comprehensive-reporting","title":"Comprehensive reporting","text":"

Chainsaw offers detailed resource diffs upon assertion failures.

In the example below, the assertion failure message (metadata.annotations.foo: Invalid value: \"null\": Expected value: \"bar\") is augmented with a resource diff.

It provides a clear view of discrepancies between expected and actual resources and gives more context around the specific failure (we can easily identify the owner of the offending pod for example).

| 09:55:50 | deployment | step-1   | ASSERT    | RUN   | v1/Pod @ chainsaw-rare-liger/*\n| 09:56:20 | deployment | step-1   | ASSERT    | ERROR | v1/Pod @ chainsaw-rare-liger/*\n    === ERROR\n    ---------------------------------------------------\n    v1/Pod/chainsaw-rare-liger/example-5477b4ff8c-tnhd9\n    ---------------------------------------------------\n    * metadata.annotations.foo: Invalid value: \"null\": Expected value: \"bar\"\n\n    --- expected\n    +++ actual\n    @@ -1,10 +1,16 @@\n      apiVersion: v1\n      kind: Pod\n      metadata:\n    -  annotations:\n    -    foo: bar\n        labels:\n          app: nginx\n    +  name: example-5477b4ff8c-tnhd9\n        namespace: chainsaw-rare-liger\n    +  ownerReferences:\n    +  - apiVersion: apps/v1\n    +    blockOwnerDeletion: true\n    +    controller: true\n    +    kind: ReplicaSet\n    +    name: example-5477b4ff8c\n    +    uid: 118abe16-ec42-4894-83db-64479c4aac6f\n      spec: {}\n| 09:56:20 | deployment | step-1   | TRY       | DONE  |\n
"},{"location":"quick-start/assertion-trees/#next-step","title":"Next step","text":"

To continue our exploration of the main Chainsaw features, let's look at bindings and resource templating next.

"},{"location":"quick-start/bindings/","title":"Use bindings","text":"

You can think of bindings as a side context where you can store and retrieve data based on keys.

This is particularly useful when some data is only known at runtime. For example, to pass data from one operation to another, to implement resource templating, to fetch data from an external system, etc.

Chainsaw offers some built-in bindings you can directly use in your tests but you can also create your own bindings if needed.

"},{"location":"quick-start/bindings/#inheritance","title":"Inheritance","text":"

Bindings can be configured at the test, step or operation level.

All bindings configured at a given level are automatically inherited in child levels.

JMESPath

Chainsaw uses the JMESPath language, and bindings are implemented using lexical scoping.

"},{"location":"quick-start/bindings/#immutability","title":"Immutability","text":"

Bindings are immutable. This means two bindings can have the same name without overwriting each other.

When a binding is registered it potentially hides other bindings with the same name.

When this binding goes out of scope, previously registered bindings with the same name become visible again.

"},{"location":"quick-start/bindings/#built-in-bindings","title":"Built-in bindings","text":"

The $namespace binding is a good example of a built-in binding provided by Chainsaw. It contains the name of the ephemeral namespace used to execute a test (by default Chainsaw will create an ephemeral namespace for each test).

In the operation below, we are assigning the value of the $namespace binding to an environment variable, and echo it in a script:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - script:\n        env:\n          # assign the value of the `$namespace` binding\n          # to the environment variable `FOO`\n        - name: FOO\n          value: ($namespace)\n        content: echo $FOO\n
"},{"location":"quick-start/bindings/#custom-bindings","title":"Custom bindings","text":"

On top of built-in bindings, you can also create your own ones, combine bindings together, call JMESPath functions using bindings as arguments, etc.

In the test below we create custom bindings at different levels in the test, combine them by calling the join function, assign the result to an environment variable, and echo it in a script:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  # bindings can be declared at the test level\n  bindings:\n  - name: chainsaw\n    value: chainsaw\n  steps:\n    # bindings can also be declared at the step level\n  - bindings:\n    - name: hello\n      value: hello\n    try:\n    - script:\n        # bindings can also be declared at the operation level\n        bindings:\n        - name: awesome\n          value: awesome\n        env:\n          # combined bindings together using the `join` functions and\n          # assign the result to the GREETINGS environment variable\n        - name: GREETINGS\n          value: (join(' ', [$hello, $chainsaw, 'is', $awesome]))\n        content: echo $GREETINGS\n
"},{"location":"quick-start/bindings/#next-step","title":"Next step","text":"

Let's see how bindings can be useful with resource templating.

"},{"location":"quick-start/cleanup/","title":"Control your cleanup","text":"

Unless configured differently, by default Chainsaw will automatically remove the resources it created after a test finishes.

Cleanup happens in reverse order of creation (created last, cleaned up first). This is important, especially when the controller being tested makes use of finalizers.

Overriding cleanup timeout

Note that Chainsaw performs a blocking deletion, that is, it will wait until the resource is not present anymore in the cluster before proceeding with the next resource cleanup.

"},{"location":"quick-start/cleanup/#timeout","title":"Timeout","text":"

A global cleanup timeout can be defined at the configuration level or using command line flags.

It can also be overridden on a per-test or per-step basis but not at the operation level.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  timeouts:\n    # cleanup timeout at the test level\n    cleanup: 30s\n  steps:\n  - timeouts:\n      # cleanup timeout at the step level\n      cleanup: 2m\n    try: ...\n
"},{"location":"quick-start/cleanup/#automatic-cleanup","title":"Automatic cleanup","text":"

After a test, every resource created by Chainsaw will be automatically deleted. This applies to create and apply operations.

In the logs below we can see Chainsaw deletes the previously created resource:

    | 15:21:29 | quick-start | @setup   | CREATE    | OK    | v1/Namespace @ chainsaw-cute-cod\n    | 15:21:29 | quick-start | step-1   | TRY       | RUN   |\n    | 15:21:29 | quick-start | step-1   | APPLY     | RUN   | v1/ConfigMap @ chainsaw-cute-cod/chainsaw-quick-start\n    | 15:21:29 | quick-start | step-1   | CREATE    | OK    | v1/ConfigMap @ chainsaw-cute-cod/chainsaw-quick-start\n    | 15:21:29 | quick-start | step-1   | APPLY     | DONE  | v1/ConfigMap @ chainsaw-cute-cod/chainsaw-quick-start\n    | 15:21:29 | quick-start | step-1   | ASSERT    | RUN   | v1/ConfigMap @ chainsaw-cute-cod/chainsaw-quick-start\n    | 15:21:29 | quick-start | step-1   | ASSERT    | DONE  | v1/ConfigMap @ chainsaw-cute-cod/chainsaw-quick-start\n    | 15:21:29 | quick-start | step-1   | TRY       | DONE  |\n    === step cleanup process start ===\n    | 15:21:29 | quick-start | step-1   | CLEANUP   | RUN   |\n    | 15:21:29 | quick-start | step-1   | DELETE    | RUN   | v1/ConfigMap @ chainsaw-cute-cod/chainsaw-quick-start\n    | 15:21:29 | quick-start | step-1   | DELETE    | OK    | v1/ConfigMap @ chainsaw-cute-cod/chainsaw-quick-start\n    | 15:21:29 | quick-start | step-1   | DELETE    | DONE  | v1/ConfigMap @ chainsaw-cute-cod/chainsaw-quick-start\n    | 15:21:29 | quick-start | step-1   | CLEANUP   | DONE  |\n    === step cleanup process end ===\n    === test cleanup process start ===\n    | 15:21:29 | quick-start | @cleanup | DELETE    | RUN   | v1/Namespace @ chainsaw-cute-cod\n    | 15:21:29 | quick-start | @cleanup | DELETE    | OK    | v1/Namespace @ chainsaw-cute-cod\n    | 15:21:34 | quick-start | @cleanup | DELETE    | DONE  | v1/Namespace @ chainsaw-cute-cod\n    === test cleanup process end ===\n
"},{"location":"quick-start/cleanup/#manual-cleanup","title":"Manual cleanup","text":"

Under certain circumstances, automatic cleanup is not enough and we want to execute custom operations.

Chainsaw allows registering cleanup operations that will be run after automatic cleanup. Custom cleanup operations live at the test step level:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n    # this step will create a local cluster\n  - try:\n    - script:\n        timeout: 1m\n        content: |\n          kind create cluster --name dynamic --kubeconfig ./dynamic\n    # at cleanup time, we want to delete the local cluster we created\n    # and remove the associated kubeconfig\n    cleanup:\n    - script:\n        content: |\n          kind delete cluster --name dynamic\n    - script:\n        content: |\n          rm -f ./dynamic\n
"},{"location":"quick-start/cleanup/#next-step","title":"Next step","text":"

At this point, we covered the main Chainsaw features.

Look at the next steps section to find out what to do next.

"},{"location":"quick-start/completion/","title":"Shell completion","text":"

Once installed, use chainsaw completion command to generate and register the autocompletion script for the specified shell.

Supported shells are:

"},{"location":"quick-start/first-test/","title":"Create a Test","text":"

To create a chainsaw test all you need to do is to create one (or more) YAML file(s).

The recommended approach is to create one folder per test, with a chainsaw-test.yaml file containing one (or more) test definition(s). The test definition can reference other files in the same folder or anywhere else on the file system as needed.

Tip

While chainsaw supports other syntaxes, we strongly recommend the explicit approach.

"},{"location":"quick-start/first-test/#what-is-a-test","title":"What is a test?","text":"

To put it simply, a test can be represented as an ordered sequence of test steps.

In turn, a test step can be represented as an ordered sequence of operations.

"},{"location":"quick-start/first-test/#lets-write-our-first-test","title":"Let's write our first test","text":"

For this quick start, we will create a (very simple) Test with one step and two operations:

  1. Create a ConfigMap from a manifest
  2. Verify the ConfigMap was created and contains the expected data

Follow the instructions below to create the folder and files defining our first test.

"},{"location":"quick-start/first-test/#create-a-test-folder","title":"Create a test folder","text":"
# create test folder\nmkdir chainsaw-quick-start\n\n# enter test folder\ncd chainsaw-quick-start\n
"},{"location":"quick-start/first-test/#create-a-configmap-manifest","title":"Create a ConfigMap manifest","text":"
# create a ConfigMap\ncat > configmap.yaml << EOF\napiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: chainsaw-quick-start\ndata:\n  foo: bar\nEOF\n
"},{"location":"quick-start/first-test/#create-a-test-manifest","title":"Create a test manifest","text":"

By default, Chainsaw will look for a file named chainsaw-test.yaml in every folder.

# create test file\ncat > chainsaw-test.yaml << EOF\napiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: quick-start\nspec:\n  steps:\n  - try:\n    # first operation: create the config map\n    - apply:\n        # file is relative to the test folder\n        file: configmap.yaml\n    # second operation: verify the config map exists and contains the expected data\n    - assert:\n        # file is relative to the test folder\n        file: configmap.yaml\nEOF\n
"},{"location":"quick-start/first-test/#next-step","title":"Next step","text":"

Now we have created our first test, you can continue to the next section to execute it.

"},{"location":"quick-start/install/","title":"Installation","text":"

You can install the pre-compiled binary (in several ways), compile from sources, or run with Docker.

We also provide a GitHub action to easily install Chainsaw in your workflows.

"},{"location":"quick-start/install/#install-the-pre-compiled-binary","title":"Install the pre-compiled binary","text":""},{"location":"quick-start/install/#homebrew-tap","title":"Homebrew tap","text":"

add tap:

brew tap kyverno/chainsaw https://github.com/kyverno/chainsaw\n

install chainsaw:

brew install kyverno/chainsaw/chainsaw\n

Don't forget to specify the tap name

Homebrew core already has a tool named chainsaw.

Be sure that you specify the tap name when installing to install the right tool.

"},{"location":"quick-start/install/#manually","title":"Manually","text":"

Download the pre-compiled binaries for your system from the releases page and copy them to the desired location.

"},{"location":"quick-start/install/#install-using-go-install","title":"Install using go install","text":"

You can install with go install with:

go install github.com/kyverno/chainsaw@latest\n
"},{"location":"quick-start/install/#run-with-docker","title":"Run with Docker","text":"

Chainsaw is also available as a Docker image which you can pull and run:

docker pull ghcr.io/kyverno/chainsaw:<version>\n

Warning

Since Chainsaw relies on files for its operation (like test definitions), you will need to bind mount the necessary directories when running it via Docker.

docker run --rm                             \\\n    -v ./testdata/e2e/:/chainsaw/           \\\n    -v ${HOME}/.kube/:/etc/kubeconfig/      \\\n    -e KUBECONFIG=/etc/kubeconfig/config    \\\n    --network=host                          \\\n    ghcr.io/kyverno/chainsaw:<version>      \\\n    test /chainsaw --config /chainsaw/config.yaml\n
"},{"location":"quick-start/install/#compile-from-sources","title":"Compile from sources","text":"

clone:

git clone https://github.com/kyverno/chainsaw.git\n

build the binaries:

cd chainsaw\ngo mod tidy\nmake build\n

verify it works:

./chainsaw version\n
"},{"location":"quick-start/install/#install-using-nix-package","title":"Install using Nix Package","text":"

To install kyverno-chainsaw, refer to the documentation.

"},{"location":"quick-start/install/#on-nixos","title":"On NixOS","text":"
nix-env -iA nixos.kyverno-chainsaw\n
"},{"location":"quick-start/install/#on-non-nixos","title":"On Non-NixOS","text":"
nix-env -iA nixpkgs.kyverno-chainsaw\n

Warning

Using nix-env permanently modifies a local profile of installed packages. This must be updated and maintained by the user in the same way as with a traditional package manager, foregoing many of the benefits that make Nix uniquely powerful. Using nix-shell or a NixOS configuration is recommended instead.

"},{"location":"quick-start/install/#using-nixos-configuration","title":"Using NixOS Configuration","text":"

Add the following Nix code to your NixOS Configuration, usually located in /etc/nixos/configuration.nix :

environment.systemPackages = [\n  pkgs.kyverno-chainsaw\n];\n
"},{"location":"quick-start/install/#using-nix-shell","title":"Using nix-shell","text":"

A nix-shell will temporarily modify your $PATH environment variable. This can be used to try a piece of software before deciding to permanently install it. Use the following command to install kyverno-chainsaw :

nix-shell -p kyverno-chainsaw\n
"},{"location":"quick-start/install/#github-action","title":"GitHub action","text":"

A GitHub action is available to install Chainsaw in your workflows. See the GitHub action dedicated documentation.

"},{"location":"quick-start/next-steps/","title":"Next steps","text":"

We covered the main features of Chainsaw in this Getting started sections.

While this should help you understand Chainsaw better, there are a lot of other things Chainsaw can do for you.

Tip

If there's anything you would like to be improved, please reach out, we will be happy to discuss and improve as much as we can.

To continue exploring the capabilities of Chainsaw:

"},{"location":"quick-start/next-steps/#additional-resources","title":"Additional resources","text":"

Resources, blog posts and videos talking about Chainsaw:

"},{"location":"quick-start/next-steps/#chainsaw-review-video","title":"Chainsaw review video","text":"

If you haven't watched the video below yet, we strongly recommend watching it to discover Chainsaw!

"},{"location":"quick-start/operation-outputs/","title":"Use operation outputs","text":"

Operation outputs can be useful for communicating and reusing computation results across operations.

"},{"location":"quick-start/operation-outputs/#lifetime-of-outputs","title":"Lifetime of outputs","text":"

Once an output has been added to the bindings context, this binding will be available to all following operations in the same step.

Currently, outputs do not cross the step boundaries.

"},{"location":"quick-start/operation-outputs/#matching","title":"Matching","text":"

An output supports an optional match field. The match is used to conditionally create a binding.

In the case of applying a file, for example, the file may contain multiple resources. The match can be used to select the resource to use for creating the binding.

"},{"location":"quick-start/operation-outputs/#load-an-existing-resource","title":"Load an existing resource","text":"

The example below invokes a kubectl command to get a configmap from the cluster in json format.

The json output is then parsed and added to the $cm binding and the next operation performs an assertion on it by reading the binding instead of querying the cluster.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - script:\n        content: kubectl get cm quick-start -n $NAMESPACE -o json\n        outputs:\n          # parse stdout json output and bind the result to `$cm`\n        - name: cm\n          value: (json_parse($stdout))\n    - assert:\n        resource:\n          ($cm):\n            metadata:\n              (uid != null): true\n
"},{"location":"quick-start/operation-outputs/#match-a-resource","title":"Match a resource","text":"

The example below applies resources from a file.

When the resource being applied is a configmap, we bind the resource to an output to print its UID in the next operation.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        file: ./resources.yaml\n        outputs:\n          # match the configmap resource and bind it to `$cm`\n        - match:\n            apiVersion: v1\n            kind: ConfigMap\n          name: cm\n          value: (@)\n    - script:\n        env:\n        - name: UID\n          value: ($cm.metadata.uid)\n        content: echo $UID\n
"},{"location":"quick-start/operation-outputs/#next-step","title":"Next step","text":"

In the next section, we will look at the three main elements of a test step, the try, catch and finally blocks.

"},{"location":"quick-start/resource-templating/","title":"Use resource templating","text":"

Chainsaw simplifies dynamic resource configuration with native resource templating support.

Sometimes things we need to create resources or assertions are only known at runtime.

In the past, users have created all sorts of hacks using tools like envsubst for dynamic substitution of env-variables. Those workarounds usually lack flexibility and introduce new problems like hiding the real resources from Chainsaw, preventing it from cleaning resources properly.

Tip

Resource templating is heavily based on bindings and uses JMESPath language.

"},{"location":"quick-start/resource-templating/#leverage-bindings","title":"Leverage bindings","text":"

In the template below, we are using the $namespace binding at two different places, effectively injecting the ephemeral namespace name in the name and the data.foo fields:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - assert:\n      resource:\n        apiVersion: v1\n        kind: ConfigMap\n        metadata:\n          name: ($namespace)\n        data:\n          foo: ($namespace)\n
"},{"location":"quick-start/resource-templating/#leverage-jmespath","title":"Leverage JMESPath","text":"

In the template below, we are using the JMESPath join function to create a unique resource name:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - apply:\n      resource:\n        apiVersion: v1\n        kind: ConfigMap\n        metadata:\n          name: (join('-', [$namespace, 'cm']))\n        data:\n          foo: bar\n
"},{"location":"quick-start/resource-templating/#next-step","title":"Next step","text":"

Combining bindings and templates with operation outputs allows even more flexibility to pass arbitrary data from one operation to another.

"},{"location":"quick-start/run-tests/","title":"Run tests","text":"

After installing chainsaw and writing tests, the next natural step is to run Chainsaw to execute the tests.

"},{"location":"quick-start/run-tests/#create-a-local-cluster","title":"Create a local cluster","text":"

To use Chainsaw you will need a Kubernetes cluster, Chainsaw won't create one for you.

Not a cluster management tool

We consider this is not the responsibility of Chainsaw to manage clusters. There are plenty of solutions to create and manage local clusters that will do that better than Chainsaw.

The command below will create a local cluster using kind. Use the tool of your choice or directly jump to the next section if you already have a KUBECONFIG configured and pointing to a valid cluster.

# create cluster\nkind create cluster --image \"kindest/node:v1.29.4\"\n
"},{"location":"quick-start/run-tests/#run-chainsaw","title":"Run Chainsaw","text":"

Now you can run the chainsaw test command.

> chainsaw test\n\nVersion: (devel)\nLoading default configuration...\n- Using test file: chainsaw-test.yaml\n- TestDirs [.]\n- SkipDelete false\n- FailFast false\n- ReportFormat ''\n- ReportName 'chainsaw-report'\n- Namespace ''\n- FullName false\n- IncludeTestRegex ''\n- ExcludeTestRegex ''\n- ApplyTimeout 5s\n- AssertTimeout 30s\n- CleanupTimeout 30s\n- DeleteTimeout 15s\n- ErrorTimeout 30s\n- ExecTimeout 5s\nLoading tests...\n- quick-start (.)\nRunning tests...\n=== RUN   chainsaw\n=== PAUSE chainsaw\n=== CONT  chainsaw\n=== RUN   chainsaw/quick-start\n=== PAUSE chainsaw/quick-start\n=== CONT  chainsaw/quick-start\n    | 10:44:26 | quick-start | @setup   | CREATE    | OK    | v1/Namespace @ chainsaw-immense-jay\n    | 10:44:26 | quick-start | step-1   | TRY       | RUN   |\n    | 10:44:26 | quick-start | step-1   | APPLY     | RUN   | v1/ConfigMap @ chainsaw-immense-jay/chainsaw-quick-start\n    | 10:44:26 | quick-start | step-1   | CREATE    | OK    | v1/ConfigMap @ chainsaw-immense-jay/chainsaw-quick-start\n    | 10:44:26 | quick-start | step-1   | APPLY     | DONE  | v1/ConfigMap @ chainsaw-immense-jay/chainsaw-quick-start\n    | 10:44:26 | quick-start | step-1   | ASSERT    | RUN   | v1/ConfigMap @ chainsaw-immense-jay/chainsaw-quick-start\n    | 10:44:26 | quick-start | step-1   | ASSERT    | DONE  | v1/ConfigMap @ chainsaw-immense-jay/chainsaw-quick-start\n    | 10:44:26 | quick-start | step-1   | TRY       | DONE  |\n    | 10:44:26 | quick-start | @cleanup | DELETE    | RUN   | v1/ConfigMap @ chainsaw-immense-jay/chainsaw-quick-start\n    | 10:44:26 | quick-start | @cleanup | DELETE    | OK    | v1/ConfigMap @ chainsaw-immense-jay/chainsaw-quick-start\n    | 10:44:26 | quick-start | @cleanup | DELETE    | DONE  | v1/ConfigMap @ chainsaw-immense-jay/chainsaw-quick-start\n    | 10:44:26 | quick-start | @cleanup | DELETE    | RUN   | v1/Namespace @ chainsaw-immense-jay\n    | 10:44:26 | quick-start | @cleanup | DELETE    | OK    | v1/Namespace @ chainsaw-immense-jay\n    | 10:44:31 | quick-start | @cleanup | DELETE    | DONE  | v1/Namespace @ chainsaw-immense-jay\n--- PASS: chainsaw (0.00s)\n    --- PASS: chainsaw/quick-start (5.25s)\nPASS\nTests Summary...\n- Passed  tests 1\n- Failed  tests 0\n- Skipped tests 0\nDone.\n

Tip

Chainsaw expects a path to the test folder and will discover tests by analyzing files recursively. When no path is provided Chainsaw will use the current path by default (.).

"},{"location":"quick-start/run-tests/#next-step","title":"Next step","text":"

The test above demonstrates the most basic usage of Chainsaw. In the next sections, we will look at the main features that make Chainsaw a very unique tool.

"},{"location":"quick-start/timeouts/","title":"Control your timeouts","text":"

Timeouts in Chainsaw are specified per type of operation. This is handy because the timeout varies greatly depending on the nature of an operation.

For example, applying a manifest in a cluster is expected to be reasonably fast, while validating a resource can be a long operation.

"},{"location":"quick-start/timeouts/#inheritance","title":"Inheritance","text":"

Timeouts can be configured globally and at the test, step or individual operation level.

All timeouts configured at a given level are automatically inherited in child levels. When looking up a timeout, the most specific one takes precedence over the others.

Info

To learn more about timeouts and how to configure global values, see the timeouts configuration page.

"},{"location":"quick-start/timeouts/#at-the-test-level","title":"At the test level","text":"

When a timeout is configured at the test level it will apply to all operations and steps in the test, unless overridden at a more specific level.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  # timeouts configured at the test level will apply to all operations and steps\n  # unless overriden at the step level and/or individual operation level\n  timeouts:\n    apply: 5s\n    assert: 1m\n    # ...\n  steps:\n  - try:\n    - apply:\n        resource:\n          apiVersion: tempo.grafana.com/v1alpha1\n          kind: TempoStack\n          metadata:\n            name: simplest\n          spec:\n            storage:\n              secret:\n                name: minio\n                type: s3\n            # ...\n    - assert:\n        resource:\n          apiVersion: tempo.grafana.com/v1alpha1\n          kind: TempoStack\n          metadata:\n            name: simplest\n          status:\n            (conditions[?type == 'Ready']):\n            - status: 'True'\n
"},{"location":"quick-start/timeouts/#at-the-step-level","title":"At the step level","text":"

When a timeout is configured at the step level it will apply to all operations in the step, unless overridden at a more specific level.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n    # timeouts configured at the step level will apply to all operations\n    # in the step unless overriden at the individual operation level\n  - timeouts:\n      apply: 5s\n      # ...\n    try:\n    - apply:\n        resource:\n          apiVersion: tempo.grafana.com/v1alpha1\n          kind: TempoStack\n          metadata:\n            name: simplest\n          spec:\n            storage:\n              secret:\n                name: minio\n                type: s3\n            # ...\n    # timeouts configured at the step level will apply to all operations\n    # in the step unless overriden at the individual operation level\n  - timeouts:\n      assert: 1m\n      # ...\n    try:\n    - assert:\n        resource:\n          apiVersion: tempo.grafana.com/v1alpha1\n          kind: TempoStack\n          metadata:\n            name: simplest\n          status:\n            (conditions[?type == 'Ready']):\n            - status: 'True'\n
"},{"location":"quick-start/timeouts/#at-the-operation-level","title":"At the operation level","text":"

When a timeout is configured at the operation level, it takes precedence over all timeouts configured at upper levels.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        # timeout configured at the operation level takes precedence\n        # over timeouts configured at upper levels\n        timeout: 5s\n        resource:\n          apiVersion: tempo.grafana.com/v1alpha1\n          kind: TempoStack\n          metadata:\n            name: simplest\n          spec:\n            storage:\n              secret:\n                name: minio\n                type: s3\n            # ...\n    - assert:\n        # timeout configured at the operation level takes precedence\n        # over timeouts configured at upper levels\n        timeout: 1m\n        resource:\n          apiVersion: tempo.grafana.com/v1alpha1\n          kind: TempoStack\n          metadata:\n            name: simplest\n          status:\n            (conditions[?type == 'Ready']):\n            - status: 'True'\n
"},{"location":"quick-start/timeouts/#next-step","title":"Next step","text":"

In the next section, we will see how Chainsaw manages cleanup.

"},{"location":"quick-start/try-catch/","title":"Use try, catch and finally","text":"

A test step is made of 3 main blocks used to determine the actions Chainsaw will perform when executing the step, depending on the test outcome.

Operations defined in the try block are executed first, then:

Note

Note that all operations coming from the catch or finally blocks are executed. If one operation fails, Chainsaw will mark the test as failed and continue executing with the next operation.

"},{"location":"quick-start/try-catch/#cleanup","title":"Cleanup","text":"

At the end of a test, Chainsaw automatically cleans up the resources created during the test (cleanup is done in the opposite order of creation).

All operations from the catch and finally blocks are executed before the cleanup process kicks in. This order allows analyzing the resources that potentially caused the step failure before they are deleted.

"},{"location":"quick-start/try-catch/#catch","title":"Catch","text":"

Operations in a catch block are executed only when the step is considered failed.

This is particularly useful to collect additional information to help understand what caused the failure.

In the example below, the test contains a catch block to collect events in the cluster when an operation fails in the step.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        # ...\n    - assert:\n        # ...\n    # collect events in the `catch` block\n    # will be executed only if an operation failed\n    catch:\n    - events: {}\n
"},{"location":"quick-start/try-catch/#finally","title":"Finally","text":"

Operations in a finally block will always execute regardless of the success or failure of the test step.

This is particularly useful to perform manual cleanup.

In the example below we create a local cluster in a script operation. The cluster deletion script is added to the finally block, guaranteeing the cluster will be deleted regardless of the test outcome.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n    # create a local cluster\n  - try:\n    - script:\n        timeout: 1m\n        content: |\n          kind create cluster --name dynamic --kubeconfig ./dynamic\n    - apply:\n        # ...\n    - assert:\n        # ...\n    # add cluster deletion script in the `finally` block\n    # to guarantee the cluster will be deleted after the test\n    finally:\n    - script:\n        content: |\n          kind delete cluster --name dynamic\n    - script:\n        content: |\n          rm -f ./dynamic\n
"},{"location":"quick-start/try-catch/#next-step","title":"Next step","text":"

Every operation in a test must be executed in a timely fashion. In the next section, we will see how you can control your timeouts.

"},{"location":"quick-start/advanced/","title":"Advanced use cases","text":"

This section is under construction and should be ready soon. Please check again in a few days.

If you are interested in contributing see the tracking issue.

"},{"location":"quick-start/advanced/cluster-wide-resources/","title":"Cluster-wide resources","text":"

This section is under construction and should be ready soon. Please check again in a few days.

If you are interested in contributing see the tracking issue.

"},{"location":"quick-start/advanced/concurrency/","title":"Concurrency control","text":"

By default, Chainsaw will run tests in parallel.

The number of concurrent tests can be configured globally using a configuration file or with the --parallel flag.

Alternatively, the concurrent nature of a test can specified at the test level:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  # concurrency can be specified per test (`true` or `false`)\n  # default value is `true`\n  concurrent: true\n  # ...\n

All non-concurrent tests are executed first, followed by the concurrent tests running in parallel.

"},{"location":"quick-start/advanced/crds/","title":"Working with CRDs","text":"

New Custom Resource Definitions are not immediately available for use in the Kubernetes API until the Kubernetes API has acknowledged them.

If a Custom Resource Definition is being defined inside of a test step, be sure to wait for the CustomResourceDefinition object to appear.

The test below applies a CRD and waits for it to become available:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        resource:\n          apiVersion: apiextensions.k8s.io/v1\n          kind: CustomResourceDefinition\n          metadata:\n            name: issues.example.com\n          spec:\n            group: example.com\n            names:\n              kind: Issue\n              listKind: IssueList\n              plural: issues\n              singular: issue\n            scope: Namespaced\n            versions: ...\n    - assert:\n        resource:\n          apiVersion: apiextensions.k8s.io/v1\n          kind: CustomResourceDefinition\n          metadata:\n            name: issues.example.com\n          status:\n            acceptedNames:\n              kind: Issue\n              listKind: IssueList\n              plural: issues\n              singular: issue\n            storedVersions:\n            - v1alpha1\n

The CRD can be used in subsequent steps.

"},{"location":"quick-start/advanced/inline/","title":"Inline resources","text":"

When an operation needs to reference a resource, it can do so using a file path or directly specify the resource inline using the resource field.

The test below is equivalent to our first test:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: quick-start\n          data:\n            foo: bar\n    - assert:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: quick-start\n          data:\n            foo: bar\n
"},{"location":"quick-start/advanced/kube-version/","title":"Check Kubernetes version","text":"

The test below fetches the Kubernetes cluster version using x_k8s_server_version. It then uses the minor version retrieved to adapt an assertion based on the value in the $minorversion binding.

Tip

You can implement a ternary operator in JMESPath using an expression like this:

<condition> && <value-if-true> || <value-if-false>

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  bindings:\n  - name: version\n    value: (x_k8s_server_version($config))\n  - name: minorversion\n    value: (to_number($version.minor))\n  steps:\n  - try:\n    - apply:\n        resource:\n          apiVersion: v1\n          kind: Pod\n          metadata:\n            name: pod01\n          spec:\n            containers:\n            - name: busybox\n              image: busybox:1.35\n    # ...\n    - assert:\n        resource:\n          apiVersion: v1\n          kind: Pod\n          metadata:\n            annotations:\n              # If the minor version of the Kubernetes cluster against which this\n              # is tested is less than 29, the annotation is expected to have the group 'system:masters' in it.\n              # Otherwise, due to a change in kubeadm, the group should be 'kubeadm:cluster-admins'.\n              kyverno.io/created-by: (($minorversion < `29` && '{\"groups\":[\"system:masters\",\"system:authenticated\"],\"username\":\"kubernetes-admin\"}') || '{\"groups\":[\"kubeadm:cluster-admins\",\"system:authenticated\"],\"username\":\"kubernetes-admin\"}')\n            name: pod01\n
"},{"location":"quick-start/advanced/label-selectors/","title":"Use label selectors","text":"

Chainsaw can filter the tests to run using label selectors.

You can pass label selectors using the --selector flag when invoking the chainsaw test command.

Given the test below:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: basic\n  labels:\n    foo: bar\nspec:\n  # ...\n

Invoking Chainsaw with the command below will take the test above into account:

chainsaw test --selector foo=bar\n
"},{"location":"quick-start/advanced/multi-cluster/","title":"Multi-cluster setup","text":"

This section is under construction and should be ready soon. Please check again in a few days.

If you are interested in contributing see the tracking issue.

"},{"location":"quick-start/advanced/negative-testing/","title":"Negative testing","text":"

Negative testing is the process of testing cases that are supposed to fail. That is, a test expects errors to happen and if the expected errors don't occur the test must fail.

Chainsaw supports negative testing by letting you decide what should be considered an error or not.

Tip

By default, Chainsaw will consider an operation failed if there was an error executing it (non-zero exit code in scripts and commands, error returned by the API server when calling into Kubernetes, etc...).

"},{"location":"quick-start/advanced/negative-testing/#script-case","title":"Script case","text":"

The test below expects an error and validates the returned error message:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - script:\n        content: kubectl get foo\n        check:\n          ($error != null): true\n          ($stderr): |-\n            error: the server doesn't have a resource type \"foo\"\n

If for whatever reason, the kubectl get foo doesn't return an error, or the message received in standard error output is not error: the server doesn't have a resource type \"foo\", Chainsaw will consider the operation failed.

If it returns an error and the expected error message, Chainsaw will consider the operation successful.

"},{"location":"quick-start/advanced/negative-testing/#working-with-resources","title":"Working with resources","text":"

The test below tries to apply resources in a cluster but expects the operation to fail:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        file: resources.yaml\n        expect:\n          # check that applying the resource failed\n        - check:\n            ($error != null): true\n

If applying the resource succeeded, Chainsaw will consider the operation failed.

On the other hand, if applying the resource fails, Chainsaw will consider the operation to be successful.

"},{"location":"quick-start/advanced/negative-testing/#resource-matching","title":"Resource matching","text":"

In the previous example, if the resources.yaml contains multiple resources, but only some of them may be expected to fail.

Chainsaw allows matching resources when evaluating checks:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        file: resources.yaml\n        expect:\n          # the check below only applies if the resource being checked\n          # matches the condition defined in the `match` field\n        - match:\n            apiVersion: v1\n            kind: ConfigMap\n            metadata:\n              name: quick-start\n          check:\n            ($error != null): true\n

Using the match field, we can easily target failures related to specific resources.

"},{"location":"quick-start/advanced/non-resource-assertions/","title":"Non-resource assertions","text":"

Under certain circumstances, it makes sense to evaluate assertions that do not depend on resources. For example, when asserting the number of nodes in a cluster is equal to a known value.

The test below uses the x_k8s_list function to query the list of nodes in the cluster. It uses the results to compare the number of nodes found with a known number (1 in this case).

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - assert:\n        resource:\n          (x_k8s_list($client, 'v1', 'Node')):\n            (length(items)): 1\n
"},{"location":"quick-start/advanced/test-output/","title":"Test command output","text":"

Chainsaw can be used to easily check terminal output from CLIs and other commands. This is useful in that convoluted bash scripts involving chaining together tools like grep can be avoided or at least minimized to only complex use cases. Output to both stdout and stderr can be checked for a given string or precise contents.

"},{"location":"quick-start/advanced/test-output/#checking-output-contains","title":"Checking Output Contains","text":"

One basic use case for content checking is that the output simply contains a given string or piece of content. For example, you might want to run automated tests on a CLI binary you build to ensure that a given command produces output that contains some content you specify somewhere in the output. Let's use the following output from the kubectl version command to show these examples.

kubectl version\n\nClient Version: v1.28.2\nKustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3\nServer Version: v1.27.4+k3s1\n

Below is an example that ensures the string '1.28' is found somewhere in that output. So long as the content is present anywhere, the test will succeed. To perform this check, the contains() JMESPath filter is used.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: test\nspec:\n  steps:\n  - name: Check kubectl\n    try:\n    - script:\n        content: kubectl version\n        check:\n          # This check ensures that the string '1.28' is found\n          # in stdout or else fails\n          (contains($stdout, '1.28')): true\n

Checks for content containing a given value can be negated as well. For example, checking to ensure the output does NOT contain the string '1.25'.

- script:\n    content: kubectl version\n    check:\n      # This check ensures that the string '1.25' is NOT found\n      # in stdout or else fails\n      (contains($stdout, '1.25')): false\n
"},{"location":"quick-start/advanced/test-output/#checking-output-is-exactly","title":"Checking Output Is Exactly","text":"

In addition to checking that CLI/command output contains some contents, you may need to ensure that the contents are exactly as intended. The Chainsaw test below accomplishes this by comparing the entire contents of stdout with those specified in the block scalar. If so much as one character, space, or line break is off, the test will fail. This is useful in that not only can content be checked but the formatting of that content can be ensured it matches a given declaration.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: test\nspec:\n  steps:\n  - name: Check kubectl\n    try:\n    - script:\n        content: kubectl version\n        check:\n          # This check ensures the contents of stdout are exactly as shown.\n          # Any deviations will cause a failure.\n          ($stdout): |-\n            Client Version: v1.28.2\n            Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3\n            Server Version: v1.27.4+k3s1\n
"},{"location":"quick-start/advanced/test-output/#checking-output-in-errors","title":"Checking Output In Errors","text":"

In addition to testing that commands succeed and with output in a given shape, it's equally valuable and necessary to perform negative tests; that tests fail and with contents that are as expected. Similarly, those checks can be for output which has some contents as well as output which appears exactly as desired. For example, you may wish to check that running the kubectl foo command not only fails as expected but that the output shown to users contains a certain word or sentence.

kubectl foo\n\nerror: unknown command \"foo\" for \"kubectl\"\n\nDid you mean this?\n        top\n

Below you can see an example where the command kubectl foo is expected to fail but that the error message returned contains some output, in this case the string 'top'.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: test\nspec:\n  steps:\n  - name: Check bad kubectl command\n    try:\n    - script:\n        content: kubectl foo\n        check:\n          # This checks that the result of the content was an error.\n          ($error != null): true\n          # This check below ensures that the string 'top' is found in stderr or else fails\n          (contains($stderr, 'top')): true\n

Likewise, this failure output can be checked that it is precise. Note that in the example below, due to the use of a tab character in the output of kubectl foo, the value of the ($stderr) field is given as a string to preserve these non-printing characters.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: test\nspec:\n  steps:\n  - name: Check kubectl\n    try:\n    - script:\n        content: kubectl foo\n        check:\n          # This checks that the result of the content was an error.\n          ($error != null): true\n          # This checks that the output is exactly as intended.\n          ($stderr): \"error: unknown command \\\"foo\\\" for \\\"kubectl\\\"\\n\\nDid you mean this?\\n\\ttop\"\n
"},{"location":"reference/","title":"Reference documentation","text":""},{"location":"steps/","title":"What is a test step","text":"

A test step is made of three main components used to determine the actions Chainsaw will perform when executing the step.

  1. The try statement (required)
  2. The catch statement (optional)
  3. The finally statement (optional)

Reference documentation

The full structure of the TestStep is documented here.

"},{"location":"steps/#test-step-lifecycle","title":"Test step lifecycle","text":"

Test step lifecycle

  1. The step starts executing operations in the try statement
  2. If an operation fails in the try statement
    1. If a catch statement is present, all operations and collectors are executed
  3. If a finally statement is present, all operations and collectors are executed
"},{"location":"steps/#example","title":"Example","text":"

Example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        file: path/to/apply.yaml\n    - assert:\n        file: path/to/assert.yaml\n    catch: []\n    finally: []\n
"},{"location":"steps/catch/","title":"Catch","text":"

A catch statement is also a sequence of operations or collectors.

Operations and collectors contained in a catch statement will be executed only if the step failed when executing the operations in the step's try statement.

Tip

All operations and collectors of a catch statement will be executed regardless of the success or failure of each of them.

"},{"location":"steps/catch/#more-general-catch-blocks","title":"More general catch blocks","text":"

Under certain circumstances, it can be useful to configure catch blocks at a higher level than the step grain. At the test or configuration level.

This allows for declaring common catch statements we want to execute when an error occurs. Those catch blocks are combined to produce the final catch block in the following order:

  1. catch statements from the configuration level are executed first (if any)
  2. catch statements from the test level are executed next (if any)
  3. catch statements from the step level are executed last (if any)
"},{"location":"steps/catch/#operations","title":"Operations","text":"

A catch statement supports only the following operations:

"},{"location":"steps/catch/#collectors","title":"Collectors","text":"

A catch statement supports all collectors:

"},{"location":"steps/catch/#examples","title":"Examples","text":"

step level catch block

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try: []\n    catch:\n    - command:\n        # ...\n    - script:\n        # ...\n    - delete:\n        # ...\n    - events:\n        # ...\n    - podLogs:\n        # ...\n    - describe:\n        # ...\n    - get:\n        # ...\n    - sleep:\n        # ...\n    - wait:\n        # ...\n    finally: []\n

test level catch block

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  catch:\n  - command:\n      # ...\n  - script:\n      # ...\n  - delete:\n      # ...\n  - events:\n      # ...\n  - podLogs:\n      # ...\n  - describe:\n      # ...\n  - get:\n      # ...\n  - sleep:\n      # ...\n  - wait:\n      # ...\n  steps:\n  - try: []\n    finally: []\n
"},{"location":"steps/finally/","title":"Finally","text":"

A finally statement is similar to a catch statement but will always execute after the try and eventual catch statements finished executing regardless of the success or failure of the test step.

Tip

All operations and collectors of a finally statement will be executed regardless of the success or failure of each of them.

"},{"location":"steps/finally/#operations","title":"Operations","text":"

A finally statement supports only the following operations:

"},{"location":"steps/finally/#collectors","title":"Collectors","text":"

A finally statement supports all collectors:

"},{"location":"steps/finally/#example","title":"Example","text":"

Example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try: []\n    catch: []\n    finally:\n    - command:\n        # ...\n    - script:\n        # ...\n    - delete:\n        # ...\n    - events:\n        # ...\n    - podLogs:\n        # ...\n    - describe:\n        # ...\n    - get:\n        # ...\n    - sleep:\n        # ...\n    - wait:\n        # ...\n
"},{"location":"steps/try/","title":"Try","text":"

A try statement is a sequence of operations executed in the same order they are declared. If an operation fails the entire step is considered failed.

The try statement is at the heart of a test step, it represents what the step is supposed to be about.

catch and finally statements should be viewed as complementary to the try statement.

Continue on error

By default, a test step stops executing when an operation fails and the following operations are not executed.

This behavior can be changed using the continueOnError field, if continueOnError is set to true the step will still be considered failed but execution will continue with the next operations.

"},{"location":"steps/try/#operations","title":"Operations","text":"

A try statement supports all operations:

"},{"location":"steps/try/#example","title":"Example","text":"

Example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - command:\n        # ...\n    - sleep:\n        # ...\n    - script:\n        # ...\n    - apply:\n        # ...\n    - assert:\n        # ...\n    - error:\n        # ...\n    - create:\n        # ...\n    - patch:\n        # ...\n    - wait:\n        # ...\n    catch: []\n    finally: []\n
"},{"location":"tests/","title":"What is a test","text":"

To put it simply, a test can be represented as an ordered sequence of test steps.

Test steps within a test are run sequentially: if any of the test steps fail, the entire test is considered failed.

A test step can consist in one or more operations:

"},{"location":"tests/#different-syntaxes","title":"Different syntaxes","text":"

Chainsaw supports two different test definition syntaxes. Each syntax has pros and cons, see the descriptions below for more details on advantages and pitfalls.

"},{"location":"tests/#manifests-based-syntax","title":"Manifests based syntax","text":"

This is the simplest and less verbose syntax.

You provide bare Kubernetes resource manifests and Chainsaw will use those manifests to create, update, or assert expectations against a cluster.

While this syntax is extremly simple, not all operations are supported. For example, delete, command, script and sleep operations are not.

Another strong limitation is that it is not possible to specify additional configuration per test, step or operation.

Finally, this syntax relies heavily on file naming conventions, it can be error prone and makes it hard to reuse files across multiple tests.

"},{"location":"tests/#test-based-syntax","title":"Test based syntax","text":"

The Test based syntax is a more verbose and explicit syntax.

It does not rely on file naming conventions to determine test steps order and allows to easily reuse files accross multiple tests.

This syntax also comes with no limitations to provide additional configuration at the test, step or operation level.

Making a choice

Choosing one syntax over the other is not a trivial choice, every one will have its own preference and/or constraints.

It's usually easier to start with the manifests based syntax. However, as test suites grow and tests become more complex, it is often necessary to configure options on a per test, step or operation basis and the Test based syntax becomes necessary.

"},{"location":"tests/#namespaced-resources","title":"Namespaced resources","text":"

Kubernetes organizes resources into two primary scopes: namespaced and cluster-scoped.

While namespaced resources belong to a specific namespace, cluster-scoped resources span across the entire Kubernetes cluster.

"},{"location":"tests/#purpose-of-the-namespacer-interface","title":"Purpose of the Namespacer Interface","text":"

The Namespacer interface ensures automated and consistent namespace assignment to Kubernetes resources.

"},{"location":"tests/#cleanup","title":"Cleanup","text":"

Unless configured differently, by default Chainsaw will automatically cleanup the resources it created after a test finishes. Cleanup happens in reverse order of creation (created last, cleaned up first).

Note that Chainsaw performs a blocking deletion, that is, it will wait the resource is actually not present anymore in the cluster before proceeding with the next resource cleanup.

This is important, especially when the controller being tested makes use of finalizers.

Overriding cleanup timeout

A global cleanup timeout can be defined at the configuration level or using command line flags.

It can also be overriden on a per test or per test step basis but not at the operation level.

"},{"location":"tests/#parallel-execution-of-tests","title":"Parallel Execution of Tests","text":"

While Chainsaw ensures that the steps within a test are executed sequentially, it is designed to run multiple tests in parallel to each other. This parallel execution helps in significantly reducing the overall time required to run an entire suite of tests, making the testing process more efficient, especially in scenarios with a large number of tests.

Parallel execution of tests

By default, Chainsaw will run tests in parallel.

This can be configured at the configuration level or using command line flags. However, individual tests can be configured to run concurrently by setting Concurrent: true in their TestSpec.

All non-concurrent tests are executed first, followed by the concurrent tests in parallel.

"},{"location":"tests/manifests-based/","title":"Manifests based syntax","text":"

This is the simplest and less verbose supported syntax.

You provide bare Kubernetes resource manifests and Chainsaw will use those manifests to create, update, or assert expectations against a cluster.

While this syntax is simple, it doesn't support deletion operations and doesn't allow specifying additional configuration per test or step.

It also relies a lot on file naming conventions and makes it hard to reuse files across multiple tests.

"},{"location":"tests/manifests-based/#file-naming-convention","title":"File naming convention","text":"

Manifest files must follow the naming convention <step index>-<name|assert|errors>.yaml.

As an example 00-configmap.yaml, 01-assert.yaml and 02-errors.yaml can all be considered valid file names.

It's also perfectly valid to have multiple files for the same step. Let's imagine we have the following files 00-resources.yaml, 00-more-resources.yaml, 00-assert.yaml and 00-errors.yaml:

With the four files above, Chainsaw will assemble a test step made of the combination of all those files.

"},{"location":"tests/manifests-based/#manifests-loading-process","title":"Manifests loading process","text":"

The logic to determine the content of a step is always:

"},{"location":"tests/manifests-based/#example","title":"Example","text":""},{"location":"tests/manifests-based/#01-configmapyaml","title":"01-configmap.yaml","text":"

The manifest below contains a config map in a file called 01-configmap.yaml. Chainsaw will associate this manifest with an apply operation in step 01.

apiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: chainsaw-quick-start\ndata:\n  foo: bar\n
"},{"location":"tests/manifests-based/#02-assertyaml","title":"02-assert.yaml","text":"

The manifest below contains an assertion statement in a file called 02-assert.yaml. Chainsaw will associate this manifest with an assert operation in step 02.

apiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: chainsaw-quick-start\ndata:\n  foo: bar\n
"},{"location":"tests/manifests-based/#03-errorsyaml","title":"03-errors.yaml","text":"

The manifest below contains an error statement in a file called 03-errors.yaml. Chainsaw will associate this manifest with an error operation in step 03.

apiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: chainsaw-quick-start\ndata:\n  lorem: ipsum\n
"},{"location":"tests/manifests-based/#conclusion","title":"Conclusion","text":"

This test will first create a config map, then assert the content of the config map contains the foo: bar data, and then verify that the configmap does not contain the lorem: ipsum data.

Assert and error statements are very similar to standard resource definitions, they share the same structure but can be partial. Chainsaw will process only the elements present in the statements against the actual resource definition.

"},{"location":"tests/test-based/","title":"Test based syntax","text":"

The Test based syntax is more verbose than the manifests based syntax but offers more flexibility and features:

"},{"location":"tests/test-based/#the-test-resource","title":"The Test resource","text":"

A Test resource, like any Kubernetes resource has an apiVersion, kind and metadata section.

It also comes with a spec section used to declaratively represent the steps of a test and other configuration elements belonging to the test being defined.

Reference documentation

The full structure of the Test resource is documented here.

"},{"location":"tests/test-based/#test-steps","title":"Test steps","text":"

A Test is mostly made of test steps. Test steps are detailed in a dedicated documentation.

"},{"location":"tests/test-based/#example","title":"Example","text":""},{"location":"tests/test-based/#chainsaw-test","title":"chainsaw-test","text":"

The manifest below contains a Test in a file called chainsaw-test.yaml (or chainsaw-test.yml).

Chainsaw will load the Test and steps defined in its spec section.

The Test uses a custom timeouts for the whole test. Note that timeouts could have been overridden in specific steps if needed.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: test-name\nspec:\n  skip: false\n  concurrent: false\n  skipDelete: false\n  # these timeouts are applied per operation\n  timeouts:\n    apply: 10s\n    assert: 10s\n    error: 10s\n  steps:\n  # first step\n  # apply a configmap to the cluster\n  # the path to the configmap is relative to the folder\n  # containing the test, hence allow reusing manifests\n  # across multiple tests\n  - try:\n    - apply:\n        file: ../resources/configmap.yaml\n  # second step\n  # execute assert statements against existing resources\n  # in the cluster\n  - try:\n    - assert:\n        file: ../resources/configmap-assert.yaml\n  # third step\n  # execute error statements against existing resources\n  # in the cluster\n  - try:\n    - error:\n        file: ../resources/configmap-error.yaml\n
"},{"location":"tests/test-based/#specifying-a-custom-test-file","title":"Specifying a Custom Test File","text":"

If you have your test defined in a different file, you can specify it when running Chainsaw:

chainsaw test . --test-file <custom-test-file-name>.yaml\n

If you don't provide a file extension, chainsaw will search for a file with the .yaml extension first and the .yml extension if not found. If you provide a file extension, chainsaw will only search for a file with the extension you provided.

"},{"location":"tests/test-based/#raw-resource-support","title":"Raw Resource Support","text":"

Chainsaw now includes the raw resource feature, allowing direct specification of Kubernetes resources within the test definitions.

This feature offers a more streamlined approach for defining resources, especially useful for simpler test scenarios or for cases where resource definitions need to be reused or slightly modified across different tests.

"},{"location":"tests/test-based/#example-raw-resource","title":"Example Raw Resource","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: test-name\nspec:\n  skip: false\n  concurrent: false\n  skipDelete: false\n  timeouts:\n    apply: 10s\n    assert: 10s\n    error: 10s\n  steps:\n  # first step applies a configmap directly to the cluster\n  - try:\n    - apply:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: chainsaw-quick-start\n          data:\n            foo: bar\n  # second step executes assert statements against existing resources\n  - try:\n    - assert:\n        file: ../resources/configmap-assert.yaml\n  # third step executes error statements against existing resources\n  - try:\n    - error:\n        file: ../resources/configmap-error.yaml\n
"},{"location":"tests/test-based/#url-support-for-file-references","title":"URL Support for File References","text":"

Chainsaw has support for URLs in file references for assert, apply, error and similar operations.

This feature enhances the reach of Chainsaw by allowing users to reference files hosted on remote locations, such as GitHub raw URLs or other web URLs, directly within their test definitions.

"},{"location":"tests/test-based/#example-url-file-reference","title":"Example URL File Reference","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: test-name\nspec:\n  skip: false\n  concurrent: false\n  skipDelete: false\n  timeouts:\n    apply: 10s\n    assert: 10s\n    error: 10s\n  steps:\n  # first step\n  # apply a Kubernetes manifest hosted at a GitHub raw URL\n  - try:\n    - apply:\n        file: https://raw.githubusercontent.com/username/repo/branch/path/to/configmap.yaml\n  # second step\n  # execute assert statements against existing resources\n  # using a file hosted on another web URL\n  - try:\n    - assert:\n        file: https://example.com/path/to/configmap-assert.yaml\n  # third step\n  # execute error statements against existing resources\n  - try:\n    - error:\n        file: https://mywebsite.com/path/to/configmap-error.yaml\n
"}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"gh-action/","title":"GitHub action","text":"

A GitHub action is available to easily install Chainsaw in your workflows.

The GitHub action is available at kyverno/action-install-chainsaw or in the marketplace.

"},{"location":"gh-action/#usage","title":"Usage","text":"

This action currently supports GitHub-provided Linux, macOS and Windows runners (self-hosted runners may not work).

Add the following entry to your Github workflow YAML file:

uses: kyverno/action-install-chainsaw@v0.1.0\nwith:\n  release: v0.1.0 # optional\n

Example using a pinned version:

jobs:\n  example:\n    runs-on: ubuntu-latest\n\n    permissions: {}\n\n    name: Install Chainsaw\n    steps:\n      - name: Install Chainsaw\n        uses: kyverno/action-install-chainsaw@v0.1.0\n        with:\n          release: v0.0.9\n      - name: Check install\n        run: chainsaw version\n

Example using the default version:

jobs:\n  example:\n    runs-on: ubuntu-latest\n\n    permissions: {}\n\n    name: Install Chainsaw\n    steps:\n      - name: Install Chainsaw\n        uses: kyverno/action-install-chainsaw@v0.1.0\n      - name: Check install\n        run: chainsaw version\n

Example using cosign verification:

jobs:\n  example:\n    runs-on: ubuntu-latest\n\n    permissions: {}\n\n    name: Install Chainsaw\n    steps:\n      - name: Install Cosign\n        uses: sigstore/cosign-installer@v3.1.1\n      - name: Install Chainsaw\n        uses: kyverno/action-install-chainsaw@v0.1.0\n        with:\n          verify: true\n      - name: Check install\n        run: chainsaw version\n

If you want to install Chainsaw from its main version by using go install under the hood, you can set release as main. Once you did that, Chainsaw will be installed via go install which means that please ensure that go is installed.

Example of installing Chainsaw via go install:

jobs:\n  example:\n    runs-on: ubuntu-latest\n\n    permissions: {}\n\n    name: Install Chainsaw via go install\n    steps:\n      - name: Install go\n        uses: actions/setup-go@v4\n        with:\n          go-version: '1.21'\n      - name: Install Chainsaw\n        uses: kyverno/action-install-chainsaw@v0.1.0\n        with:\n          release: main\n      - name: Check install\n        run: chainsaw version\n
"},{"location":"gh-action/#optional-inputs","title":"Optional Inputs","text":"

The following optional inputs:

Input Description release chainsaw version to use instead of the default. install-dir directory to place the chainsaw binary into instead of the default ($HOME/.chainsaw). use-sudo set to true if install-dir location requires sudo privs. Defaults to false. verify set to true to enable cosign verification of the downloaded archive."},{"location":"json-schemas/","title":"JSON schemas","text":"

JSON schemas for Chainsaw resources are available to enable validation and autocompletion in your IDE:

"},{"location":"json-schemas/#vs-code","title":"VS code","text":"

In VS code, simply add a comment on top of your YAML resources.

"},{"location":"json-schemas/#test","title":"Test","text":"
# yaml-language-server: $schema=https://raw.githubusercontent.com/kyverno/chainsaw/main/.schemas/json/test-chainsaw-v1alpha1.json\napiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: basic\nspec:\n  steps:\n  - try:\n    - apply:\n        file: configmap.yaml\n    - assert:\n        file: configmap-assert.yaml\n
"},{"location":"json-schemas/#configuration","title":"Configuration","text":"
# yaml-language-server: $schema=https://raw.githubusercontent.com/kyverno/chainsaw/main/.schemas/json/configuration-chainsaw-v1alpha1.json\napiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Configuration\nmetadata:\n  name: congiguration\nspec:\n  fullName: true\n  failFast: true\n  forceTerminationGracePeriod: 5s\n
"},{"location":"json-schemas/#applying-crds","title":"Applying CRDs","text":"

An alternative is to apply the Chainsaw CRDs in a kubernetes cluster and let the Kubernetes extension do the rest.

CRD definitions are provided in our GitHub repository.

The command below will apply the Chainsaw CRDs to the configured cluster:

kubectl apply -f https://raw.githubusercontent.com/kyverno/chainsaw/main/config/crds/chainsaw.kyverno.io_configurations.yaml\nkubectl apply -f https://raw.githubusercontent.com/kyverno/chainsaw/main/config/crds/chainsaw.kyverno.io_tests.yaml\n
"},{"location":"json-schemas/#exporting-schemas","title":"Exporting schemas","text":"

Chainsaw can export JSON schemas locally.

chainsaw export schemas <local path>\n

See Chainsaw export schemas reference for more details.

"},{"location":"apis/chainsaw.v1alpha1/","title":"chainsaw (v1alpha1)","text":"

Package v1alpha1 contains API Schema definitions for the v1alpha1 API group.

"},{"location":"apis/chainsaw.v1alpha1/#resource-types","title":"Resource Types","text":""},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Configuration","title":"Configuration","text":"

Configuration is the resource that contains the configuration used to run tests.

Field Type Required Inline Description apiVersion string chainsaw.kyverno.io/v1alpha1 kind string Configuration metadata meta/v1.ObjectMeta

Standard object's metadata.

spec ConfigurationSpec

Configuration spec.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Test","title":"Test","text":"

Test is the resource that contains a test definition.

Field Type Required Inline Description apiVersion string chainsaw.kyverno.io/v1alpha1 kind string Test metadata meta/v1.ObjectMeta

Standard object's metadata.

spec TestSpec

Test spec.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Apply","title":"Apply","text":"

Appears in:

Apply represents a set of configurations or resources that should be applied during testing.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

outputs []Output

Outputs defines output bindings.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

FileRefOrResource FileRefOrResource

FileRefOrResource provides a reference to the resources to be applied.

template bool

Template determines whether resources should be considered for templating.

dryRun bool

DryRun determines whether the file should be applied in dry run mode.

expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Assert","title":"Assert","text":"

Appears in:

Assert represents a test condition that is expected to hold true during the testing process.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

FileRefOrCheck FileRefOrCheck

FileRefOrAssert provides a reference to the assertion.

template bool

Template determines whether resources should be considered for templating.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Binding","title":"Binding","text":"

Appears in:

Binding represents a key/value set as a binding in an executing test.

Field Type Required Inline Description name string

Name the name of the binding.

value policy/v1alpha1.Any

Value value of the binding.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Catch","title":"Catch","text":"

Appears in:

Catch defines actions to be executed on failure.

Field Type Required Inline Description description string

Description contains a description of the operation.

podLogs PodLogs

PodLogs determines the pod logs collector to execute.

events Events

Events determines the events collector to execute.

describe Describe

Describe determines the resource describe collector to execute.

wait Wait

Wait determines the resource wait collector to execute.

get Get

Get determines the resource get collector to execute.

delete Delete

Delete represents a deletion operation.

command Command

Command defines a command to run.

script Script

Script defines a script to run.

sleep Sleep

Sleep defines zzzz.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Cluster","title":"Cluster","text":"

Appears in:

Cluster defines cluster config and context.

Field Type Required Inline Description kubeconfig string

Kubeconfig is the path to the referenced file.

context string

Context is the name of the context to use.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Command","title":"Command","text":"

Appears in:

Command describes a command to run as a part of a test step.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

outputs []Output

Outputs defines output bindings.

env []Binding

Env defines additional environment variables.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

entrypoint string

Entrypoint is the command entry point to run.

args []string

Args is the command arguments.

skipLogOutput bool

SkipLogOutput removes the output from the command. Useful for sensitive logs or to reduce noise.

check policy/v1alpha1.Any

Check is an assertion tree to validate the operation outcome.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Condition","title":"Condition","text":"

Appears in:

Condition represents parameters for waiting on a specific condition of a resource.

Field Type Required Inline Description name string

Name defines the specific condition to wait for, e.g., \"Available\", \"Ready\".

value string

Value defines the specific condition status to wait for, e.g., \"True\", \"False\".

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-ConfigurationSpec","title":"ConfigurationSpec","text":"

Appears in:

ConfigurationSpec contains the configuration used to run tests.

Field Type Required Inline Description timeouts Timeouts

Global timeouts configuration. Applies to all tests/test steps if not overridden.

skipDelete bool

If set, do not delete the resources after running the tests (implies SkipClusterDelete).

template bool

Template determines whether resources should be considered for templating.

failFast bool

FailFast determines whether the test should stop upon encountering the first failure.

parallel int

The maximum number of tests to run at once.

reportFormat ReportFormatType

ReportFormat determines test report format (JSON reportPath string

ReportPath defines the path.

reportName string

ReportName defines the name of report to create. It defaults to \"chainsaw-report\".

namespace string

Namespace defines the namespace to use for tests. If not specified, every test will execute in a random ephemeral namespace unless the namespace is overridden in a the test spec.

namespaceTemplate policy/v1alpha1.Any

NamespaceTemplate defines a template to create the test namespace.

fullName bool

FullName makes use of the full test case folder path instead of the folder name.

excludeTestRegex string

ExcludeTestRegex is used to exclude tests based on a regular expression.

includeTestRegex string

IncludeTestRegex is used to include tests based on a regular expression.

repeatCount int

RepeatCount indicates how many times the tests should be executed.

testFile string

TestFile is the name of the file containing the test to run. If no extension is provided, chainsaw will try with .yaml first and .yml if needed.

forceTerminationGracePeriod meta/v1.Duration

ForceTerminationGracePeriod forces the termination grace period on pods, statefulsets, daemonsets and deployments.

delayBeforeCleanup meta/v1.Duration

DelayBeforeCleanup adds a delay between the time a test ends and the time cleanup starts.

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

catch []Catch

Catch defines what the tests steps will execute when an error happens. This will be combined with catch handlers defined at the test and step levels.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Create","title":"Create","text":"

Appears in:

Create represents a set of resources that should be created. If a resource already exists in the cluster it will fail.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

outputs []Output

Outputs defines output bindings.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

FileRefOrResource FileRefOrResource

FileRefOrResource provides a reference to the file containing the resources to be created.

template bool

Template determines whether resources should be considered for templating.

dryRun bool

DryRun determines whether the file should be applied in dry run mode.

expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Delete","title":"Delete","text":"

Appears in:

Delete is a reference to an object that should be deleted

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

template bool

Template determines whether resources should be considered for templating.

ref ObjectReference

ObjectReference determines objects to be deleted.

expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Deletion","title":"Deletion","text":"

Appears in:

Deletion represents parameters for waiting on a resource's deletion.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Describe","title":"Describe","text":"

Appears in:

Describe defines how to describe resources.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

ResourceReference ResourceReference

ResourceReference referenced resource type.

ObjectLabelsSelector ObjectLabelsSelector

ObjectLabelsSelector determines the selection process of referenced objects.

showEvents bool

Show Events indicates whether to include related events.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Error","title":"Error","text":"

Appears in:

Error represents an anticipated error condition that may arise during testing. Instead of treating such an error as a test failure, it acknowledges it as expected.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

FileRefOrCheck FileRefOrCheck

FileRefOrAssert provides a reference to the expected error.

template bool

Template determines whether resources should be considered for templating.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Events","title":"Events","text":"

Appears in:

Events defines how to collect events.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

ObjectLabelsSelector ObjectLabelsSelector

ObjectLabelsSelector determines the selection process of referenced objects.

format Format

Format determines the output format (json or yaml).

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Expectation","title":"Expectation","text":"

Appears in:

Expectation represents a check to be applied on the result of an operation with a match filter to determine if the verification should be considered.

Field Type Required Inline Description match policy/v1alpha1.Any

Match defines the matching statement.

check policy/v1alpha1.Any

Check defines the verification statement.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-FileRef","title":"FileRef","text":"

Appears in:

FileRef represents a file reference.

Field Type Required Inline Description file string

File is the path to the referenced file. This can be a direct path to a file or an expression that matches multiple files, such as \"manifest/*.yaml\" for all YAML files within the \"manifest\" directory.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-FileRefOrCheck","title":"FileRefOrCheck","text":"

Appears in:

FileRefOrCheck represents a file reference or resource.

Field Type Required Inline Description FileRef FileRef

FileRef provides a reference to the file containing the resources to be applied.

resource policy/v1alpha1.Any

Check provides a check used in assertions.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-FileRefOrResource","title":"FileRefOrResource","text":"

Appears in:

FileRefOrResource represents a file reference or resource.

Field Type Required Inline Description FileRef FileRef

FileRef provides a reference to the file containing the resources to be applied.

resource meta/v1/unstructured.Unstructured

Resource provides a resource to be applied.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Finally","title":"Finally","text":"

Appears in:

Finally defines actions to be executed at the end of a test.

Field Type Required Inline Description description string

Description contains a description of the operation.

podLogs PodLogs

PodLogs determines the pod logs collector to execute.

events Events

Events determines the events collector to execute.

describe Describe

Describe determines the resource describe collector to execute.

wait Wait

Wait determines the resource wait collector to execute.

get Get

Get determines the resource get collector to execute.

delete Delete

Delete represents a deletion operation.

command Command

Command defines a command to run.

script Script

Script defines a script to run.

sleep Sleep

Sleep defines zzzz.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-For","title":"For","text":"

Appears in:

For specifies the condition to wait for.

Field Type Required Inline Description deletion Deletion

Deletion specifies parameters for waiting on a resource's deletion.

condition Condition

Condition specifies the condition to wait for.

jsonPath JsonPath

JsonPath specifies the json path condition to wait for.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Format","title":"Format","text":"

(Alias of string)

Appears in:

Format determines the output format (json or yaml).

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Get","title":"Get","text":"

Appears in:

Get defines how to get resources.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

ResourceReference ResourceReference

ResourceReference referenced resource type.

ObjectLabelsSelector ObjectLabelsSelector

ObjectLabelsSelector determines the selection process of referenced objects.

format Format

Format determines the output format (json or yaml).

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-JsonPath","title":"JsonPath","text":"

Appears in:

JsonPath represents parameters for waiting on a json path of a resource.

Field Type Required Inline Description path string

Path defines the json path to wait for, e.g. '{.status.phase}'.

value string

Value defines the expected value to wait for, e.g., \"Running\".

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-ObjectLabelsSelector","title":"ObjectLabelsSelector","text":"

Appears in:

ObjectLabelsSelector represents a strategy to select objects. For a single object name and namespace are used to identify the object. For multiple objects use selector.

Field Type Required Inline Description namespace string

Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

name string

Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

selector string

Selector defines labels selector.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-ObjectReference","title":"ObjectReference","text":"

Appears in:

ObjectReference represents one or more objects with a specific apiVersion and kind. For a single object name and namespace are used to identify the object. For multiple objects use labels.

Field Type Required Inline Description ObjectType ObjectType

ObjectType determines the type of referenced objects.

ObjectSelector ObjectSelector

ObjectSelector determines the selection process of referenced objects.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-ObjectSelector","title":"ObjectSelector","text":"

Appears in:

ObjectSelector represents a strategy to select objects. For a single object name and namespace are used to identify the object. For multiple objects use labels.

Field Type Required Inline Description namespace string

Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

name string

Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

labels map[string]string

Label selector to match objects to delete

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-ObjectType","title":"ObjectType","text":"

Appears in:

ObjectType represents a specific apiVersion and kind.

Field Type Required Inline Description apiVersion string

API version of the referent.

kind string

Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Operation","title":"Operation","text":"

Appears in:

Operation defines a single operation, only one action is permitted for a given operation.

Field Type Required Inline Description description string

Description contains a description of the operation.

continueOnError bool

ContinueOnError determines whether a test should continue or not in case the operation was not successful. Even if the test continues executing, it will still be reported as failed.

apply Apply

Apply represents resources that should be applied for this test step. This can include things like configuration settings or any other resources that need to be available during the test.

assert Assert

Assert represents an assertion to be made. It checks whether the conditions specified in the assertion hold true.

command Command

Command defines a command to run.

create Create

Create represents a creation operation.

delete Delete

Delete represents a deletion operation.

error Error

Error represents the expected errors for this test step. If any of these errors occur, the test will consider them as expected; otherwise, they will be treated as test failures.

patch Patch

Patch represents a patch operation.

script Script

Script defines a script to run.

sleep Sleep

Sleep defines zzzz.

update Update

Update represents an update operation.

wait Wait

Wait determines the resource wait collector to execute.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Output","title":"Output","text":"

Appears in:

Output represents an output binding with a match to determine if the binding must be considered or not.

Field Type Required Inline Description Binding Binding

Binding determines the binding to create when the match succeeds.

match policy/v1alpha1.Any

Match defines the matching statement.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Patch","title":"Patch","text":"

Appears in:

Patch represents a set of resources that should be patched. If a resource doesn't exist yet in the cluster it will fail.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

outputs []Output

Outputs defines output bindings.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

FileRefOrResource FileRefOrResource

FileRefOrResource provides a reference to the file containing the resources to be patched.

template bool

Template determines whether resources should be considered for templating.

dryRun bool

DryRun determines whether the file should be applied in dry run mode.

expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-PodLogs","title":"PodLogs","text":"

Appears in:

PodLogs defines how to collect pod logs.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

ObjectLabelsSelector ObjectLabelsSelector

ObjectLabelsSelector determines the selection process of referenced objects.

container string

Container in pod to get logs from else --all-containers is used.

tail int

Tail is the number of last lines to collect from pods. If omitted or zero, then the default is 10 if you use a selector, or -1 (all) if you use a pod name. This matches default behavior of kubectl logs.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-ReportFormatType","title":"ReportFormatType","text":"

(Alias of string)

Appears in:

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-ResourceReference","title":"ResourceReference","text":"

Appears in:

ResourceReference represents a resource (API), it can be represented with a resource or a kind. Optionally an apiVersion can be specified.

Field Type Required Inline Description apiVersion string

API version of the referent.

kind string

Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

resource string

Resource name of the referent.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Script","title":"Script","text":"

Appears in:

Script describes a script to run as a part of a test step.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

outputs []Output

Outputs defines output bindings.

env []Binding

Env defines additional environment variables.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

content string

Content defines a shell script (run with \"sh -c ...\").

skipLogOutput bool

SkipLogOutput removes the output from the command. Useful for sensitive logs or to reduce noise.

check policy/v1alpha1.Any

Check is an assertion tree to validate the operation outcome.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Sleep","title":"Sleep","text":"

Appears in:

Sleep represents a duration while nothing happens.

Field Type Required Inline Description duration meta/v1.Duration

Duration is the delay used for sleeping.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-TestSpec","title":"TestSpec","text":"

Appears in:

TestSpec contains the test spec.

Field Type Required Inline Description description string

Description contains a description of the test.

timeouts Timeouts

Timeouts for the test. Overrides the global timeouts set in the Configuration on a per operation basis.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

skip bool

Skip determines whether the test should skipped.

concurrent bool

Concurrent determines whether the test should run concurrently with other tests.

skipDelete bool

SkipDelete determines whether the resources created by the test should be deleted after the test is executed.

template bool

Template determines whether resources should be considered for templating.

namespace string

Namespace determines whether the test should run in a random ephemeral namespace or not.

namespaceTemplate policy/v1alpha1.Any

NamespaceTemplate defines a template to create the test namespace.

bindings []Binding

Bindings defines additional binding key/values.

steps []TestStep

Steps defining the test.

catch []Catch

Catch defines what the steps will execute when an error happens. This will be combined with catch handlers defined at the step level.

forceTerminationGracePeriod meta/v1.Duration

ForceTerminationGracePeriod forces the termination grace period on pods, statefulsets, daemonsets and deployments.

delayBeforeCleanup meta/v1.Duration

DelayBeforeCleanup adds a delay between the time a test ends and the time cleanup starts.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-TestStep","title":"TestStep","text":"

Appears in:

TestStep contains the test step definition used in a test spec.

Field Type Required Inline Description name string

Name of the step.

TestStepSpec TestStepSpec

TestStepSpec of the step.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-TestStepSpec","title":"TestStepSpec","text":"

Appears in:

TestStepSpec defines the desired state and behavior for each test step.

Field Type Required Inline Description description string

Description contains a description of the test step.

timeouts Timeouts

Timeouts for the test step. Overrides the global timeouts set in the Configuration and the timeouts eventually set in the Test.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

skipDelete bool

SkipDelete determines whether the resources created by the step should be deleted after the test step is executed.

template bool

Template determines whether resources should be considered for templating.

bindings []Binding

Bindings defines additional binding key/values.

try []Operation

Try defines what the step will try to execute.

catch []Catch

Catch defines what the step will execute when an error happens.

finally []Finally

Finally defines what the step will execute after the step is terminated.

cleanup []Finally

Cleanup defines what will be executed after the test is terminated.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Timeouts","title":"Timeouts","text":"

Appears in:

Timeouts contains timeouts per operation.

Field Type Required Inline Description apply meta/v1.Duration

Apply defines the timeout for the apply operation

assert meta/v1.Duration

Assert defines the timeout for the assert operation

cleanup meta/v1.Duration

Cleanup defines the timeout for the cleanup operation

delete meta/v1.Duration

Delete defines the timeout for the delete operation

error meta/v1.Duration

Error defines the timeout for the error operation

exec meta/v1.Duration

Exec defines the timeout for exec operations

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Update","title":"Update","text":"

Appears in:

Update represents a set of resources that should be updated. If a resource does not exist in the cluster it will fail.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Overrides the global timeout set in the Configuration.

bindings []Binding

Bindings defines additional binding key/values.

outputs []Output

Outputs defines output bindings.

cluster string

Cluster defines the target cluster (default cluster will be used if not specified and/or overridden).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

FileRefOrResource FileRefOrResource

FileRefOrResource provides a reference to the file containing the resources to be created.

template bool

Template determines whether resources should be considered for templating.

dryRun bool

DryRun determines whether the file should be applied in dry run mode.

expect []Expectation

Expect defines a list of matched checks to validate the operation outcome.

"},{"location":"apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Wait","title":"Wait","text":"

Appears in:

Wait specifies how to perform wait operations on resources.

Field Type Required Inline Description timeout meta/v1.Duration

Timeout for the operation. Specifies how long to wait for the condition to be met before timing out.

cluster string

Cluster defines the target cluster where the wait operation will be performed (default cluster will be used if not specified).

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

ResourceReference ResourceReference

ResourceReference referenced resource type.

ObjectLabelsSelector ObjectLabelsSelector

ObjectLabelsSelector determines the selection process of referenced objects.

for For

For specifies the condition to wait for.

format Format

Format determines the output format (json or yaml).

"},{"location":"apis/chainsaw.v1alpha2/","title":"chainsaw (v1alpha2)","text":"

Package v1alpha2 contains API Schema definitions for the v1alpha2 API group.

"},{"location":"apis/chainsaw.v1alpha2/#resource-types","title":"Resource Types","text":""},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-Configuration","title":"Configuration","text":"

Configuration is the resource that contains the configuration used to run tests.

Field Type Required Inline Description apiVersion string chainsaw.kyverno.io/v1alpha2 kind string Configuration metadata meta/v1.ObjectMeta

Standard object's metadata.

spec ConfigurationSpec

Configuration spec.

"},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-Cleanup","title":"Cleanup","text":"

Appears in:

Cleanup contains the cleanup configuration.

Field Type Required Inline Description skipDelete bool

If set, do not delete the resources after running a test.

delayBeforeCleanup meta/v1.Duration

DelayBeforeCleanup adds a delay between the time a test ends and the time cleanup starts.

"},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-ConfigurationSpec","title":"ConfigurationSpec","text":"

Appears in:

ConfigurationSpec contains the configuration used to run tests.

Field Type Required Inline Description catch []Catch

Catch defines what the tests steps will execute when an error happens. This will be combined with catch handlers defined at the test and step levels.

cleanup Cleanup

Cleanup contains cleanup configuration.

clusters map[string]Cluster

Clusters holds a registry to clusters to support multi-cluster tests.

discovery Discovery

Discovery contains tests discovery configuration.

execution Execution

Execution contains tests execution configuration.

namespace Namespace

Namespace contains properties for the namespace to use for tests.

report Report

Report contains properties for the report.

templating Templating

Templating contains the templating config.

timeouts Timeouts

Global timeouts configuration. Applies to all tests/test steps if not overridden.

"},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-Discovery","title":"Discovery","text":"

Appears in:

Discovery contains the tests discovery configuration.

Field Type Required Inline Description excludeTestRegex string

ExcludeTestRegex is used to exclude tests based on a regular expression.

includeTestRegex string

IncludeTestRegex is used to include tests based on a regular expression.

testFile string

TestFile is the name of the file containing the test to run. If no extension is provided, chainsaw will try with .yaml first and .yml if needed.

fullName bool

FullName makes use of the full test case folder path instead of the folder name.

"},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-Execution","title":"Execution","text":"

Appears in:

Execution contains the runner configuration.

Field Type Required Inline Description failFast bool

FailFast determines whether the test should stop upon encountering the first failure.

parallel int

The maximum number of tests to run at once.

repeatCount int

RepeatCount indicates how many times the tests should be executed.

forceTerminationGracePeriod meta/v1.Duration

ForceTerminationGracePeriod forces the termination grace period on pods, statefulsets, daemonsets and deployments.

"},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-Namespace","title":"Namespace","text":"

Appears in:

Namespace contains info about the namespace used for testing.

Field Type Required Inline Description name string

Name defines the namespace to use for tests. If not specified, every test will execute in a random ephemeral namespace unless the namespace is overridden in a the test spec.

template policy/v1alpha1.Any

Template defines a template to create the test namespace.

"},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-Report","title":"Report","text":"

Appears in:

Report contains info about the report.

Field Type Required Inline Description format ReportFormatType

ReportFormat determines test report format (JSON path string

ReportPath defines the path.

name string

ReportName defines the name of report to create. It defaults to \"chainsaw-report\".

"},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-ReportFormatType","title":"ReportFormatType","text":"

(Alias of string)

Appears in:

"},{"location":"apis/chainsaw.v1alpha2/#chainsaw-kyverno-io-v1alpha2-Templating","title":"Templating","text":"

Appears in:

Templating contains the templating configuration.

Field Type Required Inline Description enabled bool

Enabled determines whether resources should be considered for templating.

"},{"location":"bindings/","title":"Bindings","text":"

Chainsaw has a concept of bindings which can be seen as an execution context.

Bindings are used in assertion trees and resource templating, as well as when using the --values flag when invoking chainsaw.

While some bindings are built-in and provided by chainsaw, it's possible to define custom bindings at the test, step, or individual operation level. Those bindings can in turn be used to create custom environment variables in command and script operations.

"},{"location":"bindings/#immutability","title":"Immutability","text":"

It's important to note that bindings are immutable, it's not possible to overwrite a binding and two bindings with the same name can exist without overwriting each other.

When a binding value is resolved, chainsaw will walk the binding chain upwards until it finds a binding with the expected name. Effectively, the last one registered in the chain will be used.

"},{"location":"bindings/#templating","title":"Templating","text":"

A binding name supports templating.

The name of a binding can therefore be dynamic and depend on values or other bindings.

"},{"location":"bindings/#usage","title":"Usage","text":"

The example below defines custom bindings at the test level.

Test level bindings

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  # bindings defined at the test level are available to all steps and operations\n  bindings:\n  - name: hello\n    value: hello\n  - name: chainsaw\n    value: chainsaw\n  # `greetings` depends on `$hello` and `$chainsaw` bindings defined above\n  - name: greetings\n    value: (join(' ', [$hello, $chainsaw]))\n  steps:\n  - try:\n    - script:\n        # custom environment variables, defined using custom bindings\n        env:\n        - name: GREETINGS\n          value: ($greetings)\n        content: echo $GREETINGS\n        check:\n          ($error): ~\n          ($stdout): hello chainsaw\n

The example below is similar to the previous one but also defines custom bindings at the step level.

Step level bindings

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  # bindings defined at the test level are available to all steps and operations\n  bindings:\n  - name: hello\n    value: hello\n  - name: chainsaw\n    value: chainsaw\n  steps:\n  - bindings:\n    # `greetings` depends on `$hello` and `$chainsaw` bindings defined at higher levels\n    - name: greetings\n      value: (join(' ', [$hello, $chainsaw]))\n    try:\n    - script:\n        # custom environment variables, defined using custom bindings\n        env:\n        - name: GREETINGS\n          value: ($greetings)\n        content: echo $GREETINGS\n        check:\n          ($error): ~\n          ($stdout): hello chainsaw\n

Finally, custom bindings can also be defined at the operation level.

Operation level bindings

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  # bindings defined at the test level are available to all steps and operations\n  bindings:\n  - name: hello\n    value: hello\n  steps:\n  - bindings:\n    - name: chainsaw\n      value: chainsaw\n    try:\n    - script:\n        bindings:\n        # `greetings` depends on `$hello` and `$chainsaw` bindings defined at the higher levels\n        - name: greetings\n          value: (join(' ', [$hello, $chainsaw]))\n        # custom environment variables, defined using custom bindings\n        env:\n        - name: GREETINGS\n          value: ($greetings)\n        content: echo $GREETINGS\n        check:\n          ($error): ~\n          ($stdout): hello chainsaw\n
"},{"location":"bindings/#outputs","title":"Outputs","text":"

Under certain conditions, bindings can also be used to add computed results to the context.

See Outputs for details.

"},{"location":"bindings/outputs/","title":"Outputs","text":"

Binding outputs can be useful to communicate and reuse computation results between operations.

"},{"location":"bindings/outputs/#supported-operations","title":"Supported operations","text":"

Currently, only script and command operations support outputs.

"},{"location":"bindings/outputs/#lifetime-of-outputs","title":"Lifetime of outputs","text":"

Once an output has been added in the form of a binding, this binding will be available to all following operations in the same step.

Currently, outputs do not cross the step boundaries.

"},{"location":"bindings/outputs/#matching","title":"Matching","text":"

An output supports an optional match field. The match is used to conditionally create a binding.

In the case of applying a file, for example, the file may contain multiple resources. The match can be used to select the resource to use for creating the binding.

"},{"location":"bindings/outputs/#examples","title":"Examples","text":"

The example below defines invokes a kubectl command to get a configmap from the cluster in json format.

The json output is then parsed and added to the $cm binding and the next operation performs an assertion on it by reading the binding instead of querying the cluster.

Output in script

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  # ...\n  - try:\n    - script:\n        content: kubectl get cm quick-start -n $NAMESPACE -o json\n        outputs:\n        - match:\n            (json_parse($stdout)):\n              apiVersion: v1\n              kind: ConfigMap\n          name: cm\n          value: (json_parse($stdout))\n    - assert:\n        resource:\n          ($cm):\n            metadata:\n              (uid != null): true\n  # ...\n

Output in command

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  # ...\n  - try:\n    - command:\n        entrypoint: kubectl\n        args:\n        - get\n        - cm\n        - quick-start\n        - -n\n        - $NAMESPACE\n        - -o\n        - json\n        outputs:\n        - match:\n            (json_parse($stdout)):\n              apiVersion: v1\n              kind: ConfigMap\n          name: cm\n          value: (json_parse($stdout))\n    - assert:\n        resource:\n          ($cm):\n            metadata:\n              (uid != null): true\n  # ...\n
"},{"location":"collectors/","title":"Collectors","text":""},{"location":"collectors/#purpose","title":"Purpose","text":"

The purpose of collectors is to collect certain information about the outcome of a step should it fail (in the case of catch) or at the end of the step (in the case of finally).

The ultimate goal of collectors is to gather information about the failure of a step and therefore help understand what caused it to fail.

A test step can have an arbitrary number of collectors.

"},{"location":"collectors/#available-collectors","title":"Available collectors","text":""},{"location":"collectors/#templating","title":"Templating","text":"

All string fields in collectors support templating.

"},{"location":"collectors/describe/","title":"Describe","text":"

Describing resources present in the cluster can help understand what happened and troubleshoot test failures.

"},{"location":"collectors/describe/#configuration","title":"Configuration","text":"

Reference documentation

Deprecated syntax

You can specify the resource directly instead of using apiVersion and kind.

This is a deprecated syntax though and will be removed in a future version.

"},{"location":"collectors/describe/#clustered-resources","title":"Clustered resources","text":"

When used with a clustered resource, the namespace is ignored and is not added to the corresponding kubectl command.

"},{"location":"collectors/describe/#all-namespaces","title":"All namespaces","text":"

When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.

"},{"location":"collectors/describe/#usage-examples","title":"Usage examples","text":""},{"location":"collectors/describe/#describe-pods","title":"Describe pods","text":"

If a name is specified, Chainsaw will describe resources that have a name starting with the provided name in the test namespace (or in the cluster if it is a clustered-level resource).

Describe pods in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n    # ...\n    finally:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n    # ...\n

If a namespace is specified, Chainsaw will describe resources in the specified namespace.

Describe pods in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        namespace: foo\n    # ...\n    finally:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        namespace: foo\n    # ...\n
"},{"location":"collectors/describe/#label-selector","title":"Label selector","text":"

An optional label selector can be configured to refine the resources to be described.

Describe pods using a label selector in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        selector: app=my-app\n    # ...\n    finally:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        selector: app=my-app\n    # ...\n

If a namespace is specified, Chainsaw will describe resources in the specified namespace.

Describe pods using a label selector in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        selector: app=my-app\n        namespace: foo\n    # ...\n    finally:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        selector: app=my-app\n        namespace: foo\n    # ...\n
"},{"location":"collectors/describe/#show-events","title":"Show events","text":"

The showEvents field can be used to enable or disable showing events when describing resources.

Default

By default, showEventsis true.

Do not show events

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        namespace: foo\n        showEvents: false\n    # ...\n    finally:\n    - describe:\n        apiVersion: v1\n        kind: Pod\n        namespace: foo\n        showEvents: false\n    # ...\n
"},{"location":"collectors/events/","title":"Events","text":"

Collecting namespace events can help understand what happened inside the cluster.

"},{"location":"collectors/events/#configuration","title":"Configuration","text":"

Reference documentation

"},{"location":"collectors/events/#all-namespaces","title":"All namespaces","text":"

When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.

"},{"location":"collectors/events/#usage-examples","title":"Usage examples","text":""},{"location":"collectors/events/#single-event","title":"Single event","text":"

If a name is specified, Chainsaw will retrieve the specified event in the test namespace.

Collect event in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - events:\n        name: my-event\n    # ...\n    finally:\n    - events:\n        name: my-event\n    # ...\n

If a namespace is specified, Chainsaw will retrieve the specified event in the specified namespace.

Collect event in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - events:\n        name: my-event\n        namespace: foo\n    # ...\n    finally:\n    - events:\n        name: my-event\n        namespace: foo\n    # ...\n
"},{"location":"collectors/events/#all-events","title":"All events","text":"

If no name and namespace are specified, Chainsaw will retrieve all events in the test namespace.

Collect all events in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - events: {}\n    # ...\n    finally:\n    - events: {}\n    # ...\n

On the other hand, if a namespace is specified, Chainsaw will retrieve all events in the specified namespace.

Collect all events in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - events:\n        namespace: foo\n    # ...\n    finally:\n    - events:\n        namespace: foo\n    # ...\n
"},{"location":"collectors/events/#label-selector","title":"Label selector","text":"

An optional label selector can be configured to refine the events to be retrieved.

Collect events using a label selector in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - events:\n        selector: app=my-app\n    # ...\n    finally:\n    - events:\n        selector: app=my-app\n    # ...\n

If a namespace is specified, Chainsaw will retrieve events using the specified namespace.

Collect events using a label selector in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - events:\n        selector: app=my-app\n        namespace: foo\n    # ...\n    finally:\n    - events:\n        selector: app=my-app\n        namespace: foo\n    # ...\n
"},{"location":"collectors/events/#format","title":"Format","text":"

An optional format can be specified. Supported formats are json and yaml.

If format is not specified, results will be returned in text format.

Use json format

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - events:\n        format: json\n    # ...\n    finally:\n    - events:\n        format: json\n    # ...\n

Use yaml format

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - events:\n        format: yaml\n    # ...\n    finally:\n    - events:\n        format: yaml\n    # ...\n
"},{"location":"collectors/get/","title":"Get","text":"

The get collector is used to list and print resources in the cluster.

"},{"location":"collectors/get/#configuration","title":"Configuration","text":"

Reference documentation

Deprecated syntax

You can specify the resource directly instead of using apiVersion and kind.

This is a deprecated syntax though and will be removed in a future version.

"},{"location":"collectors/get/#clustered-resources","title":"Clustered resources","text":"

When used with a clustered resource, the namespace is ignored and is not added to the corresponding kubectl command.

"},{"location":"collectors/get/#all-namespaces","title":"All namespaces","text":"

When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.

"},{"location":"collectors/get/#usage-examples","title":"Usage examples","text":""},{"location":"collectors/get/#single-resource","title":"Single resource","text":"

If a name is specified, Chainsaw will retrieve the specified resource in the test namespace.

Get pod in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n    # ...\n    finally:\n    - get:\n        resource: pods\n        name: my-pod\n    # ...\n

If a namespace is specified, Chainsaw will retrieve the specified resource in the specified namespace.

Collect pod in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n        namespace: foo\n    # ...\n    finally:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n        namespace: foo\n    # ...\n
"},{"location":"collectors/get/#all-resources","title":"All resources","text":"

If no name and namespace are specified, Chainsaw will retrieve all resources in the test namespace.

Collect all resources in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - get:\n        apiVersion: v1\n        kind: Pod\n    # ...\n    finally:\n    - get:\n        apiVersion: v1\n        kind: Pod\n    # ...\n

On the other hand, if a namespace is specified, Chainsaw will retrieve all resources in the specified namespace.

Collect all resources in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        namespace: foo\n    # ...\n    finally:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        namespace: foo\n    # ...\n
"},{"location":"collectors/get/#label-selector","title":"Label selector","text":"

An optional label selector can be configured to refine the resources to be retrieved.

Collect resources using a label selector in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        selector: app=my-app\n    # ...\n    finally:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        selector: app=my-app\n    # ...\n

If a namespace is specified, Chainsaw will retrieve resources using the specified namespace.

Collect resources using a label selector in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        selector: app=my-app\n        namespace: foo\n    # ...\n    finally:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        selector: app=my-app\n        namespace: foo\n    # ...\n
"},{"location":"collectors/get/#format","title":"Format","text":"

An optional format can be specified. Supported formats are json and yaml.

If format is not specified, results will be returned in text format.

Use json format

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        format: json\n    # ...\n    finally:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        format: json\n    # ...\n

Use yaml format

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        format: yaml\n    # ...\n    finally:\n    - get:\n        apiVersion: v1\n        kind: Pod\n        format: yaml\n    # ...\n
"},{"location":"collectors/pod-logs/","title":"Pod logs","text":"

Collecting pod logs can help understand what happened inside one or more pods.

"},{"location":"collectors/pod-logs/#configuration","title":"Configuration","text":"

Reference documentation

"},{"location":"collectors/pod-logs/#all-namespaces","title":"All namespaces","text":"

When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.

"},{"location":"collectors/pod-logs/#usage-examples","title":"Usage examples","text":""},{"location":"collectors/pod-logs/#single-pod","title":"Single pod","text":"

If a pod name is specified, Chainsaw will retrieve logs from this specific pod in the test namespace.

Collect pod logs in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - podLogs:\n        name: my-pod\n    # ...\n    finally:\n    - podLogs:\n        name: my-pod\n    # ...\n

If a namespace is specified, Chainsaw will retrieve logs from this specific pod in the specified namespace.

Collect pod logs in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - podLogs:\n        name: my-pod\n        namespace: foo\n    # ...\n    finally:\n    - podLogs:\n        name: my-pod\n        namespace: foo\n    # ...\n
"},{"location":"collectors/pod-logs/#all-pods","title":"All pods","text":"

If no pod name and namespace is specified, Chainsaw will retrieve logs from all pods in the test namespace.

Collect all pod logs in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - podLogs: {}\n    # ...\n    finally:\n    - podLogs: {}\n    # ...\n

On the other hand, if a namespace is specified, Chainsaw will retrieve logs from all pods in the specified namespace.

Collect all pod logs in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - podLogs:\n        namespace: foo\n    # ...\n    finally:\n    - podLogs:\n        namespace: foo\n    # ...\n
"},{"location":"collectors/pod-logs/#label-selector","title":"Label selector","text":"

An optional label selector can be configured to refine the pods to retrieve logs from.

Collect pod logs using a label selector in the test namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - podLogs:\n        selector: app=my-app\n    # ...\n    finally:\n    - podLogs:\n        selector: app=my-app\n    # ...\n

If a namespace is specified, Chainsaw will retrieve pod logs using the specified namespace.

Collect pod logs using a label selector in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - podLogs:\n        selector: app=my-app\n        namespace: foo\n    # ...\n    finally:\n    - podLogs:\n        selector: app=my-app\n        namespace: foo\n    # ...\n
"},{"location":"collectors/pod-logs/#tail","title":"Tail","text":"

The tail field can be used to limit the amount of log lines retrieved when querying pod logs.

Default

By default, tail will be 10 when a label selector is used, and all if a pod name is specified.

Tail example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - podLogs:\n        selector: app=my-app\n        namespace: foo\n        tail: 30\n    # ...\n    finally:\n    - podLogs:\n        selector: app=my-app\n        namespace: foo\n        tail: 30\n    # ...\n
"},{"location":"collectors/pod-logs/#container","title":"Container","text":"

The container field can be used to retrieve logs from a specific container in the pod.

Default

By default logs from all containers will be fetched.

Container example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    catch:\n    - podLogs:\n        selector: app=my-app\n        namespace: foo\n        container: nginx\n    # ...\n    finally:\n    - podLogs:\n        selector: app=my-app\n        namespace: foo\n        container: nginx\n    # ...\n
"},{"location":"commands/chainsaw/","title":"Chainsaw","text":""},{"location":"commands/chainsaw/#chainsaw","title":"chainsaw","text":"

Stronger tool for e2e testing

chainsaw [flags]\n
"},{"location":"commands/chainsaw/#options","title":"Options","text":"
  -h, --help   help for chainsaw\n
"},{"location":"commands/chainsaw/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_assert/","title":"Chainsaw assert","text":""},{"location":"commands/chainsaw_assert/#chainsaw-assert","title":"chainsaw assert","text":"

Evaluate assertion

chainsaw assert [flags] [FILE]\n
"},{"location":"commands/chainsaw_assert/#options","title":"Options","text":"
      --clustered                           Defines if the resource is clustered (only applies when resource is loaded from a file)\n  -f, --file string                         Path to the file to assert or '-' to read from stdin\n  -h, --help                                help for assert\n      --kube-as string                      Username to impersonate for the operation\n      --kube-as-group stringArray           Group to impersonate for the operation, this flag can be repeated to specify multiple groups.\n      --kube-as-uid string                  UID to impersonate for the operation\n      --kube-certificate-authority string   Path to a cert file for the certificate authority\n      --kube-client-certificate string      Path to a client certificate file for TLS\n      --kube-client-key string              Path to a client key file for TLS\n      --kube-cluster string                 The name of the kubeconfig cluster to use\n      --kube-context string                 The name of the kubeconfig context to use\n      --kube-disable-compression            If true, opt-out of response compression for all requests to the server\n      --kube-insecure-skip-tls-verify       If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure\n  -n, --kube-namespace string               If present, the namespace scope for this CLI request\n      --kube-password string                Password for basic authentication to the API server\n      --kube-proxy-url string               If provided, this URL will be used to connect via proxy\n      --kube-request-timeout string         The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default \"0\")\n      --kube-server string                  The address and port of the Kubernetes API server\n      --kube-tls-server-name string         If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.\n      --kube-token string                   Bearer token for authentication to the API server\n      --kube-user string                    The name of the kubeconfig user to use\n      --kube-username string                Username for basic authentication to the API server\n      --namespace string                    Namespace to use (default \"default\")\n      --no-color                            Removes output colors\n  -r, --resource string                     Path to the file containing the resource\n      --timeout duration                    The assert timeout to use (default 30s)\n
"},{"location":"commands/chainsaw_assert/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_build/","title":"Chainsaw build","text":""},{"location":"commands/chainsaw_build/#chainsaw-build","title":"chainsaw build","text":"

Build commands

chainsaw build [flags]\n
"},{"location":"commands/chainsaw_build/#options","title":"Options","text":"
  -h, --help   help for build\n
"},{"location":"commands/chainsaw_build/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_build_docs/","title":"Chainsaw build docs","text":""},{"location":"commands/chainsaw_build_docs/#chainsaw-build-docs","title":"chainsaw build docs","text":"

Build tests documentation

chainsaw build docs [flags]\n
"},{"location":"commands/chainsaw_build_docs/#options","title":"Options","text":"
      --catalog string         Path to the built test catalog file\n  -h, --help                   help for docs\n      --readme-file string     Name of the built docs file (default \"README.md\")\n      --test-dir stringArray   Directories containing test cases to run\n      --test-file string       Name of the test file (default \"chainsaw-test\")\n
"},{"location":"commands/chainsaw_build_docs/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_completion/","title":"Chainsaw completion","text":""},{"location":"commands/chainsaw_completion/#chainsaw-completion","title":"chainsaw completion","text":"

Generate the autocompletion script for the specified shell

"},{"location":"commands/chainsaw_completion/#synopsis","title":"Synopsis","text":"

Generate the autocompletion script for chainsaw for the specified shell. See each sub-command's help for details on how to use the generated script.

"},{"location":"commands/chainsaw_completion/#options","title":"Options","text":"
  -h, --help   help for completion\n
"},{"location":"commands/chainsaw_completion/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_completion_bash/","title":"Chainsaw completion bash","text":""},{"location":"commands/chainsaw_completion_bash/#chainsaw-completion-bash","title":"chainsaw completion bash","text":"

Generate the autocompletion script for bash

"},{"location":"commands/chainsaw_completion_bash/#synopsis","title":"Synopsis","text":"

Generate the autocompletion script for the bash shell.

This script depends on the 'bash-completion' package. If it is not installed already, you can install it via your OS's package manager.

To load completions in your current shell session:

source <(chainsaw completion bash)\n

To load completions for every new session, execute once:

"},{"location":"commands/chainsaw_completion_bash/#linux","title":"Linux:","text":"
chainsaw completion bash > /etc/bash_completion.d/chainsaw\n
"},{"location":"commands/chainsaw_completion_bash/#macos","title":"macOS:","text":"
chainsaw completion bash > $(brew --prefix)/etc/bash_completion.d/chainsaw\n

You will need to start a new shell for this setup to take effect.

chainsaw completion bash\n
"},{"location":"commands/chainsaw_completion_bash/#options","title":"Options","text":"
  -h, --help              help for bash\n      --no-descriptions   disable completion descriptions\n
"},{"location":"commands/chainsaw_completion_bash/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_completion_fish/","title":"Chainsaw completion fish","text":""},{"location":"commands/chainsaw_completion_fish/#chainsaw-completion-fish","title":"chainsaw completion fish","text":"

Generate the autocompletion script for fish

"},{"location":"commands/chainsaw_completion_fish/#synopsis","title":"Synopsis","text":"

Generate the autocompletion script for the fish shell.

To load completions in your current shell session:

chainsaw completion fish | source\n

To load completions for every new session, execute once:

chainsaw completion fish > ~/.config/fish/completions/chainsaw.fish\n

You will need to start a new shell for this setup to take effect.

chainsaw completion fish [flags]\n
"},{"location":"commands/chainsaw_completion_fish/#options","title":"Options","text":"
  -h, --help              help for fish\n      --no-descriptions   disable completion descriptions\n
"},{"location":"commands/chainsaw_completion_fish/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_completion_powershell/","title":"Chainsaw completion powershell","text":""},{"location":"commands/chainsaw_completion_powershell/#chainsaw-completion-powershell","title":"chainsaw completion powershell","text":"

Generate the autocompletion script for powershell

"},{"location":"commands/chainsaw_completion_powershell/#synopsis","title":"Synopsis","text":"

Generate the autocompletion script for powershell.

To load completions in your current shell session:

chainsaw completion powershell | Out-String | Invoke-Expression\n

To load completions for every new session, add the output of the above command to your powershell profile.

chainsaw completion powershell [flags]\n
"},{"location":"commands/chainsaw_completion_powershell/#options","title":"Options","text":"
  -h, --help              help for powershell\n      --no-descriptions   disable completion descriptions\n
"},{"location":"commands/chainsaw_completion_powershell/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_completion_zsh/","title":"Chainsaw completion zsh","text":""},{"location":"commands/chainsaw_completion_zsh/#chainsaw-completion-zsh","title":"chainsaw completion zsh","text":"

Generate the autocompletion script for zsh

"},{"location":"commands/chainsaw_completion_zsh/#synopsis","title":"Synopsis","text":"

Generate the autocompletion script for the zsh shell.

If shell completion is not already enabled in your environment you will need to enable it. You can execute the following once:

echo \"autoload -U compinit; compinit\" >> ~/.zshrc\n

To load completions in your current shell session:

source <(chainsaw completion zsh)\n

To load completions for every new session, execute once:

"},{"location":"commands/chainsaw_completion_zsh/#linux","title":"Linux:","text":"
chainsaw completion zsh > \"${fpath[1]}/_chainsaw\"\n
"},{"location":"commands/chainsaw_completion_zsh/#macos","title":"macOS:","text":"
chainsaw completion zsh > $(brew --prefix)/share/zsh/site-functions/_chainsaw\n

You will need to start a new shell for this setup to take effect.

chainsaw completion zsh [flags]\n
"},{"location":"commands/chainsaw_completion_zsh/#options","title":"Options","text":"
  -h, --help              help for zsh\n      --no-descriptions   disable completion descriptions\n
"},{"location":"commands/chainsaw_completion_zsh/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_create/","title":"Chainsaw create","text":""},{"location":"commands/chainsaw_create/#chainsaw-create","title":"chainsaw create","text":"

Create Chainsaw resources

chainsaw create [flags]\n
"},{"location":"commands/chainsaw_create/#options","title":"Options","text":"
  -h, --help   help for create\n
"},{"location":"commands/chainsaw_create/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_create_test/","title":"Chainsaw create test","text":""},{"location":"commands/chainsaw_create_test/#chainsaw-create-test","title":"chainsaw create test","text":"

Create a Chainsaw test

chainsaw create test [flags]\n
"},{"location":"commands/chainsaw_create_test/#options","title":"Options","text":"
      --description   If set, adds description when applicable (default true)\n      --force         If set, existing test will be deleted if needed\n  -h, --help          help for test\n      --save          If set, created test will be saved\n
"},{"location":"commands/chainsaw_create_test/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_docs/","title":"Chainsaw docs","text":""},{"location":"commands/chainsaw_docs/#chainsaw-docs","title":"chainsaw docs","text":"

Generate reference documentation

chainsaw docs [flags]\n
"},{"location":"commands/chainsaw_docs/#options","title":"Options","text":"
      --autogenTag      Determines if the generated docs should contain a timestamp (default true)\n  -h, --help            help for docs\n  -o, --output string   Output path (default \".\")\n      --website         Website version\n
"},{"location":"commands/chainsaw_docs/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_export/","title":"Chainsaw export","text":""},{"location":"commands/chainsaw_export/#chainsaw-export","title":"chainsaw export","text":"

Export commands

chainsaw export [flags]\n
"},{"location":"commands/chainsaw_export/#options","title":"Options","text":"
  -h, --help   help for export\n
"},{"location":"commands/chainsaw_export/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_export_schemas/","title":"Chainsaw export schemas","text":""},{"location":"commands/chainsaw_export_schemas/#chainsaw-export-schemas","title":"chainsaw export schemas","text":"

Export JSON schemas

chainsaw export schemas [flags]\n
"},{"location":"commands/chainsaw_export_schemas/#options","title":"Options","text":"
  -h, --help   help for schemas\n
"},{"location":"commands/chainsaw_export_schemas/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_lint/","title":"Chainsaw lint","text":""},{"location":"commands/chainsaw_lint/#chainsaw-lint","title":"chainsaw lint","text":"

Lint a file or read from standard input

"},{"location":"commands/chainsaw_lint/#synopsis","title":"Synopsis","text":"

Use chainsaw lint to lint a specific file or read from standard input for either test or configuration.

chainsaw lint [test|configuration] [flags]\n
"},{"location":"commands/chainsaw_lint/#options","title":"Options","text":"
  -f, --file string   Specify the file to lint or '-' for standard input\n  -h, --help          help for lint\n
"},{"location":"commands/chainsaw_lint/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_migrate/","title":"Chainsaw migrate","text":""},{"location":"commands/chainsaw_migrate/#chainsaw-migrate","title":"chainsaw migrate","text":"

Migrate resources to Chainsaw

chainsaw migrate [flags]\n
"},{"location":"commands/chainsaw_migrate/#options","title":"Options","text":"
  -h, --help   help for migrate\n
"},{"location":"commands/chainsaw_migrate/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_migrate_kuttl/","title":"Chainsaw migrate kuttl","text":""},{"location":"commands/chainsaw_migrate_kuttl/#chainsaw-migrate-kuttl","title":"chainsaw migrate kuttl","text":"

Migrate KUTTL resources to Chainsaw

chainsaw migrate kuttl [flags]\n
"},{"location":"commands/chainsaw_migrate_kuttl/#options","title":"Options","text":"
  -h, --help   help for kuttl\n
"},{"location":"commands/chainsaw_migrate_kuttl/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_migrate_kuttl_config/","title":"Chainsaw migrate kuttl config","text":""},{"location":"commands/chainsaw_migrate_kuttl_config/#chainsaw-migrate-kuttl-config","title":"chainsaw migrate kuttl config","text":"

Migrate KUTTL config to Chainsaw

chainsaw migrate kuttl config [flags]\n
"},{"location":"commands/chainsaw_migrate_kuttl_config/#options","title":"Options","text":"
      --cleanup   If set, delete converted files\n  -h, --help      help for config\n      --save      If set, converted files will be saved\n
"},{"location":"commands/chainsaw_migrate_kuttl_config/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_migrate_kuttl_tests/","title":"Chainsaw migrate kuttl tests","text":""},{"location":"commands/chainsaw_migrate_kuttl_tests/#chainsaw-migrate-kuttl-tests","title":"chainsaw migrate kuttl tests","text":"

Migrate KUTTL tests to Chainsaw

chainsaw migrate kuttl tests [flags]\n
"},{"location":"commands/chainsaw_migrate_kuttl_tests/#options","title":"Options","text":"
      --cleanup   If set, delete converted files\n  -h, --help      help for tests\n      --save      If set, converted files will be saved\n
"},{"location":"commands/chainsaw_migrate_kuttl_tests/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_test/","title":"Chainsaw test","text":""},{"location":"commands/chainsaw_test/#chainsaw-test","title":"chainsaw test","text":"

Run tests

chainsaw test [flags]... [test directories]...\n
"},{"location":"commands/chainsaw_test/#options","title":"Options","text":"
      --apply-timeout duration                    The apply timeout to use as default for configuration (default 5s)\n      --assert-timeout duration                   The assert timeout to use as default for configuration (default 30s)\n      --cleanup-delay duration                    Adds a delay between the time a test ends and the time cleanup starts\n      --cleanup-timeout duration                  The cleanup timeout to use as default for configuration (default 30s)\n      --cluster strings                           Register cluster (format <cluster name>=<kubeconfig path>:[context name])\n      --config string                             Chainsaw configuration file\n      --delete-timeout duration                   The delete timeout to use as default for configuration (default 15s)\n      --error-timeout duration                    The error timeout to use as default for configuration (default 30s)\n      --exclude-test-regex string                 Regular expression to exclude tests\n      --exec-timeout duration                     The exec timeout to use as default for configuration (default 5s)\n      --fail-fast                                 Stop the test upon encountering the first failure\n      --force-termination-grace-period duration   If specified, overrides termination grace periods in applicable resources\n      --full-name                                 Use full test case folder path instead of folder name\n  -h, --help                                      help for test\n      --include-test-regex string                 Regular expression to include tests\n      --kube-as string                            Username to impersonate for the operation\n      --kube-as-group stringArray                 Group to impersonate for the operation, this flag can be repeated to specify multiple groups.\n      --kube-as-uid string                        UID to impersonate for the operation\n      --kube-certificate-authority string         Path to a cert file for the certificate authority\n      --kube-client-certificate string            Path to a client certificate file for TLS\n      --kube-client-key string                    Path to a client key file for TLS\n      --kube-cluster string                       The name of the kubeconfig cluster to use\n      --kube-context string                       The name of the kubeconfig context to use\n      --kube-disable-compression                  If true, opt-out of response compression for all requests to the server\n      --kube-insecure-skip-tls-verify             If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure\n  -n, --kube-namespace string                     If present, the namespace scope for this CLI request\n      --kube-password string                      Password for basic authentication to the API server\n      --kube-proxy-url string                     If provided, this URL will be used to connect via proxy\n      --kube-request-timeout string               The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default \"0\")\n      --kube-server string                        The address and port of the Kubernetes API server\n      --kube-tls-server-name string               If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.\n      --kube-token string                         Bearer token for authentication to the API server\n      --kube-user string                          The name of the kubeconfig user to use\n      --kube-username string                      Username for basic authentication to the API server\n      --namespace string                          Namespace to use for tests\n      --no-cluster                                Runs without cluster\n      --no-color                                  Removes output colors\n      --parallel int                              The maximum number of tests to run at once\n      --pause-on-failure                          Pause test execution failure (implies no concurrency)\n      --repeat-count int                          Number of times to repeat each test (default 1)\n      --report-format string                      Test report format (JSON|XML|nil)\n      --report-name string                        The name of the report to create (default \"chainsaw-report\")\n      --report-path string                        The path of the report to create\n      --selector strings                          Selector (label query) to filter on\n      --skip-delete                               If set, do not delete the resources after running the tests\n      --template                                  If set, resources will be considered for templating (default true)\n      --test-dir strings                          Directories containing test cases to run\n      --test-file string                          Name of the test file (default \"chainsaw-test\")\n      --values strings                            Values passed to the tests\n
"},{"location":"commands/chainsaw_test/#see-also","title":"SEE ALSO","text":""},{"location":"commands/chainsaw_version/","title":"Chainsaw version","text":""},{"location":"commands/chainsaw_version/#chainsaw-version","title":"chainsaw version","text":"

Print the version informations

chainsaw version [flags]\n
"},{"location":"commands/chainsaw_version/#options","title":"Options","text":"
  -h, --help   help for version\n
"},{"location":"commands/chainsaw_version/#see-also","title":"SEE ALSO","text":""},{"location":"community/","title":"Community","text":"

Chainsaw has a growing community and we would definitely love to see you join and contribute.

Everyone is welcome to make suggestions, report bugs, open feature requests, contribute code or docs, participate in discussions, write blogs or anything that can benefit the project.

Chainsaw is built and maintained under the Kyverno umbrella but decisions are Community driven Everyone's voice matters

"},{"location":"community/#slack-channel","title":"Slack channel","text":"

Join our slack channel #kyverno-chainsaw to meet with users, contributors and maintainers.

"},{"location":"community/#community-meetings","title":"Community Meetings","text":"

To attend our community meetings, join the Chainsaw group. You will then be sent a meeting invite and will have access to the agenda and meeting notes. Any member may suggest topics for discussion.

This is a public, weekly for Kyverno-Chainsaw maintainers to make announcements and provide project updates, and request input and feedback. This forum allows community members to raise agenda items of any sort, including but not limited to any PRs or issues on which they are working.

Weekly every Thursday at 2:00 PM UTC

"},{"location":"community/#roadmap","title":"RoadMap","text":"

For detailed information on our planned features and upcoming updates, please view our Roadmap.

"},{"location":"community/#contributing","title":"Contributing","text":"

Please read the contributing guide for details around:

  1. Code of Conduct
  2. Code Culture
  3. Details on how to contribute
"},{"location":"community/#adopters","title":"Adopters","text":"

If you are using Chainsaw and want to share it publicly we always appreciate a bit of support. Pull requests to the ADOPTERS LIST will put a smile on our faces

"},{"location":"configuration/","title":"Configuring Chainsaw","text":"

Chainsaw is a comprehensive tool designed to facilitate end-to-end testing in Kubernetes.

This documentation will focus on providing a breakdown of its configuration structure and how to use it.

Chainsaw can be configured in two different and complementary ways:

Precedence

If both are specified, command-line flags will take precedence over configuration coming from a configuration file.

"},{"location":"configuration/#specific-configuration-options","title":"Specific configuration options","text":"

Please pay attention to the configuration options below, they may or may not be relevant in your case but can be useful in certain cases:

"},{"location":"configuration/cleanup-delay/","title":"Delay before cleanup","text":"

At the end of each test, Chainsaw will delete the resources it created during the test.

When testing operators, it can be useful to wait a little bit before starting the cleanup process to make sure the operator/controller has the necessary time to update the internal state.

For this reason, Chainsaw provides the delayBeforeCleanup configuration option and the corresponding --delay-before-cleanup flag.

"},{"location":"configuration/cleanup-delay/#configuration","title":"Configuration","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Configuration\nmetadata:\n  name: custom-config\nspec:\n  # ...\n  delayBeforeCleanup: 5s\n  # ...\n
"},{"location":"configuration/cleanup-delay/#flag","title":"Flag","text":"
chainsaw test --delay-before-cleanup 5s ...\n
"},{"location":"configuration/file/","title":"Configuration file","text":"

Chainsaw is described as a Stronger tool for e2e testing.

With its versatile configuration options, you can customize the testing process to fit your needs.

Chainsaw prioritizes its configuration in the following order:

  1. User-specified configuration

    If you explicitly provide a configuration file using a command-line flag

  2. Default configuration file

    If no configuration is specified, Chainsaw will look for a default file named .chainsaw.yaml in the current working directory

  3. Internal default configuration

    In the absence of both the above, Chainsaw will use a default configuration file embedded in the Chainsaw binary

"},{"location":"configuration/file/#example","title":"Example","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Configuration\nmetadata:\n  name: custom-config\nspec:\n  timeouts:\n    apply: 45s\n    assert: 20s\n    cleanup: 45s\n    delete: 25s\n    error: 10s\n    exec: 45s\n  skipDelete: false\n  failFast: true\n  parallel: 4\n  # ...\n
"},{"location":"configuration/file/#how-to-specify-a-configuration","title":"How to specify a configuration","text":"

To use a custom configuration file:

chainsaw test --config path/to/your/config.yaml\n

Defaults

If you don't specify any configuration, Chainsaw will look for the default configuration file .chainsaw.yaml in the current working directory.

If that file is not found, it will fall back to its internal default configuration.

"},{"location":"configuration/file/#default-configuration","title":"Default configuration","text":"

The default configuration below is used by Chainsaw when no configuration file was provided and the default file .chainsaw.yaml does not exist.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Configuration\nmetadata:\n  name: default\nspec: {}\n
"},{"location":"configuration/file/#reference-documentation","title":"Reference documentation","text":"

See Configuration API reference for more details.

"},{"location":"configuration/flags/","title":"Command line flags","text":"

Even after a configuration file is loaded, you can override specific settings using command-line flags.

"},{"location":"configuration/flags/#example","title":"Example","text":"
chainsaw test                         \\\n  path/to/test/dir                    \\\n  --config path/to/your/config.yaml   \\\n  --assert-timeout 45s                \\\n  --skip-delete false                 \\\n  --fail-fast true                    \\\n  --parallel 4                        \\\n  ...\n

In this example, Chainsaw will load a configuration file but the timeout configuration and other settings will be overridden by the values set in the flags, regardless of the value in the loaded configuration file.

"},{"location":"configuration/flags/#usage","title":"Usage","text":"

The command below will run tests using the configuration from my-config.yaml, taking tests from /path/to/tests, and running a maximum of 10 tests simultaneously.

chainsaw test /path/to/tests --config my-config.yaml --parallel 10\n
"},{"location":"configuration/flags/#reference-documentation","title":"Reference documentation","text":"

See Chainsaw test command reference for more details.

"},{"location":"configuration/grace/","title":"Termination graceful period","text":"

Some Kubernetes resources can take time before being stopped. For example, deleting a Pod can take time if the underlying container doesn't quit quickly enough.

For this reason, Chainsaw provides the forceTerminationGracePeriod configuration option and the corresponding --force-termination-grace-period flag. If set, Chainsaw will override the terminationGracePeriodSeconds when working with the following resource kinds:

"},{"location":"configuration/grace/#configuration","title":"Configuration","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Configuration\nmetadata:\n  name: custom-config\nspec:\n  # ...\n  forceTerminationGracePeriod: 5s\n  # ...\n
"},{"location":"configuration/grace/#flag","title":"Flag","text":"
chainsaw test --force-termination-grace-period 5s ...\n
"},{"location":"configuration/multi-cluster/","title":"Multi cluster","text":"

Chainsaw supports testing against multiple clusters.

To use a specific cluster in a test (or test step) you will need to register the cluster either using the config file or using command line flags.

Since v0.2.1 you can also register clusters dynamically at the test, step and operation levels. This is particularly useful when a cluster is created in a test step and used in subsequent steps.

"},{"location":"configuration/multi-cluster/#configuration","title":"Configuration","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Configuration\nmetadata:\n  name: custom-config\nspec:\n  # ...\n  clusters:\n    cluster-1:\n      kubeconfig: /path/to/kubeconfig-1\n      context: context-name\n    cluster-2:\n      kubeconfig: /path/to/kubeconfig-2\n      context: context-name\n  # ...\n
"},{"location":"configuration/multi-cluster/#flag","title":"Flag","text":"

The --cluster flag can appear multiple times and is expected to come in the following format --cluster cluster-name=/path/to/kubeconfig[:context-name].

chainsaw test                                               \\\n    --cluster cluster-1=/path/to/kubeconfig-1               \\\n    --cluster cluster-2=/path/to/kubeconfig-2:context-name\n
"},{"location":"configuration/multi-cluster/#dynamic-cluster-registration","title":"Dynamic cluster registration","text":"

The test below illustrates dynamic cluster registration:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: dynamic-clusters\nspec:\n  steps:\n  - try:\n    - script:\n        timeout: 1m\n        content: |\n          kind create cluster --name dynamic --kubeconfig ./dynamic\n    finally:\n    - script:\n        content: |\n          kind delete cluster --name dynamic\n    - script:\n        content: |\n          rm -f ./dynamic\n  - clusters:\n      dynamic:\n        kubeconfig: ./dynamic\n    cluster: dynamic\n    try:\n    - apply:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: quick-start\n            namespace: default\n          data:\n            foo: bar\n    - assert:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: quick-start\n            namespace: default\n          data:\n            foo: bar\n

In the test above, a cluster is created in the first step of the test then registered and used in the second step.

"},{"location":"configuration/multi-cluster/#usage","title":"Usage","text":"

Once registered, a cluster can be specified at the test, step or operation level. If specified at multiple levels, the most granular one is selected, effectively overriding the cluster specified at higher levels.

Specifying cluster at the test level

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  # all steps and operations will be executed against the\n  # cluster specified below (unless overridden)\n  cluster: cluster-1\n  steps:\n  - try:\n    # ...\n    - assert:\n        resource:\n          apiVersion: v1\n          kind: Deployment\n          metadata:\n            name: foo\n          spec:\n            (replicas > 3): true\n    # ...\n

Specifying cluster at the step level

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - # all operations will be executed against the\n    # cluster specified below (unless overridden)\n    cluster: cluster-1\n    try:\n    # ...\n    - assert:\n        resource:\n          apiVersion: v1\n          kind: Deployment\n          metadata:\n            name: foo\n          spec:\n            (replicas > 3): true\n    # ...\n

Specifying cluster at the operation level

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - assert:\n        # operation will be executed against the\n        # cluster specified below\n        cluster: cluster-1\n        resource:\n          apiVersion: v1\n          kind: Deployment\n          metadata:\n            name: foo\n          spec:\n            (replicas > 3): true\n    # ...\n
"},{"location":"configuration/multi-cluster/#client-binding","title":"$client binding","text":"

When a cluster is specified (whatever the test, step or operation level), the $client binding visible in operation expressions is always the Kubernetes client corresponding to the configured cluster.

In the example below, $client binding is a client to the configured cluster-1.

$client binding

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - assert:\n        # operation will be executed against the\n        # cluster specified below\n        cluster: cluster-1\n        resource:\n          (x_k8s_list($client, 'v1', 'Node')):\n            (length(items)): 1\n    # ...\n
"},{"location":"configuration/multi-cluster/#kubeconfig-in-scripts-and-commands","title":"KUBECONFIG in scripts and commands","text":"

When a cluster is specified (whatever the test, step or operation level), a temporary KUBECONFIG is automatically created and the KUBECONFIG is set to this temporary file for every command and script operation.

This way command and script operations are executed in the context of the configured cluster.

Temporary KUBECONFIG

The default KUBECONFIG you usually observe in your local shell will be different in script and command operations.

Temporary KUBECONFIG

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - script:\n        # operation will be executed against the\n        # cluster specified below\n        cluster: cluster-1\n        # the kubectl command below will use a \n        # KUBECONFIG environment variable pointing\n        # to a temporary kubeconfig file configured\n        # for the specified cluster\n        content: kubectl get pods\n    # ...\n
"},{"location":"configuration/no-cluster/","title":"Running without a cluster","text":"

Chainsaw can be run without connection to a Kubernetes cluster. In this case, chainsaw will not try to create an ephemeral namespace and all operations requiring a Kubernetes cluster will fail.

To run chainsaw in this mode pass the --no-cluster flag.

"},{"location":"configuration/no-cluster/#example","title":"Example","text":"
# run chainsaw without connection to a Kubernetes cluster\nchainsaw test --no-cluster\n
"},{"location":"configuration/reports/","title":"Reports","text":"

Chainsaw can generate JUnit reports in XML or JSON format.

To produce a test report, configure the report format, report path and report name in the configuration or using CLI flags.

"},{"location":"configuration/reports/#configuration","title":"Configuration","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Configuration\nmetadata:\n  name: custom-config\nspec:\n  # ...\n  reportFormat: JSON\n  reportName: chainsaw-report\n  reportPath: /home/chainsaw\n  # ...\n
"},{"location":"configuration/reports/#flag","title":"Flag","text":"
chainsaw test --report-format JSON --report-name chainsaw-report --report-path /path/to/save/report ...\n

Note: The reportPath can be specified as either a relative or an absolute path.

"},{"location":"configuration/selector/","title":"Label selectors","text":"

Chainsaw can filter the tests to run using label selectors.

You can pass label selectors using the --selector flag when invoking the chainsaw test command.

"},{"location":"configuration/selector/#example","title":"Example","text":"

Given the test below:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: basic\n  labels:\n    foo: bar\nspec:\n  # ...\n

Invoking Chainsaw with the command below will take the test above into account:

chainsaw test --selector foo=bar\n
"},{"location":"configuration/templating/","title":"Resource templating","text":"

Experimental status

This is an experimental feature, and implementation could change slightly in the next versions.

Info

Templating was disabled by default in v0.1.* but is now enabled by default since v0.2.1.

Chainsaw can apply transformations to the resources before they are processed by the operation.

This is useful when a resource needs some runtime configuration.

Templating must be enabled at the configuration, test, step, or operation level for the templating process to kick in. Alternatively, templating can be enabled using the --template flag when invoking chainsaw from the command line.

Note

Unlike assertion trees, templating can only be specified in leave nodes of the YAML tree.

"},{"location":"configuration/templating/#example","title":"Example","text":"

The test below will create, assert, and delete a ConfigMap with a dynamic name configured at runtime using the $namespace binding.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: values\nspec:\n  steps:\n  - try:\n    - apply:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: ($namespace)\n    - assert:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: ($namespace)\n    - delete:\n        ref:\n          apiVersion: v1\n          kind: ConfigMap\n          name: ($namespace)\n
"},{"location":"configuration/timeouts/","title":"Timeouts","text":"

Timeouts in Chainsaw are specified per type of operation. This is required because the timeout varies greatly depending on the nature of an operation.

For example, applying a manifest in a cluster is expected to be reasonably fast, while validating a resource can be a long operation.

Chainsaw supports separately configuring the timeouts below:

Overriding timeouts

Each timeout can be overridden at the test level, test step level, or individual operation level.

Timeouts defined in the Configuration are used in operations when not overridden.

"},{"location":"configuration/timeouts/#configuration","title":"Configuration","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Configuration\nmetadata:\n  name: custom-config\nspec:\n  # ...\n  timeouts:\n    apply: 45s\n    assert: 20s\n    cleanup: 45s\n    delete: 25s\n    error: 10s\n    exec: 45s\n  # ...\n
"},{"location":"configuration/timeouts/#flags","title":"Flags","text":"
chainsaw test                     \\\n    --apply-timeout 45s             \\\n    --assert-timeout 45s            \\\n    --cleanup-timeout 45s           \\\n    --delete-timeout 45s            \\\n    --error-timeout 45s             \\\n    --exec-timeout 45s              \\\n    ...\n
"},{"location":"configuration/values/","title":"Passing data to tests","text":"

Chainsaw can pass arbitrary values when running tests using the --values flag. Values will be available to tests under the $values binding.

This is useful when a test needs to be configured externally.

"},{"location":"configuration/values/#example","title":"Example","text":"

The test below expects the $value.foo to be provided when chainsaw is invoked.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: values\nspec:\n  steps:\n  - try:\n    - assert:\n        resource:\n          ($values.foo): bar\n

Now you can invoke chainsaw like this:

# pass object { \"foo\": \"bar\"\u00a0} as values to the executed tests\n# `--values -` means values are read from standard input\necho \"foo: bar\" | chainsaw test --values -\n\n# read values from a file\nchainsaw test --values ./values.yaml\n\n# pass values using heredoc\nchainsaw test --values - <<EOF\nfoo: bar\nEOF\n
"},{"location":"examples/","title":"Examples","text":""},{"location":"examples/#setup","title":"Setup","text":"

To use Chainsaw you will need a Kubernetes cluster, Chainsaw won't create one for you.

In these examples, we will use kind but feel free to use the tool of your choice.

Not a cluster management tool

We consider this is not the responsibility of Chainsaw to manage clusters.

There are plenty of solutions to create and manage local clusters that will do that better than Chainsaw.

"},{"location":"examples/#create-a-kind-cluster","title":"Create a kind cluster","text":"

Please refer to the kind install docs to install it locally.

Once kind creating a local cluster is as simple as running:

# define kind image\nexport KIND_IMAGE=\"kindest/node:v1.28.0\"\n\n# create cluster\nkind create cluster --image $KIND_IMAGE\n
"},{"location":"examples/#install-chainsaw","title":"Install Chainsaw","text":"

Refer to install docs to install Chainsaw.

"},{"location":"examples/array-assertions/","title":"Array assertions","text":"

This example demonstrates how to perform complex assertions on arrays.

"},{"location":"examples/array-assertions/#setup","title":"Setup","text":"

See Setup docs

"},{"location":"examples/array-assertions/#manifests","title":"Manifests","text":""},{"location":"examples/array-assertions/#resourcesyaml","title":"resources.yaml","text":"
apiVersion: v1\nkind: Pod\nmetadata:\n  name: example\nspec:\n  containers:\n  - name: container-1\n    image: nginx-1\n    env:\n    - name: ENV_1\n      value: value-1\n  - name: container-2\n    image: nginx-2\n    env:\n    - name: ENV_2\n      value: value-2\n  - name: container-3\n    image: nginx-3\n    env:\n    - name: ENV_3\n      value: value-3\n
"},{"location":"examples/array-assertions/#assertionsyaml","title":"assertions.yaml","text":"
apiVersion: v1\nkind: Pod\nmetadata:\n  name: example\nspec:\n  # iterate over all containers having `name: container-1`\n  ~.(containers[?name == 'container-1']):\n    image: nginx-1\n  # iterate over all containers, bind `$index` to the element index\n  ~index.(containers):\n    image: (join('-', ['nginx', to_string($index + `1`)]))\n  # nested iteration\n  ~index2.(containers):\n    ~.(env):\n      name: (join('_', ['ENV', to_string($index2 + `1`)]))\n      value: (join('-', ['value', to_string($index2 + `1`)]))\n
"},{"location":"examples/array-assertions/#test","title":"Test","text":""},{"location":"examples/array-assertions/#chainsaw-testyaml","title":"chainsaw-test.yaml","text":"
# yaml-language-server: $schema=https://raw.githubusercontent.com/kyverno/chainsaw/main/.schemas/json/test-chainsaw-v1alpha1.json\napiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: array-assertions\nspec:\n  steps:\n  - try:\n    - apply:\n        file: resources.yaml\n    - assert:\n        file: assertions.yaml\n
"},{"location":"examples/array-assertions/#execute","title":"Execute","text":"
chainsaw test\n
"},{"location":"jp/functions/","title":"Functions","text":"

Experimental functions

Experimental functions are denoted by the x_ prefix.

These are functions that are subject to signature change in a future version.

"},{"location":"jp/functions/#built-in-functions","title":"built-in functions","text":"Name Signature abs abs(number) avg avg(array[number]) ceil ceil(number) contains contains(array|string, any) ends_with ends_with(string, string) find_first find_first(string, string, number, number) find_last find_last(string, string, number, number) floor floor(number) from_items from_items(array[array]) group_by group_by(array, expref) items items(object) join join(string, array[string]) keys keys(object) length length(string|array|object) lower lower(string) map map(expref, array) max max(array[number]|array[string]) max_by max_by(array, expref) merge merge(object) min min(array[number]|array[string]) min_by min_by(array, expref) not_null not_null(any) pad_left pad_left(string, number, string) pad_right pad_right(string, number, string) replace replace(string, string, string, number) reverse reverse(array|string) sort sort(array[string]|array[number]) sort_by sort_by(array, expref) split split(string, string, number) starts_with starts_with(string, string) sum sum(array[number]) to_array to_array(any) to_number to_number(any) to_string to_string(any) trim trim(string, string) trim_left trim_left(string, string) trim_right trim_right(string, string) type type(any) upper upper(string) values values(object) zip zip(array, array)"},{"location":"jp/functions/#kyverno-json-functions","title":"kyverno-json functions","text":"Name Signature at at(array, any) concat concat(string, string) json_parse json_parse(string) wildcard wildcard(string, string)"},{"location":"jp/functions/#kyverno-functions","title":"kyverno functions","text":"Name Signature compare compare(string, string) equal_fold equal_fold(string, string) replace replace(string, string, string, number) replace_all replace_all(string, string, string) to_upper to_upper(string) to_lower to_lower(string) trim trim(string, string) trim_prefix trim_prefix(string, string) split split(string, string) regex_replace_all regex_replace_all(string, string|number, string|number) regex_replace_all_literal regex_replace_all_literal(string, string|number, string|number) regex_match regex_match(string, string|number) pattern_match pattern_match(string, string|number) label_match label_match(object, object) to_boolean to_boolean(string) add add(any, any) sum sum(array) subtract subtract(any, any) multiply multiply(any, any) divide divide(any, any) modulo modulo(any, any) round round(number, number) base64_decode base64_decode(string) base64_encode base64_encode(string) time_since time_since(string, string, string) time_now time_now() time_now_utc time_now_utc() path_canonicalize path_canonicalize(string) truncate truncate(string, number) semver_compare semver_compare(string, string) parse_json parse_json(string) parse_yaml parse_yaml(string) lookup lookup(object|array, string|number) items items(object|array, string, string) object_from_lists object_from_lists(array, array) random random(string) x509_decode x509_decode(string) time_to_cron time_to_cron(string) time_add time_add(string, string) time_parse time_parse(string, string) time_utc time_utc(string) time_diff time_diff(string, string) time_before time_before(string, string) time_after time_after(string, string) time_between time_between(string, string, string) time_truncate time_truncate(string, string)"},{"location":"jp/functions/#chainsaw-functions","title":"chainsaw functions","text":"Name Signature env env(string) x_k8s_get x_k8s_get(any, string, string, string, string) x_k8s_list x_k8s_list(any, string, string, string) x_k8s_exists x_k8s_exists(any, string, string, string, string) x_k8s_resource_exists x_k8s_resource_exists(any, string, string) x_k8s_server_version x_k8s_server_version(any)"},{"location":"more/events/","title":"Working with events","text":"

Kubernetes events are regular Kubernetes objects and can be asserted on just like any other object:

apiVersion: v1\nkind: Event\nreason: Started\nsource:\n  component: kubelet\ninvolvedObject:\n  apiVersion: v1\n  kind: Pod\n  name: my-pod\n
"},{"location":"more/kuttl-migration/","title":"Migration from KUTTL","text":""},{"location":"more/kuttl-migration/#overview","title":"Overview","text":"

The chainsaw migrate kuttl tests and chainsaw migrate kuttl config commands are designed for the migration of KUTTL tests to Chainsaw.

Reference documentation

You can view the full command documentation here.

"},{"location":"more/kuttl-migration/#examples","title":"Examples","text":""},{"location":"more/kuttl-migration/#migrate-tests","title":"Migrate tests","text":"

The command below will migrate KUTTL tests to Chainsaw and overwrite original files with converted ones.

chainsaw migrate kuttl tests path/to/kuttl/tests --save --cleanup\n

This will generate a chainsaw-test.yaml for every KUTTL test discovered.

"},{"location":"more/kuttl-migration/#migrate-configuration","title":"Migrate configuration","text":"

The command below will migrate a KUTTL test suite file to the corresponding Chainsaw Configuration.

chainsaw migrate kuttl config path/to/kuttl/testsuite --save --cleanup\n

This will generate a .chainsaw.yaml configuration file.

"},{"location":"more/lint/","title":"Lint tests","text":""},{"location":"more/lint/#overview","title":"Overview","text":"

Chainsaw comes with a lint command to detect ill-formated tests.

Reference documentation

You can view the full command documentation here.

"},{"location":"more/lint/#usage","title":"Usage","text":"

To build the docs of a test, Chainsaw provides the chainsaw lint test -f path/to/chainsaw-test.yaml command.

chainsaw lint test -f - <<EOF\napiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: assertion-tree\nspec:\n  steps:\n  - try:\n    - assert:\n        file: assert.yaml\nEOF\n
Processing input...\nThe document is valid\n
"},{"location":"more/test-docs/","title":"Building test docs","text":""},{"location":"more/test-docs/#overview","title":"Overview","text":"

Chainsaw makes it simple to build the documentation of your tests.

As test suites grow, it becomes important to document what a test does and how it is supposed to work.

Going through the implementation of a test to understand its purpose is not an efficient strategy.

Reference documentation

You can view the full command documentation here.

"},{"location":"more/test-docs/#usage","title":"Usage","text":"

To build the docs of a test, Chainsaw provides the chainsaw build docs command.

chainsaw build docs --test-dir path/to/chainsaw/tests\n

This will automatically discover tests and document steps and operations in try, catch and finally statements.

"},{"location":"more/test-docs/#the-description-field","title":"The description field","text":"

Additionally, you can set the description field in:

Chainsaw will output them nicely in the built docs.

"},{"location":"more/test-docs/#example","title":"Example","text":"

See below for an example test and the corresponding built docs.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: basic\nspec:\n  description: This is a very simple test that creates a configmap and checks the content is as expected.\n  steps:\n  - description: This steps applies the configmap in the cluster and checks the configmap content.\n    try:\n    - description: Create the configmap.\n      apply:\n        file: configmap.yaml\n    - description: Check the configmap content.\n      assert:\n        file: configmap-assert.yaml\n
"},{"location":"more/test-docs/#test-basic","title":"Test: basic","text":"

This is a very simple test that creates a configmap and checks the content is as expected.

"},{"location":"more/test-docs/#steps","title":"Steps","text":"# Name Try Catch Finally 1 step-1 2 0 0"},{"location":"more/test-docs/#step-step-1","title":"Step: step-1","text":"

This step applies the configmap in the cluster and checks the configmap content.

"},{"location":"more/test-docs/#try","title":"Try","text":"# Operation Description 1 apply Create the configmap. 2 assert Check the configmap content."},{"location":"operations/","title":"Operations","text":"

While tests are made of test steps, test steps can be considered made of operations.

Every operation in a test step runs sequentially.

Only one action per operation

Every operation consists of a single action. While it is syntactically possible to create an operation with multiple actions, Chainsaw will verify and reject tests if operations containing multiple actions are found.

The reasoning behind this intentional choice is that it becomes harder to understand in which order actions will be executed in case an operation consists of multiple actions. For this reason, operations consisting of multiple actions are disallowed.

"},{"location":"operations/#common-fields","title":"Common fields","text":"

All operations share some configuration fields.

Reference documentation

The full structure of the Operation is documented here.

"},{"location":"operations/#continueonerror","title":"ContinueOnError","text":"

Determines whether a test step should continue or not in case the operation is not successful.

Even if the test continues executing, it will still be reported as failed.

"},{"location":"operations/#available-operations","title":"Available operations","text":""},{"location":"operations/#non-resource-assertions","title":"Non-resource assertions","text":"

It is possible to evaluate assertions that do not depend on resources.

See Non-resource assertions for details.

"},{"location":"operations/#operation-checks","title":"Operation checks","text":"

Some operations support checking the operation execution result against specific expectations.

See Operation checks for use case details and supported operations.

"},{"location":"operations/#resource-templating","title":"Resource templating","text":"

Chainsaw can apply transformations to the resources before they are processed by the operation.

See Resource templating for use case details and supported operations.

"},{"location":"operations/apply/","title":"Apply","text":"

The apply operation lets you define resources that should be applied to the Kubernetes cluster during the test step.

These can be configurations, deployments, services, or any other Kubernetes resource.

"},{"location":"operations/apply/#configuration","title":"Configuration","text":"

Reference documentation

"},{"location":"operations/apply/#usage-examples","title":"Usage examples","text":"

Below is an example of using apply in a Test resource.

Using a specific file

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - apply:\n        file: my-configmap.yaml\n    # ...\n

Using file path expressions

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example-multi\nspec:\n  steps:\n  - try:\n    # ...\n    - apply:\n        file: \"configs/*.yaml\"\n    # ...\n

Using an URL

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - apply:\n        file: https://raw.githubusercontent.com/kyverno/chainsaw/main/testdata/step/configmap.yaml\n    # ...\n

Using an inline resource

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - apply:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: chainsaw-quick-start\n          data:\n            foo: bar\n    # ...\n
"},{"location":"operations/apply/#operation-check","title":"Operation check","text":"

Below is an example of using an operation check.

With check

# ...\n- apply:\n    file: my-configmap.yaml\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: ConfigMap\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n

With check

# ...\n- apply:\n    resource:\n      apiVersion: v1\n      kind: ConfigMap\n      metadata:\n        name: chainsaw-quick-start\n      data:\n        foo: bar\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: ConfigMap\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n
"},{"location":"operations/assert/","title":"Assert","text":"

The assert operation allows you to specify conditions that should hold true for a successful test.

For example, after applying resources, you might want to ensure that a particular pod is running or a service is accessible.

Assertion trees

Assertions in Chainsaw are based on assertion trees.

Assertion trees are a solution to declaratively represent complex conditions like partial array comparisons or complex operations against an incoming data structure.

Assertion trees are compatible with standard assertions that exist in tools like KUTTL but can do a lot more. Please see the assertion trees documentation in kyverno-json for details.

"},{"location":"operations/assert/#configuration","title":"Configuration","text":"

Reference documentation

"},{"location":"operations/assert/#usage-examples","title":"Usage examples","text":"

Below is an example of using assert in a Test resource.

Using a specific file for assertions

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - assert:\n        file: ../resources/deployment-assert.yaml\n    # ...\n

Using file path expressions for assertions

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example-multi\nspec:\n  steps:\n  - try:\n    # ...\n    - assert:\n        file: \"../assertions/*.yaml\"\n    # ...\n

Using an URL

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - assert:\n        file: https://raw.githubusercontent.com/kyverno/chainsaw/main/testdata/resource/valid.yaml\n    # ...\n

Using an inline assertion tree

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - assert:\n        resource:\n          apiVersion: v1\n          kind: Deployment\n          metadata:\n            name: foo\n          spec:\n            (replicas > 3): true\n    # ...\n
"},{"location":"operations/check/","title":"Operation checks","text":"

Considering an operation's success or failure is not always as simple as checking an error code.

To support those kinds of use cases, some operations support an additional check field to evaluate the operation result against an assertion tree.

Assertion trees

Assertions in Chainsaw are based on assertion trees.

Assertion trees are a solution to declaratively represent complex conditions like partial array comparisons or complex operations against an incoming data structure.

Assertion trees are compatible with standard assertions that exist in tools like KUTTL but can do a lot more. Please see the assertion trees documentation in kyverno-json for details.

Checked model

Different operations have a different model passed through the assertion tree.

The object passed to the assertion tree is the output object of the operation. Additional data like error or standard logs are passed using bindings ($error, $stdout, $stderr)

"},{"location":"operations/check/#expect-vs-check","title":"Expect vs Check","text":"

While a simple check is enough to determine the result of a single operation, we needed a more advanced construct to cover apply and create operations. Those operations can operate on files containing multiple manifests and every manifest can have a different result.

To support more granular checks we use the expect field that contains an array of Expectations. Every expectation is made of an optional match and a check statement.

This way it is possible to control the scope of a check.

Null match

If the match statement is null, the check statement applies to all manifests in the operation.

If no expectation matches a given manifest, the default expectation will be used, checking that no error occurred.

"},{"location":"operations/check/#apply","title":"Apply","text":"

apply supports expect and has the following elements to be checked:

Name Purpose Type $values Values provided when invoking chainsaw with --values flag object $namespace Name of the current test namespace string $client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object $config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object $test.id Current test id int $step.id Current step id int $operation.id Current operation id int $operation.resourceId Current resource id int $error The error message (if any) at the end of the operation string @ The state of the resource (if any) at the end of the operation object"},{"location":"operations/check/#command","title":"Command","text":"

command supports check and has the following elements to be checked:

Name Purpose Type $values Values provided when invoking chainsaw with --values flag object $namespace Name of the current test namespace string $client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object $config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object $test.id Current test id int $step.id Current step id int $operation.id Current operation id int $operation.resourceId Current resource id int $error The error message (if any) at the end of the operation string $stdout The content of the standard console output (if any) at the end of the operation string $stderr The content of the standard console error output (if any) at the end of the operation string @ Always null"},{"location":"operations/check/#create","title":"Create","text":"

create supports expect and has the following elements to be checked:

Name Purpose Type $values Values provided when invoking chainsaw with --values flag object $namespace Name of the current test namespace string $client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object $config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object $test.id Current test id int $step.id Current step id int $operation.id Current operation id int $operation.resourceId Current resource id int $error The error message (if any) at the end of the operation string @ The state of the resource (if any) at the end of the operation object"},{"location":"operations/check/#delete","title":"Delete","text":"

delete supports check and has the following elements to be checked:

Name Purpose Type $values Values provided when invoking chainsaw with --values flag object $namespace Name of the current test namespace string $client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object $config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object $test.id Current test id int $step.id Current step id int $operation.id Current operation id int $operation.resourceId Current resource id int $error The error message (if any) at the end of the operation string @ The state of the resource (if any) at the end of the operation object"},{"location":"operations/check/#patch","title":"Patch","text":"

patch supports expect and has the following elements to be checked:

Name Purpose Type $values Values provided when invoking chainsaw with --values flag object $namespace Name of the current test namespace string $client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object $config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object $test.id Current test id int $step.id Current step id int $operation.id Current operation id int $operation.resourceId Current resource id int $error The error message (if any) at the end of the operation string @ The state of the resource (if any) at the end of the operation object"},{"location":"operations/check/#script","title":"Script","text":"

script supports check and has the following elements to be checked:

Name Purpose Type $values Values provided when invoking chainsaw with --values flag object $namespace Name of the current test namespace string $client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object $config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object $test.id Current test id int $step.id Current step id int $operation.id Current operation id int $operation.resourceId Current resource id int $error The error message (if any) at the end of the operation string $stdout The content of the standard console output (if any) at the end of the operation string $stderr The content of the standard console error output (if any) at the end of the operation string @ Always null"},{"location":"operations/check/#update","title":"Update","text":"

update supports expect and has the following elements to be checked:

Name Purpose Type $values Values provided when invoking chainsaw with --values flag object $namespace Name of the current test namespace string $client Kubernetes client chainsaw is connected to (if not running with --no-cluster) object $config Kubernetes client config chainsaw is connected to (if not running with --no-cluster) object $test.id Current test id int $step.id Current step id int $operation.id Current operation id int $operation.resourceId Current resource id int $error The error message (if any) at the end of the operation string @ The state of the resource (if any) at the end of the operation object"},{"location":"operations/command/","title":"Command","text":"

The command operation provides a means to execute a specific command during the test step.

"},{"location":"operations/command/#configuration","title":"Configuration","text":"

Reference documentation

"},{"location":"operations/command/#usage-examples","title":"Usage examples","text":"

Below is an example of using command in a Test resource.

Example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - command:\n        entrypoint: echo\n        args:\n        - hello chainsaw\n    # ...\n

When defining shell command args in YAML format, it's crucial to consider potential differences in behavior, as Chainsaw may interpret them differently compared to regular shell or bash environments, due to quote removal.

"},{"location":"operations/command/#operation-check","title":"Operation check","text":"

Below is an example of using an operation check.

With check

# ...\n- command:\n    entrypoint: echo\n    args:\n    - hello chainsaw\n    check:\n      # an error is expected, this will:\n      # - succeed if the operation failed\n      # - fail if the operation succeeded\n      ($error != null): true\n# ...\n
"},{"location":"operations/create/","title":"Create","text":"

The create operation lets you define resources that should be created in the Kubernetes cluster during the test step. These can be configurations, deployments, services, or any other Kubernetes resource.

"},{"location":"operations/create/#configuration","title":"Configuration","text":"

Reference documentation

Warning

If the resource to be created already exists in the cluster, the step will fail.

"},{"location":"operations/create/#usage-examples","title":"Usage examples","text":"

Below is an example of using create in a Test resource.

Using a specific file

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - create:\n        file: my-configmap.yaml\n    # ...\n

Using file path expressions

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example-multi\nspec:\n  steps:\n  - try:\n    # ...\n    - create:\n        file: \"configs/*.yaml\"\n    # ...\n

Using an URL

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - create:\n        file: https://raw.githubusercontent.com/kyverno/chainsaw/main/testdata/resource/valid.yaml\n    # ...\n

Using an inline resource

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - create:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: chainsaw-quick-start\n          data:\n            foo: bar\n    # ...\n
"},{"location":"operations/create/#operation-check","title":"Operation check","text":"

Below is an example of using an operation check.

With check

# ...\n- create:\n    file: my-configmap.yaml\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: ConfigMap\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n

With check

# ...\n- create:\n    resource:\n      apiVersion: v1\n      kind: ConfigMap\n      metadata:\n        name: chainsaw-quick-start\n      data:\n        foo: bar\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: ConfigMap\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n
"},{"location":"operations/delete/","title":"Delete","text":"

The delete operation allows you to specify resources that should be deleted from the Kubernetes cluster before a particular test step is executed.

Info

The propagation policy is forced to Background because some types default to Orphan (this is the case for unmanaged jobs for example) and we don't want to let dangling pods run in the cluster after cleanup.

"},{"location":"operations/delete/#configuration","title":"Configuration","text":"

Reference documentation

"},{"location":"operations/delete/#usage-examples","title":"Usage examples","text":"

Below is an example of using delete in a Test resource.

Example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - delete:\n        ref:\n          apiVersion: v1\n          kind: Pod\n          namespace: default\n          name: my-test-pod\n    # ...\n
"},{"location":"operations/delete/#operation-check","title":"Operation check","text":"

Below is an example of using an operation check.

With check

# ...\n- delete:\n    ref:\n      apiVersion: v1\n      kind: Pod\n      namespace: default\n      name: my-test-pod\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: Pod\n        metadata:\n          namespace: default\n          name: my-test-pod\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n
"},{"location":"operations/error/","title":"Error","text":"

The error operation lets you define a set of expected errors for a test step. If any of these errors occur during the test, they are treated as expected outcomes. However, if an error that's not on this list occurs, it will be treated as a test failure.

Assertion trees

Errors in Chainsaw are based on assertion trees.

Assertion trees is a solution to declaratively represent complex conditions like partial array comparisons or complex operations against an incoming data structure.

Assertion trees are compatible with standard assertions that exist in tools like KUTTL but can do a lot more. Please see the assertion trees documentation in kyverno-json for details.

"},{"location":"operations/error/#configuration","title":"Configuration","text":"

Reference documentation

"},{"location":"operations/error/#usage-examples","title":"Usage examples","text":"

Below is an example of using error in a Test resource.

Expecting an error from a specific file

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - error:\n        file: ../resources/deployment-error.yaml\n    # ...\n

Using file path expressions to expect errors from multiple files

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example-multi\nspec:\n  steps:\n  - try:\n    # ...\n    - error:\n        file: \"../errors/*.yaml\"\n    # ...\n

Using an URL

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - error:\n        file: https://raw.githubusercontent.com/user/repo/branch/path/to/deployment-error.yaml\n    # ...\n

Using an inline assertion tree

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - error:\n        resource:\n          apiVersion: v1\n          kind: Deployment\n          metadata:\n            name: foo\n          spec:\n            (replicas > 3): true\n    # ...\n
"},{"location":"operations/non-resource-assert/","title":"Non-resource assertions","text":"

Under certain circumstances, it makes sense to evaluate assertions that do not depend on resources. For example, when asserting the number of nodes in a cluster is equal to a known value.

"},{"location":"operations/non-resource-assert/#usage-examples","title":"Usage examples","text":"

Below is an example of using assert in a Test resource.

Using a file

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: non-resource-assertion\nspec:\n  steps:\n  - try:\n    - assert:\n        resource:\n          (x_k8s_list($client, 'v1', 'Node')):\n            (length(items)): 1\n    - error:\n        resource:\n          (x_k8s_list($client, 'v1', 'Node')):\n            (length(items)): 2\n
"},{"location":"operations/patch/","title":"Patch","text":"

The patch operation lets you define resources that should be modified in the Kubernetes cluster during the test step. These can be configurations, deployments, services, or any other Kubernetes resource.

"},{"location":"operations/patch/#configuration","title":"Configuration","text":"

Reference documentation

Warning

If the resource to be modified does not exist in the cluster, the step will fail.

"},{"location":"operations/patch/#usage-examples","title":"Usage examples","text":"

Below is an example of using patch in a Test resource.

Using a specific file

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - patch:\n        file: my-configmap.yaml\n    # ...\n

Using file path expressions

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example-multi\nspec:\n  steps:\n  - try:\n    # ...\n    - patch:\n        file: \"configs/*.yaml\"\n    # ...\n

Using an URL

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - patch:\n        file: https://raw.githubusercontent.com/kyverno/chainsaw/main/testdata/resource/valid.yaml\n    # ...\n

Using an inline resource

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - patch:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: chainsaw-quick-start\n          data:\n            foo: bar\n    # ...\n
"},{"location":"operations/patch/#operation-check","title":"Operation check","text":"

Below is an example of using an operation check.

With check

# ...\n- patch:\n    file: my-configmap.yaml\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: ConfigMap\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n

With check

# ...\n- patch:\n    resource:\n      apiVersion: v1\n      kind: ConfigMap\n      metadata:\n        name: chainsaw-quick-start\n      data:\n        foo: bar\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: ConfigMap\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n
"},{"location":"operations/script/","title":"Script","text":"

The script operation provides a means to run a script during the test step.

"},{"location":"operations/script/#configuration","title":"Configuration","text":"

Reference documentation

"},{"location":"operations/script/#usage-examples","title":"Usage examples","text":"

Below is an example of using script in a Test resource.

Example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - script:\n        content: |\n          echo \"hello chainsaw\"\n    # ...\n
"},{"location":"operations/script/#operation-check","title":"Operation check","text":"

Below is an example of using an operation check.

With check

# ...\n- script:\n    content: |\n      echo \"hello chainsaw\"\n    check:\n      # an error is expected, this will:\n      # - succeed if the operation failed\n      # - fail if the operation succeeded\n      ($error != null): true\n# ...\n
"},{"location":"operations/sleep/","title":"Sleep","text":"

The sleep operation provides a means to sleep for a configured duration.

"},{"location":"operations/sleep/#configuration","title":"Configuration","text":"

Reference documentation

The full structure of the Sleep is documented here.

"},{"location":"operations/sleep/#usage-examples","title":"Usage examples","text":"

Below is an example of using sleep in a Test resource.

Example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - sleep:\n        duration: 30s\n    # ...\n
"},{"location":"operations/templating/","title":"Resource templating","text":"

Experimental status

This is an experimental feature, and implementation could change slightly in the next versions.

Info

Templating was disabled by default in v0.1.* but is now enabled by default since v0.2.1.

Chainsaw can apply transformations to the resources before they are processed by the operation.

This is useful when a resource needs some runtime configuration.

Templating must be enabled at the configuration, test, step, or operation level for the templating process to kick in. Alternatively, templating can be enabled using the --template flag when invoking chainsaw from the command line.

Note

Unlike assertion trees, templating can only be specified in leave nodes of the YAML tree.

"},{"location":"operations/templating/#supported-operations","title":"Supported operations","text":"

Resource templating is supported in the following operations:

"},{"location":"operations/templating/#assert-and-error","title":"Assert and Error","text":"

When templating assert or error operations, the content is already an assertion tree.

For this reason, only the elements used for looking up the resources to be processed by the operation will be considered for templating. That is, only apiVersion, kind, name, namespace and labels are considered for templating. Other fields are not, they are part of the assertion tree only.

assert and error example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: template\nspec:\n  template: true\n  steps:\n  - assert:\n      resource:\n        # apiVersion, kind, name, namespace and labels are considered for templating\n        apiVersion: v1\n        kind: ConfigMap\n        metadata:\n          name: ($namespace)\n        # other fields are not (they are part of the assertion tree)\n        data:\n          foo: ($namespace)\n  - error:\n      resource:\n        # apiVersion, kind, name, namespace and labels are considered for templating\n        apiVersion: v1\n        kind: ConfigMap\n        metadata:\n          name: ($namespace)\n        # other fields are not (they are part of the assertion tree)\n        data:\n          bar: ($namespace)\n
"},{"location":"operations/templating/#apply-create-and-patch","title":"Apply, Create and Patch","text":"

When templating apply, create or patch operations, the whole content is considered for templating.

apply and create example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: template\nspec:\n  template: true\n  steps:\n  - apply:\n      resource:\n        # the whole content is considered for templating\n        apiVersion: v1\n        kind: ConfigMap\n        metadata:\n          name: ($namespace)\n        data:\n          foo: ($namespace)\n  - create:\n      resource:\n        # the whole content is considered for templating\n        apiVersion: v1\n        kind: ConfigMap\n        metadata:\n          name: ($namespace)\n        data:\n          foo: ($namespace)\n  - patch:\n      resource:\n        # the whole content is considered for templating\n        apiVersion: v1\n        kind: ConfigMap\n        metadata:\n          name: ($namespace)\n        data:\n          foo: ($namespace)\n
"},{"location":"operations/templating/#delete","title":"Delete","text":"

When templating delete operations, the whole content is considered for templating.

apply and create example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: template\nspec:\n  template: true\n  steps:\n  - delete:\n      ref:\n        # the whole content is considered for templating\n        apiVersion: v1\n        kind: ConfigMap\n        name: ($namespace)\n
"},{"location":"operations/update/","title":"Update","text":"

The update operation lets you define resources that should be updated in the Kubernetes cluster during the test step. These can be configurations, deployments, services, or any other Kubernetes resource.

"},{"location":"operations/update/#configuration","title":"Configuration","text":"

Reference documentation

Warning

If the resource to be updated doesn't exist in the cluster, the step will fail.

"},{"location":"operations/update/#usage-examples","title":"Usage examples","text":"

Below is an example of using update in a Test resource.

Using a specific file

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - update:\n        file: my-configmap.yaml\n    # ...\n

Using file path expressions

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example-multi\nspec:\n  steps:\n  - try:\n    # ...\n    - update:\n        file: \"configs/*.yaml\"\n    # ...\n

Using an URL

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - update:\n        file: https://raw.githubusercontent.com/kyverno/chainsaw/main/testdata/resource/valid.yaml\n    # ...\n

Using an inline resource

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - update:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: chainsaw-quick-start\n          data:\n            foo: bar\n    # ...\n
"},{"location":"operations/update/#operation-check","title":"Operation check","text":"

Below is an example of using an operation check.

With check

# ...\n- update:\n    file: my-configmap.yaml\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: ConfigMap\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n

With check

# ...\n- update:\n    resource:\n      apiVersion: v1\n      kind: ConfigMap\n      metadata:\n        name: chainsaw-quick-start\n      data:\n        foo: bar\n    expect:\n    - match:\n        # this check applies only if the match\n        # statement below evaluates to `true`\n        apiVersion: v1\n        kind: ConfigMap\n      check:\n        # an error is expected, this will:\n        # - succeed if the operation failed\n        # - fail if the operation succeeded\n        ($error != null): true\n# ...\n
"},{"location":"operations/wait/","title":"Wait","text":"

The wait operation is a wrapper around kubectl wait. It allows to wait for deletion or conditions against resources.

"},{"location":"operations/wait/#configuration","title":"Configuration","text":"

Reference documentation

The full structure of the Wait is documented here.

Deprecated syntax

You can specify the resource directly instead of using apiVersion and kind.

This is a deprecated syntax though and will be removed in a future version.

"},{"location":"operations/wait/#clustered-resources","title":"Clustered resources","text":"

When used with a clustered resource, the namespace is ignored and is not added to the corresponding kubectl command.

"},{"location":"operations/wait/#all-resources","title":"All resources","text":"

If you don't specify a name or a selector, the wait operation will consider all resources.

"},{"location":"operations/wait/#all-namespaces","title":"All namespaces","text":"

When used with a namespaced resource, it is possible to consider all namespaces in the cluster by setting namespace: '*'.

"},{"location":"operations/wait/#usage-examples","title":"Usage examples","text":"

Below is an example of using wait in a Test resource.

Wait pod ready

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n        timeout: 1m\n        for:\n          condition:\n            name: Ready\n            value: 'true'\n    # ...\n

Wait pod ready in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n        namespace: my-ns\n        timeout: 1m\n        for:\n          condition:\n            name: Ready\n            value: 'true'\n    # ...\n

Wait pods ready using a label selector

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        selector: app=foo\n        timeout: 1m\n        for:\n          condition:\n            name: Ready\n            value: 'true'\n    # ...\n

Wait pod deleted

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n        timeout: 1m\n        for:\n          deletion: {}\n    # ...\n

Wait pod deleted in a specific namespace

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        name: my-pod\n        namespace: my-ns\n        timeout: 1m\n        for:\n          deletion: {}\n    # ...\n

Wait pods deleted using a label selector

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        selector: app=foo\n        timeout: 1m\n        for:\n          deletion: {}\n    # ...\n
"},{"location":"operations/wait/#format","title":"Format","text":"

An optional format can be specified. Supported formats are json and yaml.

If format is not specified, results will be returned in text format.

Use json format

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        format: json\n        # ...\n    catch:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        format: json\n        # ...\n    finally:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        format: json\n        # ...\n

Use yaml format

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        format: yaml\n        # ...\n    catch:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        format: yaml\n        # ...\n    finally:\n    # ...\n    - wait:\n        apiVersion: v1\n        kind: Pod\n        format: yaml\n        # ...\n
"},{"location":"quick-start/","title":"Getting started","text":"

Chainsaw is a tool primarily developed to run end-to-end tests in Kubernetes clusters.

It is meant to test Kubernetes operators work as expected by running a sequence of steps and asserting various conditions.

"},{"location":"quick-start/#why-we-made-it","title":"Why we made it?","text":"

While developing Kyverno we need to run end-to-end tests to make sure our admission controller works as expected.

A typical Kyverno end-to-end test

Kyverno can validate, mutate and generate resources based on policies installed in a cluster and a typical test is:

  1. Create a policy
  2. Create a resource
  3. Check that Kyverno acted as expected
  4. Cleanup and move to the next test
"},{"location":"quick-start/#how-to-use-it","title":"How to use it?","text":"

Chainsaw is built with CI tools in mind - you only really need to download and execute it in your build script.

However, installing it on your local machine is entirely possible.

"},{"location":"quick-start/assertion-trees/","title":"Use assertions","text":"

Chainsaw allows declaring complex assertions with a simple and no-code approach, allowing assertions based on comparisons beyond simple equality, working with arrays, and other scenarios that could not be achieved before.

Tip

Under the hood, Chainsaw uses kyverno-json assertion trees. Refer to the assertion trees documentation for more details on the supported syntax.

"},{"location":"quick-start/assertion-trees/#basic-assertion","title":"Basic assertion","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - assert:\n        resource:\n          apiVersion: apps/v1\n          kind: Deployment\n          metadata:\n            name: coredns\n            namespace: kube-system\n          spec:\n            replicas: 2\n

When asking Chainsaw to execute the assertion above, it will look for a deployment named coredns in the kube-system namespace and will compare the existing resource with the (partial) resource definition contained in the assertion.

In this specific case, if the field spec.replicas is set to 2 in the existing resource, the assertion will be considered valid. If it is not equal to 2 the assertion will be considered failed.

This is the most basic assertion Chainsaw can evaluate.

"},{"location":"quick-start/assertion-trees/#slightly-less-basic-assertion","title":"Slightly less basic assertion","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - assert:\n        resource:\n          apiVersion: apps/v1\n          kind: Deployment\n          metadata:\n            labels:\n              k8s-app: kube-dns\n            namespace: kube-system\n          spec:\n            replicas: 2\n

This time we are not providing a resource name.

Chainsaw will look up all deployments with the k8s-app: kube-dns label in the kube-system namespace. The assertion will be considered valid if at least one deployment matches the (partial) resource definition contained in the assertion. If none match, the assertion will be considered failed.

Apart from the resource lookup process being a little bit more interesting, this kind of assertion is essentially the same as the previous one. Chainsaw is basically making a decision by comparing an actual and expected resource.

"},{"location":"quick-start/assertion-trees/#beyond-simple-equality","title":"Beyond simple equality","text":"

The assertion below will check that the number of replicas for a deployment is greater than 3 AND less than 6.

Chainsaw doesn't need to know the exact expected number of replicas. The (replicas > 3 && replicas < 6) expression will be evaluated until the result is true or the operation timeout expires (making the assertion fail).

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    # ...\n    - assert:\n        resource:\n          apiVersion: v1\n          kind: Deployment\n          metadata:\n            name: foo\n          spec:\n            (replicas > `3` && replicas < `6`): true\n    # ...\n

Tip

To indicate that a key or value in the YAML document is an expression, simply place the element between parentheses:

"},{"location":"quick-start/assertion-trees/#working-with-arrays","title":"Working with arrays","text":"

Chainsaw query language makes it easy to assert on arrays. You can filter and transform arrays to select what you want to assert.

"},{"location":"quick-start/assertion-trees/#filtering","title":"Filtering","text":"

In the example below we are creating a resource, then we assert that a condition with type == 'Ready' exists and has a field matching status: 'True':

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        resource:\n          apiVersion: tempo.grafana.com/v1alpha1\n          kind: TempoStack\n          metadata:\n            name: simplest\n          spec:\n            storage:\n              secret:\n                name: minio\n                type: s3\n            # ...\n    - assert:\n        resource:\n          apiVersion: tempo.grafana.com/v1alpha1\n          kind: TempoStack\n          metadata:\n            name: simplest\n          status:\n            # filter conditions array to keep elements where `type == 'Ready'`\n            # and assert there's a single element matching the filter\n            # and that this element status is `True`\n            (conditions[?type == 'Ready']):\n            - status: 'True'\n
"},{"location":"quick-start/assertion-trees/#iterating","title":"Iterating","text":"

Being able to filter arrays allows selecting the elements to be processed.

On top of that, Chainsaw allows iterating over array elements to validate each item separately.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - assert:\n        resource:\n          apiVersion: apps/v1\n          kind: Deployment\n          metadata:\n            labels:\n              k8s-app: kube-dns\n            namespace: kube-system\n          spec:\n            template:\n              spec:\n                # the `~` modifier tells Chainsaw to iterate over the array elements\n                ~.(containers):\n                  securityContext: {}\n

This assertion uses the ~ modifier and Chainsaw will evaluate descendants once per element in the array.

"},{"location":"quick-start/assertion-trees/#comprehensive-reporting","title":"Comprehensive reporting","text":"

Chainsaw offers detailed resource diffs upon assertion failures.

In the example below, the assertion failure message (metadata.annotations.foo: Invalid value: \"null\": Expected value: \"bar\") is augmented with a resource diff.

It provides a clear view of discrepancies between expected and actual resources and gives more context around the specific failure (we can easily identify the owner of the offending pod for example).

| 09:55:50 | deployment | step-1   | ASSERT    | RUN   | v1/Pod @ chainsaw-rare-liger/*\n| 09:56:20 | deployment | step-1   | ASSERT    | ERROR | v1/Pod @ chainsaw-rare-liger/*\n    === ERROR\n    ---------------------------------------------------\n    v1/Pod/chainsaw-rare-liger/example-5477b4ff8c-tnhd9\n    ---------------------------------------------------\n    * metadata.annotations.foo: Invalid value: \"null\": Expected value: \"bar\"\n\n    --- expected\n    +++ actual\n    @@ -1,10 +1,16 @@\n      apiVersion: v1\n      kind: Pod\n      metadata:\n    -  annotations:\n    -    foo: bar\n        labels:\n          app: nginx\n    +  name: example-5477b4ff8c-tnhd9\n        namespace: chainsaw-rare-liger\n    +  ownerReferences:\n    +  - apiVersion: apps/v1\n    +    blockOwnerDeletion: true\n    +    controller: true\n    +    kind: ReplicaSet\n    +    name: example-5477b4ff8c\n    +    uid: 118abe16-ec42-4894-83db-64479c4aac6f\n      spec: {}\n| 09:56:20 | deployment | step-1   | TRY       | DONE  |\n
"},{"location":"quick-start/assertion-trees/#next-step","title":"Next step","text":"

To continue our exploration of the main Chainsaw features, let's look at bindings and resource templating next.

"},{"location":"quick-start/bindings/","title":"Use bindings","text":"

You can think of bindings as a side context where you can store and retrieve data based on keys.

This is particularly useful when some data is only known at runtime. For example, to pass data from one operation to another, to implement resource templating, to fetch data from an external system, etc.

Chainsaw offers some built-in bindings you can directly use in your tests but you can also create your own bindings if needed.

"},{"location":"quick-start/bindings/#inheritance","title":"Inheritance","text":"

Bindings can be configured at the test, step or operation level.

All bindings configured at a given level are automatically inherited in child levels.

JMESPath

Chainsaw uses the JMESPath language, and bindings are implemented using lexical scoping.

"},{"location":"quick-start/bindings/#immutability","title":"Immutability","text":"

Bindings are immutable. This means two bindings can have the same name without overwriting each other.

When a binding is registered it potentially hides other bindings with the same name.

When this binding goes out of scope, previously registered bindings with the same name become visible again.

"},{"location":"quick-start/bindings/#built-in-bindings","title":"Built-in bindings","text":"

The $namespace binding is a good example of a built-in binding provided by Chainsaw. It contains the name of the ephemeral namespace used to execute a test (by default Chainsaw will create an ephemeral namespace for each test).

In the operation below, we are assigning the value of the $namespace binding to an environment variable, and echo it in a script:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - script:\n        env:\n          # assign the value of the `$namespace` binding\n          # to the environment variable `FOO`\n        - name: FOO\n          value: ($namespace)\n        content: echo $FOO\n
"},{"location":"quick-start/bindings/#custom-bindings","title":"Custom bindings","text":"

On top of built-in bindings, you can also create your own ones, combine bindings together, call JMESPath functions using bindings as arguments, etc.

In the test below we create custom bindings at different levels in the test, combine them by calling the join function, assign the result to an environment variable, and echo it in a script:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  # bindings can be declared at the test level\n  bindings:\n  - name: chainsaw\n    value: chainsaw\n  steps:\n    # bindings can also be declared at the step level\n  - bindings:\n    - name: hello\n      value: hello\n    try:\n    - script:\n        # bindings can also be declared at the operation level\n        bindings:\n        - name: awesome\n          value: awesome\n        env:\n          # combined bindings together using the `join` functions and\n          # assign the result to the GREETINGS environment variable\n        - name: GREETINGS\n          value: (join(' ', [$hello, $chainsaw, 'is', $awesome]))\n        content: echo $GREETINGS\n
"},{"location":"quick-start/bindings/#next-step","title":"Next step","text":"

Let's see how bindings can be useful with resource templating.

"},{"location":"quick-start/cleanup/","title":"Control your cleanup","text":"

Unless configured differently, by default Chainsaw will automatically remove the resources it created after a test finishes.

Cleanup happens in reverse order of creation (created last, cleaned up first). This is important, especially when the controller being tested makes use of finalizers.

Overriding cleanup timeout

Note that Chainsaw performs a blocking deletion, that is, it will wait until the resource is not present anymore in the cluster before proceeding with the next resource cleanup.

"},{"location":"quick-start/cleanup/#timeout","title":"Timeout","text":"

A global cleanup timeout can be defined at the configuration level or using command line flags.

It can also be overridden on a per-test or per-step basis but not at the operation level.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  timeouts:\n    # cleanup timeout at the test level\n    cleanup: 30s\n  steps:\n  - timeouts:\n      # cleanup timeout at the step level\n      cleanup: 2m\n    try: ...\n
"},{"location":"quick-start/cleanup/#automatic-cleanup","title":"Automatic cleanup","text":"

After a test, every resource created by Chainsaw will be automatically deleted. This applies to create and apply operations.

In the logs below we can see Chainsaw deletes the previously created resource:

    | 15:21:29 | quick-start | @setup   | CREATE    | OK    | v1/Namespace @ chainsaw-cute-cod\n    | 15:21:29 | quick-start | step-1   | TRY       | RUN   |\n    | 15:21:29 | quick-start | step-1   | APPLY     | RUN   | v1/ConfigMap @ chainsaw-cute-cod/chainsaw-quick-start\n    | 15:21:29 | quick-start | step-1   | CREATE    | OK    | v1/ConfigMap @ chainsaw-cute-cod/chainsaw-quick-start\n    | 15:21:29 | quick-start | step-1   | APPLY     | DONE  | v1/ConfigMap @ chainsaw-cute-cod/chainsaw-quick-start\n    | 15:21:29 | quick-start | step-1   | ASSERT    | RUN   | v1/ConfigMap @ chainsaw-cute-cod/chainsaw-quick-start\n    | 15:21:29 | quick-start | step-1   | ASSERT    | DONE  | v1/ConfigMap @ chainsaw-cute-cod/chainsaw-quick-start\n    | 15:21:29 | quick-start | step-1   | TRY       | DONE  |\n    === step cleanup process start ===\n    | 15:21:29 | quick-start | step-1   | CLEANUP   | RUN   |\n    | 15:21:29 | quick-start | step-1   | DELETE    | RUN   | v1/ConfigMap @ chainsaw-cute-cod/chainsaw-quick-start\n    | 15:21:29 | quick-start | step-1   | DELETE    | OK    | v1/ConfigMap @ chainsaw-cute-cod/chainsaw-quick-start\n    | 15:21:29 | quick-start | step-1   | DELETE    | DONE  | v1/ConfigMap @ chainsaw-cute-cod/chainsaw-quick-start\n    | 15:21:29 | quick-start | step-1   | CLEANUP   | DONE  |\n    === step cleanup process end ===\n    === test cleanup process start ===\n    | 15:21:29 | quick-start | @cleanup | DELETE    | RUN   | v1/Namespace @ chainsaw-cute-cod\n    | 15:21:29 | quick-start | @cleanup | DELETE    | OK    | v1/Namespace @ chainsaw-cute-cod\n    | 15:21:34 | quick-start | @cleanup | DELETE    | DONE  | v1/Namespace @ chainsaw-cute-cod\n    === test cleanup process end ===\n
"},{"location":"quick-start/cleanup/#manual-cleanup","title":"Manual cleanup","text":"

Under certain circumstances, automatic cleanup is not enough and we want to execute custom operations.

Chainsaw allows registering cleanup operations that will be run after automatic cleanup. Custom cleanup operations live at the test step level:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n    # this step will create a local cluster\n  - try:\n    - script:\n        timeout: 1m\n        content: |\n          kind create cluster --name dynamic --kubeconfig ./dynamic\n    # at cleanup time, we want to delete the local cluster we created\n    # and remove the associated kubeconfig\n    cleanup:\n    - script:\n        content: |\n          kind delete cluster --name dynamic\n    - script:\n        content: |\n          rm -f ./dynamic\n
"},{"location":"quick-start/cleanup/#next-step","title":"Next step","text":"

At this point, we covered the main Chainsaw features.

Look at the next steps section to find out what to do next.

"},{"location":"quick-start/completion/","title":"Shell completion","text":"

Once installed, use chainsaw completion command to generate and register the autocompletion script for the specified shell.

Supported shells are:

"},{"location":"quick-start/first-test/","title":"Create a test","text":"

To create a chainsaw test all you need to do is to create one (or more) YAML file(s).

The recommended approach is to create one folder per test, with a chainsaw-test.yaml file containing one (or more) test definition(s). The test definition can reference other files in the same folder or anywhere else on the file system as needed.

Tip

While chainsaw supports other syntaxes, we strongly recommend the explicit approach.

"},{"location":"quick-start/first-test/#what-is-a-test","title":"What is a test?","text":"

To put it simply, a test can be represented as an ordered sequence of test steps.

In turn, a test step can be represented as an ordered sequence of operations.

"},{"location":"quick-start/first-test/#lets-write-our-first-test","title":"Let's write our first test","text":"

For this quick start, we will create a (very simple) Test with one step and two operations:

  1. Create a ConfigMap from a manifest
  2. Verify the ConfigMap was created and contains the expected data

Follow the instructions below to create the folder and files defining our first test.

"},{"location":"quick-start/first-test/#create-a-test-folder","title":"Create a test folder","text":"
# create test folder\nmkdir chainsaw-quick-start\n\n# enter test folder\ncd chainsaw-quick-start\n
"},{"location":"quick-start/first-test/#create-a-configmap-manifest","title":"Create a ConfigMap manifest","text":"
# create a ConfigMap\ncat > configmap.yaml << EOF\napiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: chainsaw-quick-start\ndata:\n  foo: bar\nEOF\n
"},{"location":"quick-start/first-test/#create-a-test-manifest","title":"Create a test manifest","text":"

By default, Chainsaw will look for a file named chainsaw-test.yaml in every folder.

# create test file\ncat > chainsaw-test.yaml << EOF\napiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: quick-start\nspec:\n  steps:\n  - try:\n    # first operation: create the config map\n    - apply:\n        # file is relative to the test folder\n        file: configmap.yaml\n    # second operation: verify the config map exists and contains the expected data\n    - assert:\n        # file is relative to the test folder\n        file: configmap.yaml\nEOF\n
"},{"location":"quick-start/first-test/#next-step","title":"Next step","text":"

Now we have created our first test, you can continue to the next section to execute it.

"},{"location":"quick-start/install/","title":"Installation","text":"

You can install the pre-compiled binary (in several ways), compile from sources, or run with Docker.

We also provide a GitHub action to easily install Chainsaw in your workflows.

"},{"location":"quick-start/install/#install-the-pre-compiled-binary","title":"Install the pre-compiled binary","text":""},{"location":"quick-start/install/#homebrew-tap","title":"Homebrew tap","text":"

add tap:

brew tap kyverno/chainsaw https://github.com/kyverno/chainsaw\n

install chainsaw:

brew install kyverno/chainsaw/chainsaw\n

Don't forget to specify the tap name

Homebrew core already has a tool named chainsaw.

Be sure that you specify the tap name when installing to install the right tool.

"},{"location":"quick-start/install/#manually","title":"Manually","text":"

Download the pre-compiled binaries for your system from the releases page and copy them to the desired location.

"},{"location":"quick-start/install/#install-using-go-install","title":"Install using go install","text":"

You can install with go install with:

go install github.com/kyverno/chainsaw@latest\n
"},{"location":"quick-start/install/#run-with-docker","title":"Run with Docker","text":"

Chainsaw is also available as a Docker image which you can pull and run:

docker pull ghcr.io/kyverno/chainsaw:<version>\n

Warning

Since Chainsaw relies on files for its operation (like test definitions), you will need to bind mount the necessary directories when running it via Docker.

docker run --rm                             \\\n    -v ./testdata/e2e/:/chainsaw/           \\\n    -v ${HOME}/.kube/:/etc/kubeconfig/      \\\n    -e KUBECONFIG=/etc/kubeconfig/config    \\\n    --network=host                          \\\n    ghcr.io/kyverno/chainsaw:<version>      \\\n    test /chainsaw --config /chainsaw/config.yaml\n
"},{"location":"quick-start/install/#compile-from-sources","title":"Compile from sources","text":"

clone:

git clone https://github.com/kyverno/chainsaw.git\n

build the binaries:

cd chainsaw\ngo mod tidy\nmake build\n

verify it works:

./chainsaw version\n
"},{"location":"quick-start/install/#install-using-nix-package","title":"Install using Nix Package","text":"

To install kyverno-chainsaw, refer to the documentation.

"},{"location":"quick-start/install/#on-nixos","title":"On NixOS","text":"
nix-env -iA nixos.kyverno-chainsaw\n
"},{"location":"quick-start/install/#on-non-nixos","title":"On Non-NixOS","text":"
nix-env -iA nixpkgs.kyverno-chainsaw\n

Warning

Using nix-env permanently modifies a local profile of installed packages. This must be updated and maintained by the user in the same way as with a traditional package manager, foregoing many of the benefits that make Nix uniquely powerful. Using nix-shell or a NixOS configuration is recommended instead.

"},{"location":"quick-start/install/#using-nixos-configuration","title":"Using NixOS Configuration","text":"

Add the following Nix code to your NixOS Configuration, usually located in /etc/nixos/configuration.nix :

environment.systemPackages = [\n  pkgs.kyverno-chainsaw\n];\n
"},{"location":"quick-start/install/#using-nix-shell","title":"Using nix-shell","text":"

A nix-shell will temporarily modify your $PATH environment variable. This can be used to try a piece of software before deciding to permanently install it. Use the following command to install kyverno-chainsaw :

nix-shell -p kyverno-chainsaw\n
"},{"location":"quick-start/install/#github-action","title":"GitHub action","text":"

A GitHub action is available to install Chainsaw in your workflows. See the GitHub action dedicated documentation.

"},{"location":"quick-start/next-steps/","title":"Next steps","text":"

We covered the main features of Chainsaw in this Getting started sections.

While this should help you understand Chainsaw better, there are a lot of other things Chainsaw can do for you.

Tip

If there's anything you would like to be improved, please reach out, we will be happy to discuss and improve as much as we can.

To continue exploring the capabilities of Chainsaw:

"},{"location":"quick-start/next-steps/#additional-resources","title":"Additional resources","text":"

Resources, blog posts and videos talking about Chainsaw:

"},{"location":"quick-start/next-steps/#chainsaw-review-video","title":"Chainsaw review video","text":"

If you haven't watched the video below yet, we strongly recommend watching it to discover Chainsaw!

"},{"location":"quick-start/operation-outputs/","title":"Use operation outputs","text":"

Operation outputs can be useful for communicating and reusing computation results across operations.

"},{"location":"quick-start/operation-outputs/#lifetime-of-outputs","title":"Lifetime of outputs","text":"

Once an output has been added to the bindings context, this binding will be available to all following operations in the same step.

Currently, outputs do not cross the step boundaries.

"},{"location":"quick-start/operation-outputs/#matching","title":"Matching","text":"

An output supports an optional match field. The match is used to conditionally create a binding.

In the case of applying a file, for example, the file may contain multiple resources. The match can be used to select the resource to use for creating the binding.

"},{"location":"quick-start/operation-outputs/#load-an-existing-resource","title":"Load an existing resource","text":"

The example below invokes a kubectl command to get a configmap from the cluster in json format.

The json output is then parsed and added to the $cm binding and the next operation performs an assertion on it by reading the binding instead of querying the cluster.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - script:\n        content: kubectl get cm quick-start -n $NAMESPACE -o json\n        outputs:\n          # parse stdout json output and bind the result to `$cm`\n        - name: cm\n          value: (json_parse($stdout))\n    - assert:\n        resource:\n          ($cm):\n            metadata:\n              (uid != null): true\n
"},{"location":"quick-start/operation-outputs/#match-a-resource","title":"Match a resource","text":"

The example below applies resources from a file.

When the resource being applied is a configmap, we bind the resource to an output to print its UID in the next operation.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        file: ./resources.yaml\n        outputs:\n          # match the configmap resource and bind it to `$cm`\n        - match:\n            apiVersion: v1\n            kind: ConfigMap\n          name: cm\n          value: (@)\n    - script:\n        env:\n        - name: UID\n          value: ($cm.metadata.uid)\n        content: echo $UID\n
"},{"location":"quick-start/operation-outputs/#next-step","title":"Next step","text":"

In the next section, we will look at the three main elements of a test step, the try, catch and finally blocks.

"},{"location":"quick-start/resource-templating/","title":"Use resource templating","text":"

Chainsaw simplifies dynamic resource configuration with native resource templating support.

Sometimes things we need to create resources or assertions are only known at runtime.

In the past, users have created all sorts of hacks using tools like envsubst for dynamic substitution of env-variables. Those workarounds usually lack flexibility and introduce new problems like hiding the real resources from Chainsaw, preventing it from cleaning resources properly.

Tip

Resource templating is heavily based on bindings and uses JMESPath language.

"},{"location":"quick-start/resource-templating/#leverage-bindings","title":"Leverage bindings","text":"

In the template below, we are using the $namespace binding at two different places, effectively injecting the ephemeral namespace name in the name and the data.foo fields:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - assert:\n      resource:\n        apiVersion: v1\n        kind: ConfigMap\n        metadata:\n          name: ($namespace)\n        data:\n          foo: ($namespace)\n
"},{"location":"quick-start/resource-templating/#leverage-jmespath","title":"Leverage JMESPath","text":"

In the template below, we are using the JMESPath join function to create a unique resource name:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - apply:\n      resource:\n        apiVersion: v1\n        kind: ConfigMap\n        metadata:\n          name: (join('-', [$namespace, 'cm']))\n        data:\n          foo: bar\n
"},{"location":"quick-start/resource-templating/#next-step","title":"Next step","text":"

Combining bindings and templates with operation outputs allows even more flexibility to pass arbitrary data from one operation to another.

"},{"location":"quick-start/run-tests/","title":"Run tests","text":"

After installing chainsaw and writing tests, the next natural step is to run Chainsaw to execute the tests.

"},{"location":"quick-start/run-tests/#create-a-local-cluster","title":"Create a local cluster","text":"

To use Chainsaw you will need a Kubernetes cluster, Chainsaw won't create one for you.

Not a cluster management tool

We consider this is not the responsibility of Chainsaw to manage clusters. There are plenty of solutions to create and manage local clusters that will do that better than Chainsaw.

The command below will create a local cluster using kind. Use the tool of your choice or directly jump to the next section if you already have a KUBECONFIG configured and pointing to a valid cluster.

# create cluster\nkind create cluster --image \"kindest/node:v1.29.4\"\n
"},{"location":"quick-start/run-tests/#run-chainsaw","title":"Run Chainsaw","text":"

Now you can run the chainsaw test command.

> chainsaw test\n\nVersion: (devel)\nLoading default configuration...\n- Using test file: chainsaw-test.yaml\n- TestDirs [.]\n- SkipDelete false\n- FailFast false\n- ReportFormat ''\n- ReportName 'chainsaw-report'\n- Namespace ''\n- FullName false\n- IncludeTestRegex ''\n- ExcludeTestRegex ''\n- ApplyTimeout 5s\n- AssertTimeout 30s\n- CleanupTimeout 30s\n- DeleteTimeout 15s\n- ErrorTimeout 30s\n- ExecTimeout 5s\nLoading tests...\n- quick-start (.)\nRunning tests...\n=== RUN   chainsaw\n=== PAUSE chainsaw\n=== CONT  chainsaw\n=== RUN   chainsaw/quick-start\n=== PAUSE chainsaw/quick-start\n=== CONT  chainsaw/quick-start\n    | 10:44:26 | quick-start | @setup   | CREATE    | OK    | v1/Namespace @ chainsaw-immense-jay\n    | 10:44:26 | quick-start | step-1   | TRY       | RUN   |\n    | 10:44:26 | quick-start | step-1   | APPLY     | RUN   | v1/ConfigMap @ chainsaw-immense-jay/chainsaw-quick-start\n    | 10:44:26 | quick-start | step-1   | CREATE    | OK    | v1/ConfigMap @ chainsaw-immense-jay/chainsaw-quick-start\n    | 10:44:26 | quick-start | step-1   | APPLY     | DONE  | v1/ConfigMap @ chainsaw-immense-jay/chainsaw-quick-start\n    | 10:44:26 | quick-start | step-1   | ASSERT    | RUN   | v1/ConfigMap @ chainsaw-immense-jay/chainsaw-quick-start\n    | 10:44:26 | quick-start | step-1   | ASSERT    | DONE  | v1/ConfigMap @ chainsaw-immense-jay/chainsaw-quick-start\n    | 10:44:26 | quick-start | step-1   | TRY       | DONE  |\n    | 10:44:26 | quick-start | @cleanup | DELETE    | RUN   | v1/ConfigMap @ chainsaw-immense-jay/chainsaw-quick-start\n    | 10:44:26 | quick-start | @cleanup | DELETE    | OK    | v1/ConfigMap @ chainsaw-immense-jay/chainsaw-quick-start\n    | 10:44:26 | quick-start | @cleanup | DELETE    | DONE  | v1/ConfigMap @ chainsaw-immense-jay/chainsaw-quick-start\n    | 10:44:26 | quick-start | @cleanup | DELETE    | RUN   | v1/Namespace @ chainsaw-immense-jay\n    | 10:44:26 | quick-start | @cleanup | DELETE    | OK    | v1/Namespace @ chainsaw-immense-jay\n    | 10:44:31 | quick-start | @cleanup | DELETE    | DONE  | v1/Namespace @ chainsaw-immense-jay\n--- PASS: chainsaw (0.00s)\n    --- PASS: chainsaw/quick-start (5.25s)\nPASS\nTests Summary...\n- Passed  tests 1\n- Failed  tests 0\n- Skipped tests 0\nDone.\n

Tip

Chainsaw expects a path to the test folder and will discover tests by analyzing files recursively. When no path is provided Chainsaw will use the current path by default (.).

"},{"location":"quick-start/run-tests/#next-step","title":"Next step","text":"

The test above demonstrates the most basic usage of Chainsaw. In the next sections, we will look at the main features that make Chainsaw a very unique tool.

"},{"location":"quick-start/timeouts/","title":"Control your timeouts","text":"

Timeouts in Chainsaw are specified per type of operation. This is handy because the timeout varies greatly depending on the nature of an operation.

For example, applying a manifest in a cluster is expected to be reasonably fast, while validating a resource can be a long operation.

"},{"location":"quick-start/timeouts/#inheritance","title":"Inheritance","text":"

Timeouts can be configured globally and at the test, step or individual operation level.

All timeouts configured at a given level are automatically inherited in child levels. When looking up a timeout, the most specific one takes precedence over the others.

Info

To learn more about timeouts and how to configure global values, see the timeouts configuration page.

"},{"location":"quick-start/timeouts/#at-the-test-level","title":"At the test level","text":"

When a timeout is configured at the test level it will apply to all operations and steps in the test, unless overridden at a more specific level.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  # timeouts configured at the test level will apply to all operations and steps\n  # unless overriden at the step level and/or individual operation level\n  timeouts:\n    apply: 5s\n    assert: 1m\n    # ...\n  steps:\n  - try:\n    - apply:\n        resource:\n          apiVersion: tempo.grafana.com/v1alpha1\n          kind: TempoStack\n          metadata:\n            name: simplest\n          spec:\n            storage:\n              secret:\n                name: minio\n                type: s3\n            # ...\n    - assert:\n        resource:\n          apiVersion: tempo.grafana.com/v1alpha1\n          kind: TempoStack\n          metadata:\n            name: simplest\n          status:\n            (conditions[?type == 'Ready']):\n            - status: 'True'\n
"},{"location":"quick-start/timeouts/#at-the-step-level","title":"At the step level","text":"

When a timeout is configured at the step level it will apply to all operations in the step, unless overridden at a more specific level.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n    # timeouts configured at the step level will apply to all operations\n    # in the step unless overriden at the individual operation level\n  - timeouts:\n      apply: 5s\n      # ...\n    try:\n    - apply:\n        resource:\n          apiVersion: tempo.grafana.com/v1alpha1\n          kind: TempoStack\n          metadata:\n            name: simplest\n          spec:\n            storage:\n              secret:\n                name: minio\n                type: s3\n            # ...\n    # timeouts configured at the step level will apply to all operations\n    # in the step unless overriden at the individual operation level\n  - timeouts:\n      assert: 1m\n      # ...\n    try:\n    - assert:\n        resource:\n          apiVersion: tempo.grafana.com/v1alpha1\n          kind: TempoStack\n          metadata:\n            name: simplest\n          status:\n            (conditions[?type == 'Ready']):\n            - status: 'True'\n
"},{"location":"quick-start/timeouts/#at-the-operation-level","title":"At the operation level","text":"

When a timeout is configured at the operation level, it takes precedence over all timeouts configured at upper levels.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        # timeout configured at the operation level takes precedence\n        # over timeouts configured at upper levels\n        timeout: 5s\n        resource:\n          apiVersion: tempo.grafana.com/v1alpha1\n          kind: TempoStack\n          metadata:\n            name: simplest\n          spec:\n            storage:\n              secret:\n                name: minio\n                type: s3\n            # ...\n    - assert:\n        # timeout configured at the operation level takes precedence\n        # over timeouts configured at upper levels\n        timeout: 1m\n        resource:\n          apiVersion: tempo.grafana.com/v1alpha1\n          kind: TempoStack\n          metadata:\n            name: simplest\n          status:\n            (conditions[?type == 'Ready']):\n            - status: 'True'\n
"},{"location":"quick-start/timeouts/#next-step","title":"Next step","text":"

In the next section, we will see how Chainsaw manages cleanup.

"},{"location":"quick-start/try-catch/","title":"Use try, catch and finally","text":"

A test step is made of 3 main blocks used to determine the actions Chainsaw will perform when executing the step, depending on the test outcome.

Operations defined in the try block are executed first, then:

Note

Note that all operations coming from the catch or finally blocks are executed. If one operation fails, Chainsaw will mark the test as failed and continue executing with the next operation.

"},{"location":"quick-start/try-catch/#cleanup","title":"Cleanup","text":"

At the end of a test, Chainsaw automatically cleans up the resources created during the test (cleanup is done in the opposite order of creation).

All operations from the catch and finally blocks are executed before the cleanup process kicks in. This order allows analyzing the resources that potentially caused the step failure before they are deleted.

"},{"location":"quick-start/try-catch/#catch","title":"Catch","text":"

Operations in a catch block are executed only when the step is considered failed.

This is particularly useful to collect additional information to help understand what caused the failure.

In the example below, the test contains a catch block to collect events in the cluster when an operation fails in the step.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        # ...\n    - assert:\n        # ...\n    # collect events in the `catch` block\n    # will be executed only if an operation failed\n    catch:\n    - events: {}\n
"},{"location":"quick-start/try-catch/#finally","title":"Finally","text":"

Operations in a finally block will always execute regardless of the success or failure of the test step.

This is particularly useful to perform manual cleanup.

In the example below we create a local cluster in a script operation. The cluster deletion script is added to the finally block, guaranteeing the cluster will be deleted regardless of the test outcome.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n    # create a local cluster\n  - try:\n    - script:\n        timeout: 1m\n        content: |\n          kind create cluster --name dynamic --kubeconfig ./dynamic\n    - apply:\n        # ...\n    - assert:\n        # ...\n    # add cluster deletion script in the `finally` block\n    # to guarantee the cluster will be deleted after the test\n    finally:\n    - script:\n        content: |\n          kind delete cluster --name dynamic\n    - script:\n        content: |\n          rm -f ./dynamic\n
"},{"location":"quick-start/try-catch/#next-step","title":"Next step","text":"

Every operation in a test must be executed in a timely fashion. In the next section, we will see how you can control your timeouts.

"},{"location":"quick-start/advanced/","title":"Advanced use cases","text":"

This section is under construction and should be ready soon. Please check again in a few days.

If you are interested in contributing see the tracking issue.

"},{"location":"quick-start/advanced/concurrency/","title":"Concurrency control","text":"

By default, Chainsaw will run tests in parallel.

The number of concurrent tests can be configured globally using a configuration file or with the --parallel flag.

Alternatively, the concurrent nature of a test can specified at the test level:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  # concurrency can be specified per test (`true` or `false`)\n  # default value is `true`\n  concurrent: true\n  # ...\n

All non-concurrent tests are executed first, followed by the concurrent tests running in parallel.

"},{"location":"quick-start/advanced/crds/","title":"Working with CRDs","text":"

New Custom Resource Definitions are not immediately available for use in the Kubernetes API until the Kubernetes API has acknowledged them.

If a Custom Resource Definition is being defined inside of a test step, be sure to wait for the CustomResourceDefinition object to appear.

The test below applies a CRD and waits for it to become available:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        resource:\n          apiVersion: apiextensions.k8s.io/v1\n          kind: CustomResourceDefinition\n          metadata:\n            name: issues.example.com\n          spec:\n            group: example.com\n            names:\n              kind: Issue\n              listKind: IssueList\n              plural: issues\n              singular: issue\n            scope: Namespaced\n            versions: ...\n    - assert:\n        resource:\n          apiVersion: apiextensions.k8s.io/v1\n          kind: CustomResourceDefinition\n          metadata:\n            name: issues.example.com\n          status:\n            acceptedNames:\n              kind: Issue\n              listKind: IssueList\n              plural: issues\n              singular: issue\n            storedVersions:\n            - v1alpha1\n

The CRD can be used in subsequent steps.

"},{"location":"quick-start/advanced/inline/","title":"Inline resources","text":"

When an operation needs to reference a resource, it can do so using a file path or directly specify the resource inline using the resource field.

The test below is equivalent to our first test:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: quick-start\n          data:\n            foo: bar\n    - assert:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: quick-start\n          data:\n            foo: bar\n
"},{"location":"quick-start/advanced/kube-version/","title":"Check Kubernetes version","text":"

The test below fetches the Kubernetes cluster version using x_k8s_server_version. It then uses the minor version retrieved to adapt an assertion based on the value in the $minorversion binding.

Tip

You can implement a ternary operator in JMESPath using an expression like this:

<condition> && <value-if-true> || <value-if-false>

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  bindings:\n  - name: version\n    value: (x_k8s_server_version($config))\n  - name: minorversion\n    value: (to_number($version.minor))\n  steps:\n  - try:\n    - apply:\n        resource:\n          apiVersion: v1\n          kind: Pod\n          metadata:\n            name: pod01\n          spec:\n            containers:\n            - name: busybox\n              image: busybox:1.35\n    # ...\n    - assert:\n        resource:\n          apiVersion: v1\n          kind: Pod\n          metadata:\n            annotations:\n              # If the minor version of the Kubernetes cluster against which this\n              # is tested is less than 29, the annotation is expected to have the group 'system:masters' in it.\n              # Otherwise, due to a change in kubeadm, the group should be 'kubeadm:cluster-admins'.\n              kyverno.io/created-by: (($minorversion < `29` && '{\"groups\":[\"system:masters\",\"system:authenticated\"],\"username\":\"kubernetes-admin\"}') || '{\"groups\":[\"kubeadm:cluster-admins\",\"system:authenticated\"],\"username\":\"kubernetes-admin\"}')\n            name: pod01\n
"},{"location":"quick-start/advanced/label-selectors/","title":"Use label selectors","text":"

Chainsaw can filter the tests to run using label selectors.

You can pass label selectors using the --selector flag when invoking the chainsaw test command.

Given the test below:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: basic\n  labels:\n    foo: bar\nspec:\n  # ...\n

Invoking Chainsaw with the command below will take the test above into account:

chainsaw test --selector foo=bar\n
"},{"location":"quick-start/advanced/multi-cluster/","title":"Multi-cluster setup","text":"

This section is under construction and should be ready soon. Please check again in a few days.

If you are interested in contributing see the tracking issue.

"},{"location":"quick-start/advanced/negative-testing/","title":"Negative testing","text":"

Negative testing is the process of testing cases that are supposed to fail. That is, a test expects errors to happen and if the expected errors don't occur the test must fail.

Chainsaw supports negative testing by letting you decide what should be considered an error or not.

Tip

By default, Chainsaw will consider an operation failed if there was an error executing it (non-zero exit code in scripts and commands, error returned by the API server when calling into Kubernetes, etc...).

"},{"location":"quick-start/advanced/negative-testing/#script-case","title":"Script case","text":"

The test below expects an error and validates the returned error message:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - script:\n        content: kubectl get foo\n        check:\n          ($error != null): true\n          ($stderr): |-\n            error: the server doesn't have a resource type \"foo\"\n

If for whatever reason, the kubectl get foo doesn't return an error, or the message received in standard error output is not error: the server doesn't have a resource type \"foo\", Chainsaw will consider the operation failed.

If it returns an error and the expected error message, Chainsaw will consider the operation successful.

"},{"location":"quick-start/advanced/negative-testing/#working-with-resources","title":"Working with resources","text":"

The test below tries to apply resources in a cluster but expects the operation to fail:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        file: resources.yaml\n        expect:\n          # check that applying the resource failed\n        - check:\n            ($error != null): true\n

If applying the resource succeeded, Chainsaw will consider the operation failed.

On the other hand, if applying the resource fails, Chainsaw will consider the operation to be successful.

"},{"location":"quick-start/advanced/negative-testing/#resource-matching","title":"Resource matching","text":"

In the previous example, if the resources.yaml contains multiple resources, but only some of them may be expected to fail.

Chainsaw allows matching resources when evaluating checks:

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        file: resources.yaml\n        expect:\n          # the check below only applies if the resource being checked\n          # matches the condition defined in the `match` field\n        - match:\n            apiVersion: v1\n            kind: ConfigMap\n            metadata:\n              name: quick-start\n          check:\n            ($error != null): true\n

Using the match field, we can easily target failures related to specific resources.

"},{"location":"quick-start/advanced/non-resource-assertions/","title":"Non-resource assertions","text":"

Under certain circumstances, it makes sense to evaluate assertions that do not depend on resources. For example, when asserting the number of nodes in a cluster is equal to a known value.

The test below uses the x_k8s_list function to query the list of nodes in the cluster. It uses the results to compare the number of nodes found with a known number (1 in this case).

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - assert:\n        resource:\n          (x_k8s_list($client, 'v1', 'Node')):\n            (length(items)): 1\n
"},{"location":"quick-start/advanced/test-output/","title":"Test command output","text":"

Chainsaw can be used to easily check terminal output from CLIs and other commands. This is useful in that convoluted bash scripts involving chaining together tools like grep can be avoided or at least minimized to only complex use cases. Output to both stdout and stderr can be checked for a given string or precise contents.

"},{"location":"quick-start/advanced/test-output/#checking-output-contains","title":"Checking Output Contains","text":"

One basic use case for content checking is that the output simply contains a given string or piece of content. For example, you might want to run automated tests on a CLI binary you build to ensure that a given command produces output that contains some content you specify somewhere in the output. Let's use the following output from the kubectl version command to show these examples.

kubectl version\n\nClient Version: v1.28.2\nKustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3\nServer Version: v1.27.4+k3s1\n

Below is an example that ensures the string '1.28' is found somewhere in that output. So long as the content is present anywhere, the test will succeed. To perform this check, the contains() JMESPath filter is used.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: test\nspec:\n  steps:\n  - name: Check kubectl\n    try:\n    - script:\n        content: kubectl version\n        check:\n          # This check ensures that the string '1.28' is found\n          # in stdout or else fails\n          (contains($stdout, '1.28')): true\n

Checks for content containing a given value can be negated as well. For example, checking to ensure the output does NOT contain the string '1.25'.

- script:\n    content: kubectl version\n    check:\n      # This check ensures that the string '1.25' is NOT found\n      # in stdout or else fails\n      (contains($stdout, '1.25')): false\n
"},{"location":"quick-start/advanced/test-output/#checking-output-is-exactly","title":"Checking Output Is Exactly","text":"

In addition to checking that CLI/command output contains some contents, you may need to ensure that the contents are exactly as intended. The Chainsaw test below accomplishes this by comparing the entire contents of stdout with those specified in the block scalar. If so much as one character, space, or line break is off, the test will fail. This is useful in that not only can content be checked but the formatting of that content can be ensured it matches a given declaration.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: test\nspec:\n  steps:\n  - name: Check kubectl\n    try:\n    - script:\n        content: kubectl version\n        check:\n          # This check ensures the contents of stdout are exactly as shown.\n          # Any deviations will cause a failure.\n          ($stdout): |-\n            Client Version: v1.28.2\n            Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3\n            Server Version: v1.27.4+k3s1\n
"},{"location":"quick-start/advanced/test-output/#checking-output-in-errors","title":"Checking Output In Errors","text":"

In addition to testing that commands succeed and with output in a given shape, it's equally valuable and necessary to perform negative tests; that tests fail and with contents that are as expected. Similarly, those checks can be for output which has some contents as well as output which appears exactly as desired. For example, you may wish to check that running the kubectl foo command not only fails as expected but that the output shown to users contains a certain word or sentence.

kubectl foo\n\nerror: unknown command \"foo\" for \"kubectl\"\n\nDid you mean this?\n        top\n

Below you can see an example where the command kubectl foo is expected to fail but that the error message returned contains some output, in this case the string 'top'.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: test\nspec:\n  steps:\n  - name: Check bad kubectl command\n    try:\n    - script:\n        content: kubectl foo\n        check:\n          # This checks that the result of the content was an error.\n          ($error != null): true\n          # This check below ensures that the string 'top' is found in stderr or else fails\n          (contains($stderr, 'top')): true\n

Likewise, this failure output can be checked that it is precise. Note that in the example below, due to the use of a tab character in the output of kubectl foo, the value of the ($stderr) field is given as a string to preserve these non-printing characters.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: test\nspec:\n  steps:\n  - name: Check kubectl\n    try:\n    - script:\n        content: kubectl foo\n        check:\n          # This checks that the result of the content was an error.\n          ($error != null): true\n          # This checks that the output is exactly as intended.\n          ($stderr): \"error: unknown command \\\"foo\\\" for \\\"kubectl\\\"\\n\\nDid you mean this?\\n\\ttop\"\n
"},{"location":"reference/","title":"Reference documentation","text":""},{"location":"steps/","title":"What is a test step","text":"

A test step is made of three main components used to determine the actions Chainsaw will perform when executing the step.

  1. The try statement (required)
  2. The catch statement (optional)
  3. The finally statement (optional)

Reference documentation

The full structure of the TestStep is documented here.

"},{"location":"steps/#test-step-lifecycle","title":"Test step lifecycle","text":"

Test step lifecycle

  1. The step starts executing operations in the try statement
  2. If an operation fails in the try statement
    1. If a catch statement is present, all operations and collectors are executed
  3. If a finally statement is present, all operations and collectors are executed
"},{"location":"steps/#example","title":"Example","text":"

Example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - apply:\n        file: path/to/apply.yaml\n    - assert:\n        file: path/to/assert.yaml\n    catch: []\n    finally: []\n
"},{"location":"steps/catch/","title":"Catch","text":"

A catch statement is also a sequence of operations or collectors.

Operations and collectors contained in a catch statement will be executed only if the step failed when executing the operations in the step's try statement.

Tip

All operations and collectors of a catch statement will be executed regardless of the success or failure of each of them.

"},{"location":"steps/catch/#more-general-catch-blocks","title":"More general catch blocks","text":"

Under certain circumstances, it can be useful to configure catch blocks at a higher level than the step grain. At the test or configuration level.

This allows for declaring common catch statements we want to execute when an error occurs. Those catch blocks are combined to produce the final catch block in the following order:

  1. catch statements from the configuration level are executed first (if any)
  2. catch statements from the test level are executed next (if any)
  3. catch statements from the step level are executed last (if any)
"},{"location":"steps/catch/#operations","title":"Operations","text":"

A catch statement supports only the following operations:

"},{"location":"steps/catch/#collectors","title":"Collectors","text":"

A catch statement supports all collectors:

"},{"location":"steps/catch/#examples","title":"Examples","text":"

step level catch block

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try: []\n    catch:\n    - command:\n        # ...\n    - script:\n        # ...\n    - delete:\n        # ...\n    - events:\n        # ...\n    - podLogs:\n        # ...\n    - describe:\n        # ...\n    - get:\n        # ...\n    - sleep:\n        # ...\n    - wait:\n        # ...\n    finally: []\n

test level catch block

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  catch:\n  - command:\n      # ...\n  - script:\n      # ...\n  - delete:\n      # ...\n  - events:\n      # ...\n  - podLogs:\n      # ...\n  - describe:\n      # ...\n  - get:\n      # ...\n  - sleep:\n      # ...\n  - wait:\n      # ...\n  steps:\n  - try: []\n    finally: []\n
"},{"location":"steps/finally/","title":"Finally","text":"

A finally statement is similar to a catch statement but will always execute after the try and eventual catch statements finished executing regardless of the success or failure of the test step.

Tip

All operations and collectors of a finally statement will be executed regardless of the success or failure of each of them.

"},{"location":"steps/finally/#operations","title":"Operations","text":"

A finally statement supports only the following operations:

"},{"location":"steps/finally/#collectors","title":"Collectors","text":"

A finally statement supports all collectors:

"},{"location":"steps/finally/#example","title":"Example","text":"

Example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try: []\n    catch: []\n    finally:\n    - command:\n        # ...\n    - script:\n        # ...\n    - delete:\n        # ...\n    - events:\n        # ...\n    - podLogs:\n        # ...\n    - describe:\n        # ...\n    - get:\n        # ...\n    - sleep:\n        # ...\n    - wait:\n        # ...\n
"},{"location":"steps/try/","title":"Try","text":"

A try statement is a sequence of operations executed in the same order they are declared. If an operation fails the entire step is considered failed.

The try statement is at the heart of a test step, it represents what the step is supposed to be about.

catch and finally statements should be viewed as complementary to the try statement.

Continue on error

By default, a test step stops executing when an operation fails and the following operations are not executed.

This behavior can be changed using the continueOnError field, if continueOnError is set to true the step will still be considered failed but execution will continue with the next operations.

"},{"location":"steps/try/#operations","title":"Operations","text":"

A try statement supports all operations:

"},{"location":"steps/try/#example","title":"Example","text":"

Example

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: example\nspec:\n  steps:\n  - try:\n    - command:\n        # ...\n    - sleep:\n        # ...\n    - script:\n        # ...\n    - apply:\n        # ...\n    - assert:\n        # ...\n    - error:\n        # ...\n    - create:\n        # ...\n    - patch:\n        # ...\n    - wait:\n        # ...\n    catch: []\n    finally: []\n
"},{"location":"tests/","title":"What is a test","text":"

To put it simply, a test can be represented as an ordered sequence of test steps.

Test steps within a test are run sequentially: if any of the test steps fail, the entire test is considered failed.

A test step can consist in one or more operations:

"},{"location":"tests/#different-syntaxes","title":"Different syntaxes","text":"

Chainsaw supports two different test definition syntaxes. Each syntax has pros and cons, see the descriptions below for more details on advantages and pitfalls.

"},{"location":"tests/#manifests-based-syntax","title":"Manifests based syntax","text":"

This is the simplest and less verbose syntax.

You provide bare Kubernetes resource manifests and Chainsaw will use those manifests to create, update, or assert expectations against a cluster.

While this syntax is extremly simple, not all operations are supported. For example, delete, command, script and sleep operations are not.

Another strong limitation is that it is not possible to specify additional configuration per test, step or operation.

Finally, this syntax relies heavily on file naming conventions, it can be error prone and makes it hard to reuse files across multiple tests.

"},{"location":"tests/#test-based-syntax","title":"Test based syntax","text":"

The Test based syntax is a more verbose and explicit syntax.

It does not rely on file naming conventions to determine test steps order and allows to easily reuse files accross multiple tests.

This syntax also comes with no limitations to provide additional configuration at the test, step or operation level.

Making a choice

Choosing one syntax over the other is not a trivial choice, every one will have its own preference and/or constraints.

It's usually easier to start with the manifests based syntax. However, as test suites grow and tests become more complex, it is often necessary to configure options on a per test, step or operation basis and the Test based syntax becomes necessary.

"},{"location":"tests/#namespaced-resources","title":"Namespaced resources","text":"

Kubernetes organizes resources into two primary scopes: namespaced and cluster-scoped.

While namespaced resources belong to a specific namespace, cluster-scoped resources span across the entire Kubernetes cluster.

"},{"location":"tests/#purpose-of-the-namespacer-interface","title":"Purpose of the Namespacer Interface","text":"

The Namespacer interface ensures automated and consistent namespace assignment to Kubernetes resources.

"},{"location":"tests/#cleanup","title":"Cleanup","text":"

Unless configured differently, by default Chainsaw will automatically cleanup the resources it created after a test finishes. Cleanup happens in reverse order of creation (created last, cleaned up first).

Note that Chainsaw performs a blocking deletion, that is, it will wait the resource is actually not present anymore in the cluster before proceeding with the next resource cleanup.

This is important, especially when the controller being tested makes use of finalizers.

Overriding cleanup timeout

A global cleanup timeout can be defined at the configuration level or using command line flags.

It can also be overriden on a per test or per test step basis but not at the operation level.

"},{"location":"tests/#parallel-execution-of-tests","title":"Parallel Execution of Tests","text":"

While Chainsaw ensures that the steps within a test are executed sequentially, it is designed to run multiple tests in parallel to each other. This parallel execution helps in significantly reducing the overall time required to run an entire suite of tests, making the testing process more efficient, especially in scenarios with a large number of tests.

Parallel execution of tests

By default, Chainsaw will run tests in parallel.

This can be configured at the configuration level or using command line flags. However, individual tests can be configured to run concurrently by setting Concurrent: true in their TestSpec.

All non-concurrent tests are executed first, followed by the concurrent tests in parallel.

"},{"location":"tests/manifests-based/","title":"Manifests based syntax","text":"

This is the simplest and less verbose supported syntax.

You provide bare Kubernetes resource manifests and Chainsaw will use those manifests to create, update, or assert expectations against a cluster.

While this syntax is simple, it doesn't support deletion operations and doesn't allow specifying additional configuration per test or step.

It also relies a lot on file naming conventions and makes it hard to reuse files across multiple tests.

"},{"location":"tests/manifests-based/#file-naming-convention","title":"File naming convention","text":"

Manifest files must follow the naming convention <step index>-<name|assert|errors>.yaml.

As an example 00-configmap.yaml, 01-assert.yaml and 02-errors.yaml can all be considered valid file names.

It's also perfectly valid to have multiple files for the same step. Let's imagine we have the following files 00-resources.yaml, 00-more-resources.yaml, 00-assert.yaml and 00-errors.yaml:

With the four files above, Chainsaw will assemble a test step made of the combination of all those files.

"},{"location":"tests/manifests-based/#manifests-loading-process","title":"Manifests loading process","text":"

The logic to determine the content of a step is always:

"},{"location":"tests/manifests-based/#example","title":"Example","text":""},{"location":"tests/manifests-based/#01-configmapyaml","title":"01-configmap.yaml","text":"

The manifest below contains a config map in a file called 01-configmap.yaml. Chainsaw will associate this manifest with an apply operation in step 01.

apiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: chainsaw-quick-start\ndata:\n  foo: bar\n
"},{"location":"tests/manifests-based/#02-assertyaml","title":"02-assert.yaml","text":"

The manifest below contains an assertion statement in a file called 02-assert.yaml. Chainsaw will associate this manifest with an assert operation in step 02.

apiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: chainsaw-quick-start\ndata:\n  foo: bar\n
"},{"location":"tests/manifests-based/#03-errorsyaml","title":"03-errors.yaml","text":"

The manifest below contains an error statement in a file called 03-errors.yaml. Chainsaw will associate this manifest with an error operation in step 03.

apiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: chainsaw-quick-start\ndata:\n  lorem: ipsum\n
"},{"location":"tests/manifests-based/#conclusion","title":"Conclusion","text":"

This test will first create a config map, then assert the content of the config map contains the foo: bar data, and then verify that the configmap does not contain the lorem: ipsum data.

Assert and error statements are very similar to standard resource definitions, they share the same structure but can be partial. Chainsaw will process only the elements present in the statements against the actual resource definition.

"},{"location":"tests/test-based/","title":"Test based syntax","text":"

The Test based syntax is more verbose than the manifests based syntax but offers more flexibility and features:

"},{"location":"tests/test-based/#the-test-resource","title":"The Test resource","text":"

A Test resource, like any Kubernetes resource has an apiVersion, kind and metadata section.

It also comes with a spec section used to declaratively represent the steps of a test and other configuration elements belonging to the test being defined.

Reference documentation

The full structure of the Test resource is documented here.

"},{"location":"tests/test-based/#test-steps","title":"Test steps","text":"

A Test is mostly made of test steps. Test steps are detailed in a dedicated documentation.

"},{"location":"tests/test-based/#example","title":"Example","text":""},{"location":"tests/test-based/#chainsaw-test","title":"chainsaw-test","text":"

The manifest below contains a Test in a file called chainsaw-test.yaml (or chainsaw-test.yml).

Chainsaw will load the Test and steps defined in its spec section.

The Test uses a custom timeouts for the whole test. Note that timeouts could have been overridden in specific steps if needed.

apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: test-name\nspec:\n  skip: false\n  concurrent: false\n  skipDelete: false\n  # these timeouts are applied per operation\n  timeouts:\n    apply: 10s\n    assert: 10s\n    error: 10s\n  steps:\n  # first step\n  # apply a configmap to the cluster\n  # the path to the configmap is relative to the folder\n  # containing the test, hence allow reusing manifests\n  # across multiple tests\n  - try:\n    - apply:\n        file: ../resources/configmap.yaml\n  # second step\n  # execute assert statements against existing resources\n  # in the cluster\n  - try:\n    - assert:\n        file: ../resources/configmap-assert.yaml\n  # third step\n  # execute error statements against existing resources\n  # in the cluster\n  - try:\n    - error:\n        file: ../resources/configmap-error.yaml\n
"},{"location":"tests/test-based/#specifying-a-custom-test-file","title":"Specifying a Custom Test File","text":"

If you have your test defined in a different file, you can specify it when running Chainsaw:

chainsaw test . --test-file <custom-test-file-name>.yaml\n

If you don't provide a file extension, chainsaw will search for a file with the .yaml extension first and the .yml extension if not found. If you provide a file extension, chainsaw will only search for a file with the extension you provided.

"},{"location":"tests/test-based/#raw-resource-support","title":"Raw Resource Support","text":"

Chainsaw now includes the raw resource feature, allowing direct specification of Kubernetes resources within the test definitions.

This feature offers a more streamlined approach for defining resources, especially useful for simpler test scenarios or for cases where resource definitions need to be reused or slightly modified across different tests.

"},{"location":"tests/test-based/#example-raw-resource","title":"Example Raw Resource","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: test-name\nspec:\n  skip: false\n  concurrent: false\n  skipDelete: false\n  timeouts:\n    apply: 10s\n    assert: 10s\n    error: 10s\n  steps:\n  # first step applies a configmap directly to the cluster\n  - try:\n    - apply:\n        resource:\n          apiVersion: v1\n          kind: ConfigMap\n          metadata:\n            name: chainsaw-quick-start\n          data:\n            foo: bar\n  # second step executes assert statements against existing resources\n  - try:\n    - assert:\n        file: ../resources/configmap-assert.yaml\n  # third step executes error statements against existing resources\n  - try:\n    - error:\n        file: ../resources/configmap-error.yaml\n
"},{"location":"tests/test-based/#url-support-for-file-references","title":"URL Support for File References","text":"

Chainsaw has support for URLs in file references for assert, apply, error and similar operations.

This feature enhances the reach of Chainsaw by allowing users to reference files hosted on remote locations, such as GitHub raw URLs or other web URLs, directly within their test definitions.

"},{"location":"tests/test-based/#example-url-file-reference","title":"Example URL File Reference","text":"
apiVersion: chainsaw.kyverno.io/v1alpha1\nkind: Test\nmetadata:\n  name: test-name\nspec:\n  skip: false\n  concurrent: false\n  skipDelete: false\n  timeouts:\n    apply: 10s\n    assert: 10s\n    error: 10s\n  steps:\n  # first step\n  # apply a Kubernetes manifest hosted at a GitHub raw URL\n  - try:\n    - apply:\n        file: https://raw.githubusercontent.com/username/repo/branch/path/to/configmap.yaml\n  # second step\n  # execute assert statements against existing resources\n  # using a file hosted on another web URL\n  - try:\n    - assert:\n        file: https://example.com/path/to/configmap-assert.yaml\n  # third step\n  # execute error statements against existing resources\n  - try:\n    - error:\n        file: https://mywebsite.com/path/to/configmap-error.yaml\n
"}]} \ No newline at end of file diff --git a/main/sitemap.xml b/main/sitemap.xml index c0352c360..938a1723c 100644 --- a/main/sitemap.xml +++ b/main/sitemap.xml @@ -410,11 +410,6 @@ 2024-05-22 daily - - https://github.io/kyverno/chainsaw/main/quick-start/advanced/cluster-wide-resources/ - 2024-05-22 - daily - https://github.io/kyverno/chainsaw/main/quick-start/advanced/concurrency/ 2024-05-22 diff --git a/main/sitemap.xml.gz b/main/sitemap.xml.gz index f5ded2857..24fe51c49 100644 Binary files a/main/sitemap.xml.gz and b/main/sitemap.xml.gz differ diff --git a/main/steps/catch/index.html b/main/steps/catch/index.html index 687b9ff18..ef6f2783f 100644 --- a/main/steps/catch/index.html +++ b/main/steps/catch/index.html @@ -1,4 +1,4 @@ - Catch - Chainsaw
Skip to content

Catch

A catch statement is also a sequence of operations or collectors.

Operations and collectors contained in a catch statement will be executed only if the step failed when executing the operations in the step's try statement.

Tip

All operations and collectors of a catch statement will be executed regardless of the success or failure of each of them.

More general catch blocks

Under certain circumstances, it can be useful to configure catch blocks at a higher level than the step grain. At the test or configuration level.

This allows for declaring common catch statements we want to execute when an error occurs. Those catch blocks are combined to produce the final catch block in the following order:

  1. catch statements from the configuration level are executed first (if any)
  2. catch statements from the test level are executed next (if any)
  3. catch statements from the step level are executed last (if any)

Operations

A catch statement supports only the following operations:

Collectors

A catch statement supports all collectors:

Examples

step level catch block

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Catch - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Catch
 
A catch statement is also a sequence of operations or collectors.
 
Operations and collectors contained in a catch statement will be executed only if the step failed when executing the operations in the step's try statement.
 
 
Tip
 
All operations and collectors of a catch statement will be executed regardless of the success or failure of each of them.
 
 
More general catch blocks
 
Under certain circumstances, it can be useful to configure catch blocks at a higher level than the step grain. At the test or configuration level.
 
This allows for declaring common catch statements we want to execute when an error occurs. Those catch blocks are combined to produce the final catch block in the following order:
 
     
  1. catch statements from the configuration level are executed first (if any)
    2.  
  2. catch statements from the test level are executed next (if any)
    3.  
  3. catch statements from the step level are executed last (if any)
    4.  
 
Operations
 
A catch statement supports only the following operations:
  
Collectors
 
A catch statement supports all collectors:
  
Examples
 
 
step level catch block
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -52,4 +52,4 @@
   steps:
   - try: []
     finally: []
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/steps/finally/index.html b/main/steps/finally/index.html index 4190f51eb..f93fe3b8a 100644 --- a/main/steps/finally/index.html +++ b/main/steps/finally/index.html @@ -1,4 +1,4 @@ - Finally - Chainsaw
Skip to content

Finally

A finally statement is similar to a catch statement but will always execute after the try and eventual catch statements finished executing regardless of the success or failure of the test step.

Tip

All operations and collectors of a finally statement will be executed regardless of the success or failure of each of them.

Operations

A finally statement supports only the following operations:

Collectors

A finally statement supports all collectors:

Example

Example

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Finally - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Finally
 
A finally statement is similar to a catch statement but will always execute after the try and eventual catch statements finished executing regardless of the success or failure of the test step.
 
 
Tip
 
All operations and collectors of a finally statement will be executed regardless of the success or failure of each of them.
 
 
Operations
 
A finally statement supports only the following operations:
  
Collectors
 
A finally statement supports all collectors:
  
Example
 
 
Example
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -25,4 +25,4 @@
         # ...
     - wait:
         # ...
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/steps/index.html b/main/steps/index.html index ec817fa44..dac8d73be 100644 --- a/main/steps/index.html +++ b/main/steps/index.html @@ -1,4 +1,4 @@ - What is a test step - Chainsaw
Skip to content

What is a test step

A test step is made of three main components used to determine the actions Chainsaw will perform when executing the step.

  1. The try statement (required)
  2. The catch statement (optional)
  3. The finally statement (optional)

Reference documentation

The full structure of the TestStep is documented here.

Test step lifecycle

Test step lifecycle

  1. The step starts executing operations in the try statement
  2. If an operation fails in the try statement
    1. If a catch statement is present, all operations and collectors are executed
  3. If a finally statement is present, all operations and collectors are executed

Example

Example

apiVersion: chainsaw.kyverno.io/v1alpha1
+ What is a test step - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
What is a test step
 
A test step is made of three main components used to determine the actions Chainsaw will perform when executing the step.
 
     
  1. The try statement (required)
    2.  
  2. The catch statement (optional)
    3.  
  3. The finally statement (optional)
    4.  
 
 
Reference documentation
 
The full structure of the TestStep is documented here.
 
 
Test step lifecycle
 
 
Test step lifecycle
 
     
  1. The step starts executing operations in the try statement
    2.  
  2. If an operation fails in the try statement
       
    1. If a catch statement is present, all operations and collectors are executed
      2.  
     
    3.  
  3. If a finally statement is present, all operations and collectors are executed
    4.  
 
 
Example
 
 
Example
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -11,4 +11,4 @@
         file: path/to/assert.yaml
     catch: []
     finally: []
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/steps/try/index.html b/main/steps/try/index.html index a48d550c1..9a4c1c03f 100644 --- a/main/steps/try/index.html +++ b/main/steps/try/index.html @@ -1,4 +1,4 @@ - Try - Chainsaw
Skip to content

Try

A try statement is a sequence of operations executed in the same order they are declared. If an operation fails the entire step is considered failed.

The try statement is at the heart of a test step, it represents what the step is supposed to be about.

catch and finally statements should be viewed as complementary to the try statement.

Continue on error

By default, a test step stops executing when an operation fails and the following operations are not executed.

This behavior can be changed using the continueOnError field, if continueOnError is set to true the step will still be considered failed but execution will continue with the next operations.

Operations

A try statement supports all operations:

Example

Example

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Try - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Try
 
A try statement is a sequence of operations executed in the same order they are declared. If an operation fails the entire step is considered failed.
 
The try statement is at the heart of a test step, it represents what the step is supposed to be about.
 
catch and finally statements should be viewed as complementary to the try statement.
 
 
Continue on error
 
By default, a test step stops executing when an operation fails and the following operations are not executed.
 
This behavior can be changed using the continueOnError field, if continueOnError is set to true the step will still be considered failed but execution will continue with the next operations.
 
 
Operations
 
A try statement supports all operations:
  
Example
 
 
Example
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: example
@@ -25,4 +25,4 @@
         # ...
     catch: []
     finally: []
-
 
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file diff --git a/main/tests/index.html b/main/tests/index.html index 61308b0db..a1f970e03 100644 --- a/main/tests/index.html +++ b/main/tests/index.html @@ -1 +1 @@ - What is a test - Chainsaw
Skip to content

What is a test

To put it simply, a test can be represented as an ordered sequence of test steps.

Test steps within a test are run sequentially: if any of the test steps fail, the entire test is considered failed.

A test step can consist in one or more operations:

  • To delete resources present in a cluster
  • To create or update resources in a cluster
  • To assert one or more resources in a cluster meet the expectations (or the opposite)
  • To run arbitrary commands or scripts

Different syntaxes

Chainsaw supports two different test definition syntaxes. Each syntax has pros and cons, see the descriptions below for more details on advantages and pitfalls.

Manifests based syntax

This is the simplest and less verbose syntax.

You provide bare Kubernetes resource manifests and Chainsaw will use those manifests to create, update, or assert expectations against a cluster.

While this syntax is extremly simple, not all operations are supported. For example, delete, command, script and sleep operations are not.

Another strong limitation is that it is not possible to specify additional configuration per test, step or operation.

Finally, this syntax relies heavily on file naming conventions, it can be error prone and makes it hard to reuse files across multiple tests.

Test based syntax

The Test based syntax is a more verbose and explicit syntax.

It does not rely on file naming conventions to determine test steps order and allows to easily reuse files accross multiple tests.

This syntax also comes with no limitations to provide additional configuration at the test, step or operation level.

Making a choice

Choosing one syntax over the other is not a trivial choice, every one will have its own preference and/or constraints.

It's usually easier to start with the manifests based syntax. However, as test suites grow and tests become more complex, it is often necessary to configure options on a per test, step or operation basis and the Test based syntax becomes necessary.

Namespaced resources

Kubernetes organizes resources into two primary scopes: namespaced and cluster-scoped.

While namespaced resources belong to a specific namespace, cluster-scoped resources span across the entire Kubernetes cluster.

Purpose of the Namespacer Interface

The Namespacer interface ensures automated and consistent namespace assignment to Kubernetes resources.

  • Automated Namespacing

    Automatically assign namespaces to resources that don't have one.

  • Ephemeral Namespaces

    Handles temporary namespaces for specific tasks.

Cleanup

Unless configured differently, by default Chainsaw will automatically cleanup the resources it created after a test finishes. Cleanup happens in reverse order of creation (created last, cleaned up first).

Note that Chainsaw performs a blocking deletion, that is, it will wait the resource is actually not present anymore in the cluster before proceeding with the next resource cleanup.

This is important, especially when the controller being tested makes use of finalizers.

Overriding cleanup timeout

A global cleanup timeout can be defined at the configuration level or using command line flags.

It can also be overriden on a per test or per test step basis but not at the operation level.

Parallel Execution of Tests

While Chainsaw ensures that the steps within a test are executed sequentially, it is designed to run multiple tests in parallel to each other. This parallel execution helps in significantly reducing the overall time required to run an entire suite of tests, making the testing process more efficient, especially in scenarios with a large number of tests.

Parallel execution of tests

By default, Chainsaw will run tests in parallel.

This can be configured at the configuration level or using command line flags. However, individual tests can be configured to run concurrently by setting Concurrent: true in their TestSpec.

All non-concurrent tests are executed first, followed by the concurrent tests in parallel.

\ No newline at end of file + What is a test - Chainsaw
Skip to content

What is a test

To put it simply, a test can be represented as an ordered sequence of test steps.

Test steps within a test are run sequentially: if any of the test steps fail, the entire test is considered failed.

A test step can consist in one or more operations:

  • To delete resources present in a cluster
  • To create or update resources in a cluster
  • To assert one or more resources in a cluster meet the expectations (or the opposite)
  • To run arbitrary commands or scripts

Different syntaxes

Chainsaw supports two different test definition syntaxes. Each syntax has pros and cons, see the descriptions below for more details on advantages and pitfalls.

Manifests based syntax

This is the simplest and less verbose syntax.

You provide bare Kubernetes resource manifests and Chainsaw will use those manifests to create, update, or assert expectations against a cluster.

While this syntax is extremly simple, not all operations are supported. For example, delete, command, script and sleep operations are not.

Another strong limitation is that it is not possible to specify additional configuration per test, step or operation.

Finally, this syntax relies heavily on file naming conventions, it can be error prone and makes it hard to reuse files across multiple tests.

Test based syntax

The Test based syntax is a more verbose and explicit syntax.

It does not rely on file naming conventions to determine test steps order and allows to easily reuse files accross multiple tests.

This syntax also comes with no limitations to provide additional configuration at the test, step or operation level.

Making a choice

Choosing one syntax over the other is not a trivial choice, every one will have its own preference and/or constraints.

It's usually easier to start with the manifests based syntax. However, as test suites grow and tests become more complex, it is often necessary to configure options on a per test, step or operation basis and the Test based syntax becomes necessary.

Namespaced resources

Kubernetes organizes resources into two primary scopes: namespaced and cluster-scoped.

While namespaced resources belong to a specific namespace, cluster-scoped resources span across the entire Kubernetes cluster.

Purpose of the Namespacer Interface

The Namespacer interface ensures automated and consistent namespace assignment to Kubernetes resources.

  • Automated Namespacing

    Automatically assign namespaces to resources that don't have one.

  • Ephemeral Namespaces

    Handles temporary namespaces for specific tasks.

Cleanup

Unless configured differently, by default Chainsaw will automatically cleanup the resources it created after a test finishes. Cleanup happens in reverse order of creation (created last, cleaned up first).

Note that Chainsaw performs a blocking deletion, that is, it will wait the resource is actually not present anymore in the cluster before proceeding with the next resource cleanup.

This is important, especially when the controller being tested makes use of finalizers.

Overriding cleanup timeout

A global cleanup timeout can be defined at the configuration level or using command line flags.

It can also be overriden on a per test or per test step basis but not at the operation level.

Parallel Execution of Tests

While Chainsaw ensures that the steps within a test are executed sequentially, it is designed to run multiple tests in parallel to each other. This parallel execution helps in significantly reducing the overall time required to run an entire suite of tests, making the testing process more efficient, especially in scenarios with a large number of tests.

Parallel execution of tests

By default, Chainsaw will run tests in parallel.

This can be configured at the configuration level or using command line flags. However, individual tests can be configured to run concurrently by setting Concurrent: true in their TestSpec.

All non-concurrent tests are executed first, followed by the concurrent tests in parallel.

\ No newline at end of file diff --git a/main/tests/manifests-based/index.html b/main/tests/manifests-based/index.html index facca21d2..65bc43c1c 100644 --- a/main/tests/manifests-based/index.html +++ b/main/tests/manifests-based/index.html @@ -1,4 +1,4 @@ - Manifests based syntax - Chainsaw
Skip to content

Manifests based syntax

This is the simplest and less verbose supported syntax.

You provide bare Kubernetes resource manifests and Chainsaw will use those manifests to create, update, or assert expectations against a cluster.

While this syntax is simple, it doesn't support deletion operations and doesn't allow specifying additional configuration per test or step.

It also relies a lot on file naming conventions and makes it hard to reuse files across multiple tests.

File naming convention

Manifest files must follow the naming convention <step index>-<name|assert|errors>.yaml.

As an example 00-configmap.yaml, 01-assert.yaml and 02-errors.yaml can all be considered valid file names.

It's also perfectly valid to have multiple files for the same step. Let's imagine we have the following files 00-resources.yaml, 00-more-resources.yaml, 00-assert.yaml and 00-errors.yaml:

  • 00-resources.yaml and 00-more-resources.yaml contain resources that will be applied in step 00
  • 00-assert.yaml contains assert statements in step 00
  • 00-errors.yaml contains error statements in step 00

With the four files above, Chainsaw will assemble a test step made of the combination of all those files.

Manifests loading process

The logic to determine the content of a step is always:

  • The step index is obtained from the beginning of the file name, it must be composed of two numbers between 0 and 9 (from 00 to 99)
  • The next character acts as a separator and is expected to be -
  • The rest of the file name (without extension) is then evaluated
    • If it is equal to assert, the content is considered assertion statements
    • If it is equal to error, the content is considered error statements
    • Else the content is considered resources to be applied
  • The extension must be .yaml or .yml

Example

01-configmap.yaml

The manifest below contains a config map in a file called 01-configmap.yaml. Chainsaw will associate this manifest with an apply operation in step 01.

apiVersion: v1
+ Manifests based syntax - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Manifests based syntax
 
This is the simplest and less verbose supported syntax.
 
You provide bare Kubernetes resource manifests and Chainsaw will use those manifests to create, update, or assert expectations against a cluster.
 
While this syntax is simple, it doesn't support deletion operations and doesn't allow specifying additional configuration per test or step.
 
It also relies a lot on file naming conventions and makes it hard to reuse files across multiple tests.
 
File naming convention
 
Manifest files must follow the naming convention <step index>-<name|assert|errors>.yaml.
 
As an example 00-configmap.yaml, 01-assert.yaml and 02-errors.yaml can all be considered valid file names.
 
It's also perfectly valid to have multiple files for the same step. Let's imagine we have the following files 00-resources.yaml, 00-more-resources.yaml, 00-assert.yaml and 00-errors.yaml:
 
     
  • 00-resources.yaml and 00-more-resources.yaml contain resources that will be applied in step 00
    •  
  • 00-assert.yaml contains assert statements in step 00
    •  
  • 00-errors.yaml contains error statements in step 00
    •  
 
With the four files above, Chainsaw will assemble a test step made of the combination of all those files.
 
Manifests loading process
 
The logic to determine the content of a step is always:
 
     
  • The step index is obtained from the beginning of the file name, it must be composed of two numbers between 0 and 9 (from 00 to 99)
    •  
  • The next character acts as a separator and is expected to be -
    •  
  • The rest of the file name (without extension) is then evaluated
       
    • If it is equal to assert, the content is considered assertion statements
      •  
    • If it is equal to error, the content is considered error statements
      •  
    • Else the content is considered resources to be applied
      •  
     
    •  
  • The extension must be .yaml or .yml
    •  
 
Example
 
01-configmap.yaml
 
The manifest below contains a config map in a file called 01-configmap.yaml. Chainsaw will associate this manifest with an apply operation in step 01.
 
apiVersion: v1
 kind: ConfigMap
 metadata:
   name: chainsaw-quick-start
@@ -16,4 +16,4 @@
   name: chainsaw-quick-start
 data:
   lorem: ipsum
-
 
Conclusion
 
This test will first create a config map, then assert the content of the config map contains the foo: bar data, and then verify that the configmap does not contain the lorem: ipsum data.
 
Assert and error statements are very similar to standard resource definitions, they share the same structure but can be partial. Chainsaw will process only the elements present in the statements against the actual resource definition.
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+

Conclusion

This test will first create a config map, then assert the content of the config map contains the foo: bar data, and then verify that the configmap does not contain the lorem: ipsum data.

Assert and error statements are very similar to standard resource definitions, they share the same structure but can be partial. Chainsaw will process only the elements present in the statements against the actual resource definition.

\ No newline at end of file diff --git a/main/tests/test-based/index.html b/main/tests/test-based/index.html index 3d141f6c6..a75f389a5 100644 --- a/main/tests/test-based/index.html +++ b/main/tests/test-based/index.html @@ -1,4 +1,4 @@ - Test based syntax - Chainsaw
Skip to content

Test based syntax

The Test based syntax is more verbose than the manifests based syntax but offers more flexibility and features:

  • Does not rely on file naming conventions for operations ordering
  • Allows to easily reuse files accross multiple tests
  • Offers the flexibility to provide additional configuration at the test, step and operation level
  • Supports all operations and collectors

The Test resource

A Test resource, like any Kubernetes resource has an apiVersion, kind and metadata section.

It also comes with a spec section used to declaratively represent the steps of a test and other configuration elements belonging to the test being defined.

Reference documentation

The full structure of the Test resource is documented here.

Test steps

A Test is mostly made of test steps. Test steps are detailed in a dedicated documentation.

Example

chainsaw-test

The manifest below contains a Test in a file called chainsaw-test.yaml (or chainsaw-test.yml).

Chainsaw will load the Test and steps defined in its spec section.

The Test uses a custom timeouts for the whole test. Note that timeouts could have been overridden in specific steps if needed.

apiVersion: chainsaw.kyverno.io/v1alpha1
+ Test based syntax - Chainsaw      
 
  
  
 
  
 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
       
Test based syntax
 
The Test based syntax is more verbose than the manifests based syntax but offers more flexibility and features:
 
     
  • Does not rely on file naming conventions for operations ordering
    •  
  • Allows to easily reuse files accross multiple tests
    •  
  • Offers the flexibility to provide additional configuration at the test, step and operation level
    •  
  • Supports all operations and collectors
    •  
 
The Test resource
 
A Test resource, like any Kubernetes resource has an apiVersion, kind and metadata section.
 
It also comes with a spec section used to declaratively represent the steps of a test and other configuration elements belonging to the test being defined.
 
 
Reference documentation
 
The full structure of the Test resource is documented here.
 
 
Test steps
 
A Test is mostly made of test steps. Test steps are detailed in a dedicated documentation.
 
Example
 
chainsaw-test
 
The manifest below contains a Test in a file called chainsaw-test.yaml (or chainsaw-test.yml).
 
Chainsaw will load the Test and steps defined in its spec section.
 
The Test uses a custom timeouts for the whole test. Note that timeouts could have been overridden in specific steps if needed.
 
apiVersion: chainsaw.kyverno.io/v1alpha1
 kind: Test
 metadata:
   name: test-name
@@ -93,4 +93,4 @@
   - try:
     - error:
         file: https://mywebsite.com/path/to/configmap-error.yaml
-
 
 
  
  
  
 
 
 
     
\ No newline at end of file
+
\ No newline at end of file