function-shell

This Crossplane composition function is written in go following this function guide. It runs in a docker container. The package docs are a useful reference when writing functions.

This is the v1alpha1 version of function-shell. Once this pull request to introduce how to support passing credentials to composition functions has been merged, the current functionality for how to pass secrets in function-shell is expected to be enhanced with the above pattern.

The function-shell accepts commands to run in a shell and it returns the output to specified fields.

Quick Start

Here's a minimal example to get started:

input:
  apiVersion: shell.fn.crossplane.io/v1alpha1
  kind: Parameters
  shellCommand: echo "Hello from shell!"
  stdoutField: status.atFunction.shell.stdout
  stderrField: status.atFunction.shell.stderr

Parameters

The function accepts the following parameters:

shellEnvVarsRef - referencing environment variables in the function-shell Kubernetes pod that were loaded through a deploymentRuntimeConfig. The file MUST be in JSON format. It can be a Kubernetes secret. shellEnvVarsRef requires a name for the pod environment variable, and keys for the keys Inside of the JSON formatted pod environment variable that have associated values.

Example secret:

{
    "ENV_FOO": "foo value",
    "ENV_BAR": "bar value"
}

Example deploymentRuntimeConfig:

---
apiVersion: pkg.crossplane.io/v1beta1
kind: DeploymentRuntimeConfig
metadata:
  name: function-shell
spec:
  deploymentTemplate:
    spec:
      selector: {}
      replicas: 1
      template:
        spec:
          containers:
            - name: package-runtime
              args:
                - --debug
              env:
                - name: DATADOG_SECRET
                  valueFrom:
                    secretKeyRef:
                      key: credentials
                      name: datadog-secret

shellEnvVars - an array of environment variables with a key and value each. Also supports reading from Composite fields with fieldRef.
shellCommand - a shell command line that can contain pipes and redirects and calling multiple programs.
shellCommandField - a reference to a field that contains the shell command line that should be run.
stdoutField - the path to the field where the shell standard output should be written.
stderrField - the path to the field where the shell standard error output should be written.

Error Handling and Output Capture

The function-shell captures both stdout and stderr output regardless of command success or failure. This provides complete observability for debugging shell command execution.

Behavior on Success

Command exit code 0: stdout/stderr written to specified fields
Function execution marked as successful
No fatal results generated

Behavior on Failure

Command exit code != 0: stdout/stderr is captured and written to specified fields
Function execution marked as failed with SEVERITY_FATAL result
Error message includes details about the failure and captured stderr
This allows inspection of both successful output and error details

Caching Function Outputs

In Crossplane 1.20.0 and 2.0.0, Function Response Caching was added as an alpha feature. Crossplane will cache the results of a function invocation until a Time-To-Live (TTL) has been exceeded. This can significantly reduce the number of times the function is called.

To enable Function Response Caching, update the crossplane deployment by adding --enable-function-response-cache to the args of the Crossplane deployment. Enabling --debug on the Crossplane deployment will log cache activity.

helm upgrade --install crossplane --namespace crossplane-system \
--create-namespace crossplane-stable/crossplane --wait \
--set args='{"--debug","--enable-function-response-cache"}'

Next, set the cacheTTL, using a time duration like 90s, 5m, or 4h30m:

input:
  apiVersion: shell.fn.crossplane.io/v1alpha1
  kind: Parameters
  cacheTTL: 5m
  shellEnvVars:
    - key: ECHO
      value: "SGVsbG8gZnJvbSBzaGVsbAo="
  shellCommand: |
      echo ${ECHO}|base64 -d|sed s/^h/H/
  stdoutField: status.atFunction.shell.stdout
  stderrField: status.atFunction.shell.stderr

Crossplane will cache the returned results of the function for the duration of the TTL.

If the Composite Resource or any of the Managed Resources in the Composition are updated, Crossplane will invoke the function and set a new cache duration for the output.

See the echo composition.yaml for an example.

Examples

This repository includes the following examples in the example/ directory:

echo - Basic shell command execution with output capture
datadog-dashboard-ids - API integration with secret management
fieldRef - Using field references for dynamic values
ip-addr-validation - Input validation patterns

Example: Obtain Dashboard Ids from Datadog

The composition calls the function-shell instructing it to obtain dashboard ids from a Datadog account. For this, the composition specifies the name of a Kubernetes pod environment variable called DATADOG_SECRET. This environment variable was populated with the JSON of a Kubernetes datadog-secret through a deploymentRuntimeConfig. The JSON includes the DATADOG_API_KEY and DATADOG_APP_KEY keys and their values. The Datadog API endpoint is passed in a clear text DATADOG_API_URL environment variable. The shell command uses a curl to the endpoint with a header that contains the access credentials. The command output is piped into jq and filtered for the ids.

The function-shell writes the dashboard ids to the specified output status field, and any output that went to stderr into the specified stderr status field.

The composition is for illustration purposes only. When using the function-shell in your own compositions, you may want to patch function input from claim and other composition field values.

The deploymentRuntimeConfig reads a datadog secret that looks like below. Replace YOUR_API_KEY and YOUR_APP_KEY with your respective keys.

{
    "DATADOG_API_KEY": "YOUR_API_KEY",
    "DATADOG_APP_KEY": "YOUR_APP_KEY"
}

---
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: shell.upbound.io
spec:
  compositeTypeRef:
    apiVersion: upbound.io/v1alpha1
    kind: XShell
  mode: Pipeline
  pipeline:
    - step: shell
      functionRef:
        # When installed through a package manager, use
        # name: crossplane-contrib-function-shell
        name: function-shell
      input:
        apiVersion: shell.fn.crossplane.io/v1alpha1
        kind: Parameters
        # Load shellEnvVarsRef from a Kubernetes secret
        # through a deploymentRuntimeConfig into the
        # function-shell pod.
        shellEnvVarsRef:
          name: DATADOG_SECRET
          keys:
            - DATADOG_API_KEY
            - DATADOG_APP_KEY
        shellEnvVars:
          - key: DATADOG_API_URL
            value: "https://api.datadoghq.com/api/v1/dashboard"
        shellCommand: |
          curl -X GET "${DATADOG_API_URL}" \
            -H "Accept: application/json" \
            -H "DD-API-KEY: ${DATADOG_API_KEY}" \
            -H "DD-APPLICATION-KEY: ${DATADOG_APP_KEY}"|jq '.dashboards[] .id'
        stdoutField: status.atFunction.shell.stdout
        stderrField: status.atFunction.shell.stderr

The composition is selected through the following XR.

---
apiVersion: upbound.io/v1alpha1
kind: Shell
metadata:
  name: shell-1
spec: {}

The API definition is as follows. Note that the API contains status fields that are populated by function-shell.

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xshells.upbound.io
spec:
  group: upbound.io
  names:
    kind: XShell
    plural: xshells
  claimNames:
    kind: Shell
    plural: shells
  defaultCompositionRef:
    name: shell.upbound.io
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          properties:
            spec:
              properties:
                cmd:
                  type: string
            status:
              properties:
                atFunction:
                  type: object
                  x-kubernetes-preserve-unknown-fields: true

The crossplane beta trace output after applying the in-cluster shell-claim.yaml is as follows:

crossplane beta trace shell.upbound.io/datadog-dashboard-ids
NAME                                    SYNCED   READY   STATUS
Shell/datadog-dashboard-ids (default)   True     True    Available
└─ XShell/datadog-dashboard-ids-cbb6x   True     True    Available

The XShell/shell-1-ttfbh yaml output looks as per below. Notice the dashboard ids in the status.atFunction.shell.stdout field, and the curl stderr output in the status.atFunction.shell.stderr field.

kubectl get XShell/datadog-dashboard-ids-cbb6x -o yaml
apiVersion: upbound.io/v1alpha1
kind: XShell
metadata:
  creationTimestamp: "2024-04-24T04:15:53Z"
  finalizers:
  - composite.apiextensions.crossplane.io
  generateName: datadog-dashboard-ids-
  generation: 6
  labels:
    crossplane.io/claim-name: datadog-dashboard-ids
    crossplane.io/claim-namespace: default
    crossplane.io/composite: datadog-dashboard-ids-cbb6x
  name: datadog-dashboard-ids-cbb6x
  resourceVersion: "167413"
  uid: 601d3f66-80df-4f1a-8917-533ea05255cc
spec:
  claimRef:
    apiVersion: upbound.io/v1alpha1
    kind: Shell
    name: datadog-dashboard-ids
    namespace: default
  compositionRef:
    name: shell.upbound.io
  compositionRevisionRef:
    name: shell.upbound.io-e981893
  compositionUpdatePolicy: Automatic
  resourceRefs: []
status:
  atFunction:
    shell:
      stderr: "% Total    % Received % Xferd  Average Speed   Time    Time     Time
        \ Current\n                                 Dload  Upload   Total   Spent
        \   Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:--
        --:--:-- --:--:--     0\r100  4255  100  4255    0     0  10361      0 --:--:--
        --:--:-- --:--:-- 10378"
      stdout: |-
        "vn4-agn-ftd"
        "9pt-bhb-uwj"
        "6su-nff-222"
        "sm3-cxs-q98"
        "ssx-sci-uvi"
        "3fd-h4e-7w6"
        "qth-94z-ip5"
  conditions:
  - lastTransitionTime: "2024-04-24T04:20:09Z"
    reason: ReconcileSuccess
    status: "True"
    type: Synced
  - lastTransitionTime: "2024-04-24T04:15:54Z"
    reason: Available
    status: "True"
    type: Ready

Development and test

Crossplane has a cli with useful commands for building packages.

Function code generation

go generate ./...

Build the function's runtime image - see Dockerfile

docker build . --tag=runtime

Render example function output

In Terminal 1

go run . --insecure --debug

In Terminal 2

crossplane beta render \
    example/out-of-cluster/xr.yaml \
    example/out-of-cluster/composition.yaml \
    example/out-of-cluster/functions.yaml

Lint code

golangci-lint run

Run tests

go test -v -cover .

Docker build amd64 image

docker build . --quiet --platform=linux/amd64 --tag runtime-amd64

Docker build arm64 image

docker build . --quiet --platform=linux/arm64 --tag runtime-arm64

Crossplane build amd64 package

crossplane xpkg build \
    --package-root=package \
    --embed-runtime-image=runtime-amd64 \
    --package-file=function-amd64.xpkg

Crossplane build arm64 package

crossplane xpkg build \
    --package-root=package \
    --embed-runtime-image=runtime-arm64 \
    --package-file=function-arm64.xpkg

Name		Name	Last commit message	Last commit date
Latest commit History 122 Commits
.github		.github
example		example
input		input
package		package
.gitignore		.gitignore
CODEOWNERS		CODEOWNERS
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
Dockerfile		Dockerfile
LICENSE		LICENSE
Makefile		Makefile
NOTES.txt		NOTES.txt
OWNERS.md		OWNERS.md
README.md		README.md
air.toml		air.toml
env.go		env.go
env_test.go		env_test.go
fn.go		fn.go
fn_test.go		fn_test.go
go.mod		go.mod
go.sum		go.sum
main.go		main.go
renovate.json		renovate.json
validate.go		validate.go

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Repository files navigation

function-shell

Table of Contents

Quick Start

Parameters

Error Handling and Output Capture

Behavior on Success

Behavior on Failure

Caching Function Outputs

Examples

Example: Obtain Dashboard Ids from Datadog

Development and test

Function code generation

Build the function's runtime image - see Dockerfile

Render example function output

Lint code

Run tests

Docker build amd64 image

Docker build arm64 image

Crossplane build amd64 package

Crossplane build arm64 package

References

About

Uh oh!

Releases 5

Packages

Uh oh!

Uh oh!

Contributors 7

Uh oh!

Languages

License

crossplane-contrib/function-shell

Folders and files

Latest commit

History

Repository files navigation

function-shell

Table of Contents

Quick Start

Parameters

Error Handling and Output Capture

Behavior on Success

Behavior on Failure

Caching Function Outputs

Examples

Example: Obtain Dashboard Ids from Datadog

Development and test

Function code generation

Build the function's runtime image - see Dockerfile

Render example function output

Lint code

Run tests

Docker build amd64 image

Docker build arm64 image

Crossplane build amd64 package

Crossplane build arm64 package

References

About

Resources

License

Code of conduct

Uh oh!

Stars

Watchers

Forks

Releases 5

Packages 0

Uh oh!

Uh oh!

Contributors 7

Uh oh!

Languages

Packages