From 7476dd218d21c27fc4c23619fa0200df8b61354f Mon Sep 17 00:00:00 2001 From: Yoshiki Fujikane Date: Thu, 5 Oct 2023 15:40:12 +0900 Subject: [PATCH 1/3] Add rfc for script run stage Signed-off-by: Yoshiki Fujikane --- docs/rfcs/0011-script-run-stage.md | 107 +++++++++++++++++++++++++++++ 1 file changed, 107 insertions(+) create mode 100644 docs/rfcs/0011-script-run-stage.md diff --git a/docs/rfcs/0011-script-run-stage.md b/docs/rfcs/0011-script-run-stage.md new file mode 100644 index 0000000000..840e175cbb --- /dev/null +++ b/docs/rfcs/0011-script-run-stage.md @@ -0,0 +1,107 @@ +- Start Date: (fill me in with today's date, YYYY-MM-DD) +- Target Version: (1.x / 2.x) + +# Summary + +This RFC introduces a new way to enable users to use "script run stages” that users can execute any commands in their pipelines. + +# Motivation + +Currently, users can use only stages that PipeCD has already defined. However some users want to define new stages by their use-cases as below. + +- Deploying infrastructure by tools other than that PipeCD supports (terraform and kubernetes) such as SAM, cloud formation…. +- Running End to End tests +- Interacting with external systems +- Performing database migrations +- notifying the deployed result + +`CUSTOM_SYNC` is implemented for the above use-cases, but it is for sync. +So more simply, some users want to execute commands. + +# Detailed design + +## feature + +1. execute any commands in their pipeline. + +``` +apiVersion: pipecd.dev/v1beta1 +kind: LambdaApp +spec: + encryptedSecrets: + password: encrypted-secrets + pipeline: + stages: + - name: SCRIPT_RUN + env: + AWS_PROFILE: default + runs: + - "echo {{ .encryptedSecrets.password }} | sudo -S su" + - "sam build" + - "sam deploy -g --profile $AWS_PROFILE" +``` + +2. combine with other stage + +Compared to CUSTOM_SYNC, this stage can be combined with other stage. +For example, + +``` +apiVersion: pipecd.dev/v1beta1 +kind: KubernetesApp +spec: + pipeline: + stages: + - name: K8S_CANARY_ROLLOUT + with: + replicas: 10% + - name: WAIT_APPROVAL + with: + timeout: 30m + - name: K8S_PRIMARY_ROLLOUT + - name: K8S_CANARY_CLEAN + - name: SCRIPT_RUN + env: + SLACK_WEBHOOK_URL: "" + runs: + - "curl -X POST -H 'Content-type: application/json' --data '{"text":"successfully deployed!!"}' $SLACK_WEBHOOK_URL" +``` + +3. execute any commands when rollback + +Users can define commands to execute when rolling back with `runsOnRollback`. +If `runsOnRollback` is not set, nothing to execute when rolling back. + +``` +apiVersion: pipecd.dev/v1beta1 +kind: KubernetesApp +spec: + pipeline: + stages: + - name: K8S_CANARY_ROLLOUT + with: + replicas: 10% + - name: WAIT_APPROVAL + with: + timeout: 30m + - name: K8S_PRIMARY_ROLLOUT + - name: K8S_CANARY_CLEAN + - name: SCRIPT_RUN + env: + SLACK_WEBHOOK_URL: "" + runs: + - "curl -X POST -H 'Content-type: application/json' --data '{"text":"successfully deployed!!"}' $SLACK_WEBHOOK_URL" + runsOnRollback: + - "curl -X POST -H 'Content-type: application/json' --data '{"text":"failed to deploy: rollback"}' $SLACK_WEBHOOK_URL" +``` + +# Alternatives + +**What's the difference between "CUSTOM_SYNC" and "SCRIPT_SYNC"?** + +"CUSTOM_SYNC" is one of the stages to **sync**, but "SCRIPT_RUN" is the stage to **execute commands**. + +# Unresolved questions + +It might be better to add `runsOnRollback` on each sync stage if users want to control when rollback. +This is a just idea. From e705908d7d074338577130c4417ae9f7f3a3a90e Mon Sep 17 00:00:00 2001 From: Yoshiki Fujikane Date: Wed, 18 Oct 2023 16:53:15 +0900 Subject: [PATCH 2/3] Fix docs Signed-off-by: Yoshiki Fujikane --- docs/rfcs/0011-script-run-stage.md | 141 ++++++++++++++++++++++++----- 1 file changed, 118 insertions(+), 23 deletions(-) diff --git a/docs/rfcs/0011-script-run-stage.md b/docs/rfcs/0011-script-run-stage.md index 840e175cbb..bb61594bc0 100644 --- a/docs/rfcs/0011-script-run-stage.md +++ b/docs/rfcs/0011-script-run-stage.md @@ -24,7 +24,7 @@ So more simply, some users want to execute commands. 1. execute any commands in their pipeline. -``` +```yaml apiVersion: pipecd.dev/v1beta1 kind: LambdaApp spec: @@ -33,12 +33,13 @@ spec: pipeline: stages: - name: SCRIPT_RUN - env: - AWS_PROFILE: default - runs: - - "echo {{ .encryptedSecrets.password }} | sudo -S su" - - "sam build" - - "sam deploy -g --profile $AWS_PROFILE" + with: + env: + AWS_PROFILE: default + runs: + - "echo {{ .encryptedSecrets.password }} | sudo -S su" + - "sam build" + - "sam deploy -g --profile $AWS_PROFILE" ``` 2. combine with other stage @@ -46,7 +47,7 @@ spec: Compared to CUSTOM_SYNC, this stage can be combined with other stage. For example, -``` +```yaml apiVersion: pipecd.dev/v1beta1 kind: KubernetesApp spec: @@ -61,18 +62,40 @@ spec: - name: K8S_PRIMARY_ROLLOUT - name: K8S_CANARY_CLEAN - name: SCRIPT_RUN - env: - SLACK_WEBHOOK_URL: "" - runs: - - "curl -X POST -H 'Content-type: application/json' --data '{"text":"successfully deployed!!"}' $SLACK_WEBHOOK_URL" + with: + env: + SLACK_WEBHOOK_URL: "" + runs: + - "curl -X POST -H 'Content-type: application/json' --data '{"text":"successfully deployed!!"}' $SLACK_WEBHOOK_URL" ``` -3. execute any commands when rollback +## when to rollback -Users can define commands to execute when rolling back with `runsOnRollback`. -If `runsOnRollback` is not set, nothing to execute when rolling back. +Users can define commands to execute with `onRollback` when rolling back. +If `onRollback` is not set, nothing to execute when rolling back. +```yaml +apiVersion: pipecd.dev/v1beta1 +kind: KubernetesApp +spec: + pipeline: + stages: + - name: SCRIPT_RUN + with: + env: + SLACK_WEBHOOK_URL: "" + runs: + - "curl -X POST -H 'Content-type: application/json' --data '{"text":"successfully deployed!!"}' $SLACK_WEBHOOK_URL" + onRollback: + - "curl -X POST -H 'Content-type: application/json' --data '{"text":"failed to deploy: rollback"}' $SLACK_WEBHOOK_URL" ``` + +**SCRIPT_SYNC stage also rollbacks** when the deployment status is `DeploymentStatus_DEPLOYMENT_CANCELLED` or `DeploymentStatus_DEPLOYMENT_FAILURE` even though other rollback stage is also executed. + +For example, here is a deploy pipeline combined with other k8s stages. +The result status of the pipeline is FAIL or CANCELED, piped rollbacks the stages `K8S_CANARY_ROLLOUT`, `K8S_PRIMARY_ROLLOUT`, and `SCRIPT_RUN`. + +```yaml apiVersion: pipecd.dev/v1beta1 kind: KubernetesApp spec: @@ -87,21 +110,93 @@ spec: - name: K8S_PRIMARY_ROLLOUT - name: K8S_CANARY_CLEAN - name: SCRIPT_RUN - env: - SLACK_WEBHOOK_URL: "" - runs: - - "curl -X POST -H 'Content-type: application/json' --data '{"text":"successfully deployed!!"}' $SLACK_WEBHOOK_URL" - runsOnRollback: - - "curl -X POST -H 'Content-type: application/json' --data '{"text":"failed to deploy: rollback"}' $SLACK_WEBHOOK_URL" + with: + env: + SLACK_WEBHOOK_URL: "" + runs: + - "curl -X POST -H 'Content-type: application/json' --data '{"text":"successfully deployed!!"}' $SLACK_WEBHOOK_URL" + onRollback: + - "curl -X POST -H 'Content-type: application/json' --data '{"text":"failed to deploy: rollback"}' $SLACK_WEBHOOK_URL" ``` +## prepare environment for execution + +Commands are executed on the container of piped or on the host OS(standalone). + # Alternatives -**What's the difference between "CUSTOM_SYNC" and "SCRIPT_SYNC"?** +## What's the difference between "CUSTOM_SYNC" and "SCRIPT_SYNC"? "CUSTOM_SYNC" is one of the stages to **sync**, but "SCRIPT_RUN" is the stage to **execute commands**. +## How about other CD tools? + +### Argo + +#### strategy + +- **execute command with another k8s resources** +#### details +Resource Hooks +https://argo-cd.readthedocs.io/en/stable/user-guide/resource_hooks/#resource-hooks +> Hooks are ways to run scripts before, during, and after a Sync operation. Hooks can also be run if a Sync operation fails at any point. + +- There are four points to execute command. + - PreSync: before sync + - Sync: during sync + - PostSync: after sync + - SyncFaill: failed to sync +- **To execute command, ArgoCD applys k8s resources such as Job or Pod, [[Argo Workflows]] ...** +- users set some annotations and ArgoCD detects them to control the order to execute command + - e.g. https://argo-cd.readthedocs.io/en/stable/user-guide/resource_hooks/#using-a-hook-to-send-a-slack-message + +#### pros/cons + +**pros** +- can separate respolibility for delivery and executing any command + +**cons** +- users need to prepare and manage the resource to execute any command + +### FluxCD +- There is no functions to realize that + +### Flagger + +#### strategy +- Call api set as webhooks on each points and execute command on the api +- Flagger just call api registerd as webhooks. + +#### details + +**Webhooks** +https://fluxcd.io/flagger/usage/webhooks/#load-testing +>Flagger will call each webhook URL and determine from the response status code (HTTP 2xx) if the canary is failing or not. + +- There are some webhook points. + - confirm-rollout + - pre-rollout + - rollout + - confirm-traffic-increase + - confirm-promotion + - post-rollout + - rollback + - event +- Flagger calls webhooks on each points. + - e.g. load testing with Flagger: https://fluxcd.io/flagger/usage/webhooks/#load-testing +- if users want to execute command with webhook, set the command text to `metadata` section and prepare webhook handler to execute cmd parsed from metadata. + +#### pros/cons + +**pros** +- can separate respolibility for delivery and executing any command + +**cons** +- users need to prepare api for the webhooks. + + # Unresolved questions -It might be better to add `runsOnRollback` on each sync stage if users want to control when rollback. +- It might be better to change from alpine to debian and so on to provide users popular commands (e.g. curl) by default. +- It might be better to add `runsOnRollback` on each sync stage if users want to control when rollback. This is a just idea. From 5f74cfdf6816fdf3051b43775e3c15975d55f225 Mon Sep 17 00:00:00 2001 From: Yoshiki Fujikane Date: Tue, 24 Oct 2023 18:45:51 +0900 Subject: [PATCH 3/3] Fix document Signed-off-by: Yoshiki Fujikane --- docs/rfcs/0011-script-run-stage.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/rfcs/0011-script-run-stage.md b/docs/rfcs/0011-script-run-stage.md index bb61594bc0..4e37f8d385 100644 --- a/docs/rfcs/0011-script-run-stage.md +++ b/docs/rfcs/0011-script-run-stage.md @@ -123,6 +123,12 @@ spec: Commands are executed on the container of piped or on the host OS(standalone). +## CUSTOM_SYNC in the future + +"CUSTOM_SYNC" stage will be deprecated because the "SCRIPT_RUN" has also similar features. +I expect users to use "SCRIPT_RUN" for executing any command before or after other stages. + + # Alternatives ## What's the difference between "CUSTOM_SYNC" and "SCRIPT_SYNC"?