Introduce script run stage

[ffjlabo]

What would you like to be added:
follow: #1089

implement stage to execute any command.

• alpha: only for sync Script run stage #4720
• beta: include thinking for rollback
  • k8s Implement rollback for script run as pre defined stage #4743
  • ecs
  • cloud run
  • lambda
  • terraform

Why is this needed:

[ffjlabo]

TODO

• just executing commands not considered about rollbacking Add rfc for script run stage #4603
• consider about rollback (fix rfc and implement)
• consider about managing commands

[ffjlabo]

📝

• SCRIPT_RUN stage can cancel on the Web UI
• commands are executed in the dir for current application

[ffjlabo]

refer the implementation
https://github.com/pipe-cd/pipecd/blob/master/pkg/app/piped/executor/customsync/customsync.go#L55

[ffjlabo]

📝 Share my current opinion when to rollback :)

Behavior during rollback of SCRIPT_RUN

• Execute the command to rollback SCRIPT_RUN to the point where the deployment was canceled or failed.
• When there are multiple SCRIPT_RUN stages to be rolled back, they are executed in the same order as SCRIPT_RUN on the pipeline.

For example, consider when deployment proceeds in the following order from 1 to 7.

1. K8S_CANARY_ROLLOUT
2. WAIT
3. SCRIPT_RUN
4. K8S_PRIMARY_ROLLOUT
5. SCRIPT_RUN
6. K8S_CANARY_CLEAN
7. SCRIPT_RUN

Then

• If 4 is canceled or fails while running, only SCRIPT_RUN of 3 will be rolled back.
• If 6 is canceled or fails while running, only SCRIPT_RUNs 3 and 5 will be rolled back.

---

How to realize it

We can choose 2 ways to realize the rollback.

1. execute command on ROLLBACK stage
2. execute command on another stage e.g. ROLLBACK_SCRIPTRUN

[khanhtc1202]

@ffjlabo I want to vote for (2) We're going to have a plugin for each platform, in such case, each platform ROLLBACK stage implementation would be run/implemented by the plugin author. At the time the plugin is implemented, there would be no way for them to notice that the ROLLBACK stage has to be called the ROLLBACK_SCRIPTRUN. For simplicity, ROLLBACK stage for the platform should only focus on ROLLBACK for platform, ROLLBACK_SCRIPTRUN should be implemented by SCRIPTRUN executor.

[khanhtc1202]

Another way to think is, if we treat this SCRIPT_RUN as a plugin for piped, then it should provide its ROLLBACK, not rely on other stages/platform ROLLBACK.

[ffjlabo]

Thank you for your opinion :) I got it!
I will try it 👍 At first I will consider how to execute the rollback for each stage because currently, the rollback is for each application kind.

[ffjlabo]

Currently, k8s only. I will apply it to other kind of apps.

[ffjlabo]

📝 It might be nice to have the option to ignore the failure on the stage.

[ffjlabo]

for ECS app request: #4813

[khanhtc1202]

Related issue: #4814
Context: Users want to use deployment information in script which be run in SCRIPT_RUN stage execution