A mechanism to remotely obtain runtime parameters of TaskRun and PipelineRun

[l-qing]

Feature request

A mechanism to remotely obtain runtime parameters of TaskRun and PipelineRun.

Since executing a PipelineRun involves configuring many parameters, such as workspace and runtime container resources, it can be very time-consuming to reconfigure these mostly similar parameters every time.
If I could reuse the runtime parameters of an existing PipelineRun, or even modify them as needed to execute a new pipeline, that would be ideal.

This is a more advanced version of another request:

• Allow modify the spec of a PipelineRun when it is in the PipelineRunPending state #8150

But this involves more changes to the current mechanism.

Use case

1. I predefine some sets of execution parameters for a certain Pipeline.
   • For example, only execution parameters params are configured in pr-spec-params
   • For example, only workspace related parameters are configured in pr-spec-workspaces
   • For example, only allowed container quotas are configured in pr-spec-resources

apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: pr-spec-params
spec:
  status: PipelineRunPending
  pipelineRef:
    name: pipeline
  params:
    - name: key-1
      value: value-1
---

apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: pr-spec-workspaces
spec:
  status: PipelineRunPending
  pipelineRef:
    name: pipeline
  workspaces:
  - configMap:
      name: docker-config
    name: config

---
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: pr-spec-resources
spec:
  status: PipelineRunPending
  pipelineRef:
    name: pipeline
  taskRunSpecs:
    - pipelineTaskName: echo
      stepSpecs:
        - computeResources:
            limits:
              cpu: "8"
              memory: 10Gi
            requests:
              cpu: "8"
              memory: 10Gi
          name: echo

2. When I execute the Pipeline, I can choose one or more predefined sets of parameters.
   • For example, both pr-spec-params and pr-spec-workspaces can be configured simultaneously with params and workspace related parameters

apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: pr-spec-params
spec:
  pipelineRef:
    name: pipeline

  pipelineRunRef:
    resolver: run-resolver
    params:
      - name: kind
        value: pipelinerun
      - name: name
        value: pr-spec-params, pr-spec-workspaces
      - name: namespace
        value: default

3. The final PipelineRun configuration at execution looks like this: (since spec fields is not allowed to change, the actual run configuration should be stored in status?)

apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: pr-spec-params
spec:
  pipelineRef:
    name: pipeline

  params:
    - name: key-1
      value: value-1

  workspaces:
    - configMap:
        name: docker-config
      name: config

[vdemeester]

@l-qing could this be achieved with with a kustomize based resolver ?

[l-qing]

@l-qing could this be achieved with with a kustomize based resolver ?

I think it's possible.

This is essentially similar to using kustomize to merge multiple yaml resources.

The question is whether the resolver logic needs to be implemented for each case individually (pac), or if Tekton provides such a resolver capability that can be reused in other scenarios.

[vdemeester]

The question is whether the resolver logic needs to be implemented for each case individually (pac), or if Tekton provides such a resolver capability that can be reused in other scenarios.

My take on this would be, let's experiment with a kustomize-based resolver (somewhere else), and if we feel it make sense and should be built-in, let's adopt it in tektoncd 👼🏼. It probably make sense to be supported built-in, but I don't want to commit (support) on it early though.

[l-qing]

@vdemeester A significant issue is that once a PipelineRun is created, modifications the spec fields are not allowed. If it's an external mechanism, this problem cannot be avoided.

So I raised another issue, a feature gate that allows modifying the PipelineRun's spec under certain conditions.

• Allow modify the spec of a PipelineRun when it is in the PipelineRunPending state #8150