PodTemplate param substitution on TaskRunSpecs to enable multi-arch builds with Matrix

[dorzel]

Changes

This PR originally aimed to enable simpler multi-arch builds with Matrix by adding PodTemplate to Tasks (originally prompted by #6742). After some discussion, it now aims to do the same thing by adding param substitution on nodeSelector fields in TaskRunSpecs in a PipelineRun.

The current changes do work to accomplish this goal, with an example PipelineRun as below working:

apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  generateName: matrix-podtemplate-taskrunspecs-test-
spec:
  taskRunSpecs:
  - pipelineTaskName: build-and-push-manifest
    podTemplate:
      nodeSelector:
        kubernetes.io/arch: $(params.arch)
  - pipelineTaskName: create-manifest-list
    podTemplate:
      nodeSelector:
        kubernetes.io/arch: amd64
  pipelineSpec:
    tasks:
      - name: build-and-push-manifest
        matrix:
          params:
          - name: arch
            value: ["amd64", "arm64"]
        taskSpec:
          results:
            - name: manifest
              type: string
          params:
            - name: arch
          steps:
            - name: build-and-push
              image: ubuntu
              script: |
                echo "building on $(params.arch)"
                echo "testmanifest-$(params.arch)" | tee $(results.manifest.path)
      - name: create-manifest-list
        params:
          - name: manifest
            value: $(tasks.build-and-push-manifest.results.manifest[*])
        taskSpec:
          steps:
            - name: echo-manifests
              image: ubuntu
              args: ["$(params.manifest[*])"]
              script: echo "$@"

Feedback appreciated, thanks!

Fixes #6742

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

• Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
• Has Tests included if any functionality added or changed
• pre-commit Passed
• Follows the commit message standard
• Meets the Tekton contributor standards (including functionality, content, code)
• Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
• Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings). See some examples of good release notes.
• Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

TaskRuns and TaskRunSpecs now support param substitution on podTemplate fields. This allows operability with Matrix.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: dorzel / name: Dylan Orzel (f44cec6, 8ea3bd8, c22bec2)

[afrittoli]

/kind feature

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/pod/pod.go	93.3%	92.9%	-0.3
pkg/reconciler/taskrun/resources/apply.go	99.4%	98.7%	-0.6

[jeffdyoung]

cc: @aleskandro @Prashanth684 @arewm @adambkaplan

[waveywaves]

/assign

[waveywaves]

@dorzel thank you for opening the draft PR, not sure if a TEP is required as the issue #6742 mentioned doesn't mention a requirement for one. But I will defer to the rest of the team for views on this.

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/taskrun/resources/apply.go	99.4%	64.7%	-34.7

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/taskrun/resources/apply.go	99.4%	99.6%	0.3

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/taskrun/resources/apply.go	99.4%	99.6%	0.3

[dorzel]

Ok, I pushed up some updates to this for tests as well as param substitution on all podTemplate fields (at least where it made sense. Bool and Int fields I have left out). Not sure if I covered all the needed areas for testing, let me know.

@chmouel @waveywaves Would you be able to take another look at this?

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/taskrun/resources/apply.go	99.4%	99.6%	0.3

[dorzel]

e2e tests passed except for an error with my example pipeline where it (assumedly) needed to be in the beta folder. Waiting on retest to confirm that worked

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/taskrun/resources/apply.go	99.4%	99.2%	-0.2
pkg/reconciler/taskrun/taskrun.go	85.9%	86.1%	0.3

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/taskrun/resources/apply.go	99.4%	99.2%	-0.2
pkg/reconciler/taskrun/taskrun.go	85.9%	86.1%	0.3

[waveywaves]

/retest

[dorzel]

thanks @waveywaves, that retest didn't seem to work for some reason however

[aThorp96]

/retest

[waveywaves]

/retest

[waveywaves]

/ok-to-test

[afrittoli]

/approve

[tekton-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: afrittoli, vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [afrittoli,vdemeester]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[afrittoli]

@waveywaves I'll let you lgtm since you requested changes on the PR 🙏

[vdemeester]

/lgtm