Using podSpecPatch with templateReferencing: Secure mode can still override container image and command

[michaelncy]

Pre-requisites

• I have double-checked my configuration
• I have tested with the :latest image tag (i.e. quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on :latest. If not, I have explained why, in detail, in my description below.
• I have searched existing issues and could not find a match for this bug
• I'd like to contribute the fix myself (see contributing guide)

What happened? What did you expect to happen?

We are using templateReferencing: Secure configuration in our argo workflows in order to limit the calls to workflowTemplates only.
We have discovered that when using the podSpecPatch field and overriding the container image and commands you can practically override this setting to execute any arbitrary workflow you want.
Also, the UI shows that the container image and command are the original ones, although in reality it runs the overridden image and commands

Version(s)

v3.6.0-rc1

Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: workflow-template-hello-world-
spec:
  workflowTemplateRef:
    name: workflow-template-submittable
  # No matter what image you had in the template it gets overriden
  podSpecPatch: |
    containers:
      - name: main
        image: busybox
        command: [echo]
        args: ["hello world"]

Logs from the workflow controller

time="2024-11-06T13:01:59.480Z" level=info msg="Created pod: wonderful-poochenheimer-bpbjt (wonderful-poochenheimer-bpbjt)" namespace=test workflow=wonderful-poochenheimer-bpbjt
time="2024-11-06T13:01:59.480Z" level=info msg="TaskSet Reconciliation" namespace=test workflow=wonderful-poochenheimer-bpbjt
time="2024-11-06T13:01:59.481Z" level=info msg=reconcileAgentPod namespace=test workflow=wonderful-poochenheimer-bpbjt
time="2024-11-06T13:01:59.503Z" level=info msg="Workflow update successful" namespace=test phase=Running resourceVersion=37261467 workflow=wonderful-poochenheimer-bpbjt
time="2024-11-06T13:02:09.485Z" level=info msg="Processing workflow" Phase=Running ResourceVersion=37261467 namespace=test workflow=wonderful-poochenheimer-bpbjt
time="2024-11-06T13:02:09.486Z" level=info msg="Task-result reconciliation" namespace=test numObjs=1 workflow=wonderful-poochenheimer-bpbjt
time="2024-11-06T13:02:09.486Z" level=info msg="node changed" namespace=test new.message= new.phase=Succeeded new.progress=0/1 nodeID=wonderful-poochenheimer-bpbjt old.message= old.phase=Pending old.progress=0/1 workflow=wonderful-poochenheimer-bpbjt
time="2024-11-06T13:02:09.487Z" level=info msg="TaskSet Reconciliation" namespace=test workflow=wonderful-poochenheimer-bpbjt
time="2024-11-06T13:02:09.487Z" level=info msg=reconcileAgentPod namespace=test workflow=wonderful-poochenheimer-bpbjt
time="2024-11-06T13:02:09.487Z" level=info msg="Updated phase Running -> Succeeded" namespace=test workflow=wonderful-poochenheimer-bpbjt
time="2024-11-06T13:02:09.487Z" level=info msg="Marking workflow completed" namespace=test workflow=wonderful-poochenheimer-bpbjt

Logs from in your workflow's wait container

kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded