ci: azure-ipam patch upgrade testing #3678
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR introduces patch upgrade testing for Azure Container Networking by adding upgrade-specific pipeline parameters and stages to mirror the existing Cilium patch upgrade flow.
- Added new parameters (upgradeScenario, upgradeAzureIpam) and conditions to support upgrade testing.
- Updated multiple pipeline and load test templates to conditionally publish logs and deploy upgrade-specific jobs.
- Extended the cilium overlay load test template with upgrade logic for IPAM and CNS versions.
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| .pipelines/templates/log-template.yaml | Added conditional artifact publishing based on logType. |
| .pipelines/cni/pipeline.yaml | Introduced new upgradeScenario parameter and upgrade job stages. |
| .pipelines/cni/load-test-templates/restart-node-template.yaml | Updated logType condition to support upgrade scenarios. |
| .pipelines/cni/load-test-templates/restart-cns-template.yaml | Updated logType condition to support upgrade scenarios. |
| .pipelines/cni/load-test-templates/pod-deployment-template.yaml | Added conditional logType assignment for pod deployment testing. |
| .pipelines/cni/cilium/cilium-overlay-load-test-template.yaml | Extended upgrade logic with conditional deployment steps for IPAM/CNS. |
| @@ -60,7 +63,10 @@ stages: | |||
| pool: | |||
| name: "$(BUILD_POOL_NAME_DEFAULT)" | |||
| dependsOn: | |||
| - create_${{ parameters.name }} | |||
| - ${{ if and(eq(parameters.upgradeAzureIpam, ''), eq(parameters.upgradeScenario, false)) }}: | |||
There was a problem hiding this comment.
Is this the only condition where we depend on cluster creation ?
what if parameters.upgradeScenario is "" ? Can we add a table of the combination/expectation of these variables in the description ?
There was a problem hiding this comment.
for this PR, yes. eventually when we have cns or cni patch upgrades, we will need to skip the cluster create there as well.
| echo "TEST_CNS_VERSION is not set, using default value" | ||
| CNS=$(make cns-version) | ||
| echo "Upgrade scenario is true, using upgrade azure ipam and cns version from pipeline variables" | ||
| IPAM=$(UPGRADE_AZURE_IPAM_VERSION) |
There was a problem hiding this comment.
Even if upgradeAzureIpam is false, will we use the UPGRADE_AZURE_IPAM_VERSION ?
There was a problem hiding this comment.
oh, will need to add in another condition to check whether or not UPGRADE_AZURE_IPAM_VERSION is set because even if upgradeScenario is true we may only be testing the upgrade for cns and not azure-ipam
|
|
||
|
|
||
| ## If upgradeScenario is set, redeploy new IPAM version to existing clusters and run tests | ||
| - ${{if eq(parameters.upgradeScenario, true)}}: |
There was a problem hiding this comment.
if this is performing the same e2e test as above, would it be possible to make a template that performs the upgrade test where we pass all these parameters and the template automatically creates the before and after upgrade stages? The template could potentially also take in the upgrade scenario as true or false or a pipeline variable/expression for enable/disable. If we add more scenarios in the future I feel like it would be easy to forget to add the upgrade stage
There was a problem hiding this comment.
will keep this in mind to address in a follow up
| nodeCount: ${{ parameters.nodeCount }} | ||
| vmSize: ${{ parameters.vmSize }} | ||
| region: $(location) | ||
| - ${{if eq(parameters.upgradeScenario, false)}}: |
There was a problem hiding this comment.
If upgradeScenario is false we are not creating cluster ? the description says we need to create the cluster for upgrade scenario ?
There was a problem hiding this comment.
if upgradeScenario is false we create the cluster, if upgradeScenario is true we skip this since the cluster is already created
There was a problem hiding this comment.
in the pipeline.yaml I am calling this template and passing the parameter upgradeScenario.
- template: cilium/cilium-overlay-load-test-template.yaml
for each scenario there are two calls to the template, the first contains upgradeScenario: false, and the next contains upgradeScenario: true
Reason for Change:
First pr for patch upgrade testing in CNI release pipeline. The CNI release patch upgrade tests will follow the same flow that we use for cilium patch upgrade tests.
Create cluster > deploy current images > e2e > deploy upgrade images > e2e > delete cluster
Adding TEST_AZURE_IPAM_VERSION/UPGRADE_AZURE_IPAM_VERSION and TEST_CNS_VERSION/UPGRADE_CNS_VERSION and new conditions to check these vars in the upgrade scenario.
(CNS patch upgrade work still needs to be completed for non cilium)
Adding parameter upgradeScenario as a field to check off in pipeline run to determine if new patch needs to be deployed/tested
further documentation/instruction for use will be added when pipeline work is complete, but here is a quick guide for how values are expected to be set
upgradeScenario[T/F] - true will deploy patches and retest, false will run pipeline as usualTEST_AZURE_IPAM_VERSION: set to current azure ipam patchUPGRADE_AZURE_IPAM_VERSION: set to upgrade azure ipam patchTEST_CNS_VERSION: set to current cns patchUPGRADE_CNS_VERSION: set to upgrade cns patchif the
TEST_* orUPGRADE_* values are not set it will use make to get the pipeline image buildIssue Fixed:
Requirements:
Notes: