Test Fleet in Rancher on self-hosted runner #2804
Commits on Sep 26, 2024
-
Fix Fleet upgrade script to install Fleet in Rancher
This is the basis for being able to test any given Fleet commit in Rancher, installing the latter through Helm.
-
This simplifies Rancher installation, preventing a costly local build and simply making use of existing configuration options to override the Fleet version to install from a custom repository and branch. The Rancher Docker image to use is hard-coded for now.
-
Run Fleet-in-Rancher test workflow on self-hosted runner
This is an attempt to use Rancher's org-wide hosted runners to test Fleet within Rancher, instead of dealing with GCP.
-
This may help prevent unknown authority errors when installing the Ginkgo CLI.
-
Use bci/bci-base:15.6 base image
That image comes with CA certificates, curl and tar installed.
-
Images are already built when releasing Fleet charts against a test charts repository.
-
This makes use of a dedicated step to install remaining dependencies.
-
Run Fleet-in-Rancher tests against Ubuntu
Do we really need a dedicated VM or self-hosted runner for this? Reusing the same setup as Fleet's multi-cluster tests to verify it.
-
Installing Rancher should directly take care of installing our test Fleet version.
-
Install kubectl and other dependencies into ~/.local/bin
This should fix permissions issues.
-
This prevents issues about package `helm` not being found in Ubuntu repositories.
-
Add WIP script for testing Fleet in Rancher
This could be reused in CI, after a few improvements.
-
Use non-containerized self-hosted runner
`rancher/fleet` is now approved to use such runners, with help from EIO.
-
Add rancher bin path to $PATH where needed
Registering downstream clusters with Rancher requires the `rancher` CLI.
-
Print shell commands when setting up Rancher
This should help troubleshoot failures with `Process completed with exit code 1`.
-
Grant cluster-admin privilege to downstream cluster admin
This enables downstream cluster registration to succeed, as specified in the official Rancher docs [1]. [1]: https://ranchermanager.docs.rancher.com/v2.0-v2.4/how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/import-existing-clusters#prerequisites
-
Compute public IP instead of hard-coding it
This may eliminate errors with downstream cluster registration.
-
Enable direct Fleet-in-Rancher workflow call
This eases troubleshooting and enables testing Fleet against existing Rancher releases.
-
This is not needed when installing Rancher from Helm instead of building it.
-
Replace cert-manager CRDs install parameter
Old parameter `installCRDs` is deprecated.
-
Require a non-empty command to import a cluster
The Rancher CLI may output an empty command for a bit, leading `register-downstream-clusters.sh` to fail. Instead of trying to reverse engineer why that might be, we simply run `rancher cluster import` repeatedly until the returned command is non-empty.
-
Restore Tmate step for troubleshooting
This could help us understand why downstream cluster registration fails in CI although it works locally.
-
Fix downstream cluster registration
When installing Rancher through Helm: * the `CATTLE_SERVER_URL` needs to be set to the same value as `hostname` * TLS mode must be set to `system-store`, to prevent cert-related errors when running a Fleet agent in a downstream cluster
-
Ensure API server IP is available before setting it
This waits for the upstream cluster to be ready, preventing an empty IP from being set.
-
Environment variables are not necessary, as a dedicated Helm value exists.
-
Document testing Fleet in Rancher
This moves `test-in-rancher` to the `dev` scripts directory, and briefly explains how to use it.
-
Restore consistency in job blocks layout
A newline follows the initial `-`, as in other workflows.
-
Rename Fleet-in-Rancher test script
The script does not actually test anything, and is now named consistently with other scripts living in the `dev/` folder.
-
Remove cluster setup from Fleet-in-Rancher dev script
The script would not manage a cluster's lifecycle beyond its creation anyway, and doing so would be harder to automate. It is therefore left out of the script's scope.
-
Create new workflow for testing Fleet in Rancher
This leaves the original workflow in place, upgrading Fleet in Rancher.
-
That block is not necessary, as tests are run against clusters afterwards.
-
Add acceptance tests to Fleet-in-Rancher test workflow
This ensures that Fleet examples are validated against Fleet in Rancher.
-
That check is only relevant when using a custom charts branch.
