Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SURE-9542] fleet apply logs from fleet job Pod do not indicate source of "no chart name found" errors in GitRepo containing multiple bundles #3160

Open
kkaempf opened this issue Dec 19, 2024 · 0 comments
Labels
Milestone

Comments

@kkaempf
Copy link
Collaborator

kkaempf commented Dec 19, 2024

SURE-9542

Issue description:

fleet apply logs from fleet job Pod do not indicate source of "no chart name found" errors in GitRepo containing multiple bundles. If one (or more) charts defined by a fleet.yaml bundle in the GitRepo do no exist in the index.yaml for their configured repository, only a generic level=fatal msg="no chart name found" is logged in the Job Pod, with no indication of which chart/bundle the error originates from.

Business impact:

Makes troubleshooting of the issue a heavy lift manual task

Repro steps:

Provision a Rancher v2.9.4 instance with a single all-role node custom RKE cluster (I used github.com/axeal/tf-do-rancher2)
In fleet-default workspace of Fleet, add a GitRepo with the reposistory https://github.com/axeal/fleet-test.git and branch 01563984
Observe successful deployment of the two charts/bundles
Update the GitRepo branch to 01563984-invalid, in which the rancher-monitoring-crd chart name is updated to the invalid rancher-monitoring-crds (use of the the rancher-logging-crd and rancher-monitoring-crd charts was arbitrary for the purpose of reproduction, in customer environment they are deploying their internal applications).
Observe after a short time that the GitRepo goes into an error state with Job Failed. failed: 3/1time="2024-12-18T13:24:13Z" level=fatal msg="no chart name found"

Observe the fleet Job pods for the GitRepo contain only the following logs:

HOSTNAME=test-ec8c6-wmczp
KUBERNETES_PORT_443_TCP_PROTO=tcp
COMMIT=8d6c3473d9d1b4afe6b1d479466226c247c6783b
KUBERNETES_PORT_443_TCP_ADDR=10.43.0.1
KUBERNETES_PORT=tcp://10.43.0.1:443
PWD=/workspace/source
HOME=/fleet-home
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP=tcp://10.43.0.1:443
SHLVL=1
KUBERNETES_SERVICE_PORT=443
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
KUBERNETES_SERVICE_HOST=10.43.0.1
_=/usr/bin/env
time="2024-12-18T13:23:51Z" level=fatal msg="no chart name found"

Enable debug logging for fleet by Upgrading the fleet app in the local cluster and setting the value debug to true
Force Update the GitRepo
Observe there are no additional log messages in the new Job pod logs for the GitRepo

Workaround:

Is a workaround available and implemented? Yes
What is the workaround: Manually investigate each fleet.yaml bundle within the GitRepo to validate whether the specified helm chart is present within the index.yaml of the defined helm repository.

Actual behavior:

fleet apply logs from fleet job Pod do not indicate source of "no chart name found" errors in GitRepo containing multiple bundles

Expected behavior:

fleet apply logs from fleet job Pod indicate the source chart/bundle of "no chart name found" errors in a GitRepo containing multiple bundles

@kkaempf kkaempf added this to the v2.11.0 milestone Dec 19, 2024
@kkaempf kkaempf added this to Fleet Dec 19, 2024
@github-project-automation github-project-automation bot moved this to 🆕 New in Fleet Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: 🆕 New
Development

No branches or pull requests

1 participant