Cluster provisioning logs entries not logged on UI

[cpinjani]

Rancher Server Setup
Rancher version: 2.7-head - fbca7c3
Installation option: Helm Chart
If Helm Chart, Kubernetes Cluster and version: EKS 1.25

Information about the Cluster
Kubernetes version: EKS with Kubernetes 1.25
Cluster Type (Downstream): Hosted = EKS

User Information
What is the role of the user logged in? (Admin/Cluster Owner/Cluster Member/Project Owner/Project Member/Custom) - Admin

Describe the bug
Cluster provisioning logs entries not logged on UI
Issue exists for other Hosted providers also.

To Reproduce
Add valid AWS cloud credentials
Provision downstream EKS cluster
Check Cluster > Provisioning Log tab

Result
User unable to check provisioning logs

Screenshots

[furkatgofurov7]

I started looking into this one and so far here are my initial findings:

1. I can repro it and provisioning logs are not visible in GKE at least and should probably be the case in other hosted providers (AKS/EKS)
2. Provisioning logs used to be added to the last line of cluster object and UI read it from there in the past, but there was a backend change so now logs should be written to the configMap and UI has to read the logs from cm. UI part looks to be adapted to that backend change in Cluster provisioning log dashboard#4605, however, there is a chance it needs more changes in case of hosted providers
3. Other assumption would be the Rancher core itself, where we need to enable or expose those logs from when provisioning Hosted providers

[mjura]

Provisioning logs in Rancher are generated by "github.com/rancher/rancher/pkg/clusterprovisioninglogger"
https://github.com/rancher/rancher/blob/release/v2.8/pkg/controllers/management/clusterprovisioner/driver.go#L22
but it is skipped for Hosted Providers KEv2
https://github.com/rancher/rancher/blob/release/v2.8/pkg/controllers/management/clusterprovisioner/provisioner.go#L115

@mattfarina Question is if this should fixed on Rancher site or we should add logging service to Hosted Providers ?

[mjura]

This problem has to be discussed with Rancher Team

[mattfarina]

@mjura I think you're going to need to add the logging to the hosted providers. The code you link to is the provisioner. It essentially says to skip provisioning handling for those cases. In those cases the hosted provider controllers would do the work. They will need to supply the logs, if I understand it correctly.

@jakefhyde Does this look right?

[jakefhyde]

@mjura I think you're going to need to add the logging to the hosted providers. The code you link to is the provisioner. It essentially says to skip provisioning handling for those cases. In those cases the hosted provider controllers would do the work. They will need to supply the logs, if I understand it correctly.

@jakefhyde Does this look right?

Yes, this looks correct, it will be the responsibility of the hosted provider controllers to populate the provisioning-log configmap.

[gaktive]

@salasberryfin & @yiannistri to review this ticket and figure out what to do next. Azure is the 1st place to look but this appears to be the case across multiple platforms.

[mjura]

@salasberryfin & @yiannistri to review this ticket and figure out what to do next. Azure is the 1st place to look but this appears
to be the case across multiple platforms.

In my opinion it is design decision related to what we have currently with
rancher/aks-operator#179 (comment)