Helm Chart should allow annotations to be added to Service Accounts

[avo-sepp]

Checklist

• I've searched the issue queue to verify this is not a duplicate feature request.
• I've pasted the output of kargo version, if applicable.
• I've pasted logs, if applicable.

Proposed Feature

The Helm chart does not have a values section for adding annotations to the service accounts. This would be a nice to have feature, for example when integrating with Azure Workload Identities or EKS IRSA/Pod Identities.

I may need different service account annotations. As I may want the Dex server to use one service account, but the controller to use a different one.

Motivation

I need to add annotations for Tenant and Client IDs so I can apply Azure Workload Identity logins to the applications.

Suggested Implementation

With each app section, like api, controller, or oidc, I'd like an additional annotations section allowing me to annotate the service account tied with that application.

Alternative Approach

I can post-render the Helm chart with Kustomize and achieve the same goal. It's just more complicated for implementors to do.

[krancour]

This could be generally useful, so it's something we could consider.

In the meantime, there is some pretty rich support for some of what you're asking about baked right into Kargo.

This would be a nice to have feature, for example when integrating with Azure Workload Identities or EKS IRSA/Pod Identities.

I assume this would be mainly to access ACR (Azure) or ECR (Amazon)? Or GAR (Google) for that matter?

At startup, Kargo can detect if it's running in EKS with either Pod Identity or IRSA configured. If it is, it will automatically try to use it when looking for ECR credentials. Quite similarly it can detect if it's running in GKE with Workload Identity Federation configured. If it is, it will automatically try to use it when looking for GAR credentials.

With EKS and IRSA/Pod Identity, Kargo will even attempt to use the SA's credentials to assume a Project-specific IAM role. In this way, each Project's ECS can be limited to relevant repositories. Managing an IAM role for every Project can certainly be a bit of an undertaking, so if it cannot find a Project-specific role to assume, it will fall back on directly using the SA's role for all ECR access -- which is probably all you wanted to do in the first place, but it's just nice that this option for Project-specific roles exists.

Things work similarly for GKE + GAR, although it only uses the SA's credentials to impersonate a Project-specific Google SAs. It doesn't ever fall back on using the controller's Google SA to access GAR directly, but only because no one's implemented it yet.

Either of these can be setup without any need to annotate your Service Accounts and it's well-documented, too:

https://docs.kargo.io/how-to-guides/managing-credentials#eks-pod-identity-or-iam-roles-for-service-accounts-irsa

I need to add annotations for Tenant and Client IDs so I can apply Azure Workload Identity logins to the applications.

Things are less rosy with Azure + ACR because they don't have anything along the lines of assume role or account impersonation, so Project-specific Azure Managed Identities with narrow AKS access won't be possible (without jumping through an absurd number of hoops), but we'll at least eventually add the option to use the controller's Azure Managed Identity to access AKS directly. iirc, in Azure's case, there's no getting around annotating the controller's SA, so in this case, the requested feature would indeed come in handy.

As I may want the Dex server to use one service account

What Azure/AWS/GCP services do you anticipate Dex accessing? Most users do not even need Dex at all because most IDPs, including Azure EntraID and AWS IAM Identity Center support PKCE, which means "public clients" (the Kargo UI and CLI) can safely authenticate users with the IDP directly. Dex is really needed only when an IDP doesn't support OIDC + PKCE.

With each app section, like api, controller, or oidc

OIDC's covered in my previous paragraph, but the API server is another component that won't benefit from any sort of workload identity -- the API server doesn't communicate with any external services. It only talks to the Kubernetes cluster's own API server.

To re-iterate, none of this is to say that we cannot entertain the notion of exposing options to configure annotations on ServiceAccounts. I just wanted to make you aware of features and factors that may lessen or eliminate your need to add those annotations in the first place.