Skip to content

azure.json cannot be parsed when working with workload identity #33882

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
corax15 opened this issue May 26, 2025 · 5 comments · May be fixed by #33951
Open

azure.json cannot be parsed when working with workload identity #33882

corax15 opened this issue May 26, 2025 · 5 comments · May be fixed by #33951
Assignees
Labels
external-dns tech-issues The user has a technical issue about an application triage Triage is needed

Comments

@corax15
Copy link

corax15 commented May 26, 2025

Name and Version

bitnami/external-dns 8.8.3

What architecture are you using?

Azure Kubernetes Services

What steps will reproduce the bug?

We are using external-dns togehter with Azure DNS. external-dns authenticates using Azure workload identity.
After updating from 8.8.2 to 8.8.3 the external-dns pod crashes during startup with this error message:

failed to read Azure config file '/etc/kubernetes/azure.json': failed to parse Azure config file '/etc/kubernetes/azure.json': invalid character '}' looking for beginning of object key string

Are you using any custom parameters or values?

We use the following parameters with the helm chart, where useWorkloadIdentityExtension is causing the issue.

azure:
  useWorkloadIdentityExtension: true
  resourceGroup: rg-name
  subscriptionId: our-subscription-id
  tenantId: our-tenant-id

This configuration results in the following JSON file stored in the chart's Kubernetes secret:

{
  "tenantId": "our-tenant-id",
  "subscriptionId": "our-subscription-id",
  "resourceGroup": "rg-name",
  "useWorkloadIdentityExtension":  true,
}

Note the comma at the end of the last line. This is causing the issue. If I remove it and recreate the secret, the pod starts successfully.

What is the expected behavior?

With 8.8.3, using workload identity with Azure DNS should not cause the external-dns pod to crash.

Additional information

The _helpers.tpl defines the variable "external-dns.azure-credentials" which always adds "useWorkloadIdentityExtension": true, when using workload identity, no matter if some more values are following.

Could a solution be to either add the line with or without a trailing comma, depending on whether the managed identity extension values are set afterwards? Is there even a scenario in which workload identity and managed identity can be used together?

@corax15 corax15 added the tech-issues The user has a technical issue about an application label May 26, 2025
@github-actions github-actions bot added the triage Triage is needed label May 26, 2025
@carrodher
Copy link
Member

Thank you for bringing this issue to our attention. We appreciate your involvement! If you're interested in contributing a solution, we welcome you to create a pull request. The Bitnami team is excited to review your submission and offer feedback. You can find the contributing guidelines here.

Your contribution will greatly benefit the community. Feel free to reach out if you have any questions or need assistance.

@pinkfloydx33
Copy link

pinkfloydx33 commented May 27, 2025

We are hitting this as well with the upgrade to 8.8.3. I am not certain why this is suddenly an issue as the diffs on artifacthub don't show anything that would cause it. It seems to be a problem for both azure and azure-private-dns`

@MaxAnderson95
Copy link
Contributor

Also seeing this issue. I noticed that while the chart bump was just a patch increase 8.8.2 to 8.8.3, this actually bumped the underlying version of external-dns from 0.16.1 to 0.17.0. You'd think this would constitute an equal bump in the chart version to 8.9.0 but I guess not?

I suspect there was some change to the code which parses the JSON file, and that prior to 0.17.0 this was incorrectly parsing this file with a trailing comma as valid JSON. Looking at another cluster still running 0.16.1, I can also see the file has a trailing comma, and isn't causing this issue. The fix to me seems to be that it should not generate with a trailing comma.

Looking at the _helpers.tpl function which generates the contents of the secret, it looks to be generating a raw JSON string, which is probably not the best way to do that.

I will open a PR soon to convert this function to generate a native templating dictionary, and then convert that to JSON, so that the values can be in any order, and we don't need to worry about trailing commas.

A rough POC of what I'm thinking:

{{- define "external-dns.azure-credentials" -}}
{{- $credentials := dict -}}
{{- if .Values.azure.cloud -}}
{{- $_ := set $credentials "cloud" .Values.azure.cloud -}}
{{- end -}}
{{- if .Values.azure.tenantId -}}
{{- $_ := set $credentials "tenantId" .Values.azure.tenantId -}}
{{- end -}}
{{- if .Values.azure.subscriptionId -}}
{{- $_ := set $credentials "subscriptionId" .Values.azure.subscriptionId -}}
{{- end -}}
{{- $_ := set $credentials "resourceGroup" .Values.azure.resourceGroup -}}
{{- if not (or .Values.azure.useManagedIdentityExtension .Values.azure.useWorkloadIdentityExtension) -}}
{{- $_ := set $credentials "aadClientId" .Values.azure.aadClientId -}}
{{- $_ := set $credentials "aadClientSecret" .Values.azure.aadClientSecret -}}
{{- end -}}
{{- if .Values.azure.useWorkloadIdentityExtension -}}
{{- $_ := set $credentials "useWorkloadIdentityExtension" true -}}
{{- end -}}
{{- if and .Values.azure.useManagedIdentityExtension .Values.azure.userAssignedIdentityID -}}
{{- $_ := set $credentials "useManagedIdentityExtension" true -}}
{{- $_ := set $credentials "userAssignedIdentityID" .Values.azure.userAssignedIdentityID -}}
{{- else if and .Values.azure.useManagedIdentityExtension (not .Values.azure.userAssignedIdentityID) -}}
{{- $_ := set $credentials "useManagedIdentityExtension" true -}}
{{- end -}}
{{- $credentials | toJsonPretty -}}
{{- end -}}

@pinkfloydx33
Copy link

That's kind of what I was thinking. I'm wondering though if you can just

{{ omit .Values.azure "secretName" | toJsonPretty }} 

Since all the values of the azure object are one-to-one mappings with that file except for the secretName. I think after all that logic is performed you literally just end up with a clone of the object. You probably wouldn't set any of the values if you didn't expect them in the config file

@MaxAnderson95
Copy link
Contributor

That's kind of what I was thinking. I'm wondering though if you can just

{{ omit .Values.azure "secretName" | toJsonPretty }} 

Since all the values of the azure object are one-to-one mappings with that file except for the secretName. I think after all that logic is performed you literally just end up with a clone of the object. You probably wouldn't set any of the values if you didn't expect them in the config file

Yes that's a much more elegant solution! I should have a PR submitted by tonight.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
external-dns tech-issues The user has a technical issue about an application triage Triage is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants