Merging AUTH0_KEYWORD_REPLACE_MAPPINGS from env and config file

[mutefiRe]

Checklist

• I agree to the terms within the Auth0 Code of Conduct.

Describe the problem you'd like to have solved

Like recommended, we are using multiple config.json files, say one per environment and having a lot of variables in the AUTH0_KEYWORD_REPLACE_MAPPINGS which are not secret but should be version controlled. On the other hand we have multiple secrets (like aws, google) which need to be injected into the mapping as well, which are not version controlled and come from the CI.

So I'm not sure how to best still define the keyword replace mapping with all the hardcoded values per environment but still be able to inject secrets without doing some additional scripting around this.

Describe the ideal solution

One solution would be to merge the value coming from the ENV and from the config file and prioritize the ENV.

Alternatives and current workarounds

Replacement of ENV variables in the config.json from the environment.

Current workarounds:

• define everything in the environment variable (e.g. with bash substitition)
• script around it to merge/replace

Additional context

If this is already somehow possible without scripting it would be nice to see a guide in the docs.

[Marchelune]

Hi there!
We are in a similar situation, it would be nice to have a workflow to manage secrets - such as email providers API key.

As @mutefiRe rightly highlights, it's possible with some custom scripting, but perhaps it could be integrated in the CLI, to reduce the tooling maintenance.

[Tzinov15]

I'm not sure if this will exactly solve your problems, but we had to do something similar and somewhat out of chance stumbled upon a solution that works for us. We use a single config.json file and inside it actually have empty entries for our entire set of swapped in template variables. It looks something like this

  "AUTH0_KEYWORD_REPLACE_MAPPINGS": {
    "TENANT_FRIENDLY_NAME": "",
    "TWILIO_FROM_NUMBER": "",
    "TWILIO_SID": "",
    "TWILIO_TOKEN": "",
    "NAMESPACE": "",
    "DATADOG_API_KEY": "",
   ...etc

Then we inject all of these values during our CI deployment (Github Actions) as environment variables, some of which come from Environment Variables, and some which come from Environment Secrets. It seems like merely defining the values in the config mapping, even if they're empty, allows the Deploy CLI to pick them up and anticipate them being present in the env as a fallback

[philippefutureboy]

I have a similar issue, and worked wayyy to long for what this is, trying to find an optimal solution to this.
The main issue was to keep the secrets synchronized with the version of the tenant.yaml so that we can have a history of (tenant, secrets) in case something breaks after a deployment.

Since the Auth0 servers are ultimately the source of truth, and since it's not possible for two developers to work on the same Auth0 tenant at the same time, we decided that having the tenant.yaml in svc and a simple versioning on the secrets was sufficient.
We decided not to track each tenant's config.json, and instead we push each config.json to Google Cloud Secrets Manager, which is later on pulled in CI to be applied against the tenant alongside the tenant.yaml.

Alternatives we considered:

• Have a config.yaml.j2 (jinja2) template that is kept in svc, and generate a config.yaml not under svc using an env file, which itself is pulled from Google Cloud Secrets Manager
• Upload each env var as a separate secret in Google Cloud Secrets Manager, recompose the env file from all secrets tagged with auth0 and the tenant (allows better tracking of secrets which are disabled, compromised, etc)
• Associate specific secret versions with a hash(tenant.yaml) to avoid loading secret versions that are out of date from the tenant.yaml; but the issue is that tenant.yaml is not stable - Auth0 creates resources every once in a while without our knowledge because their product changes, and does not respect keyword mapping (directly embeds some values) in tenant.yaml, which prevents tracking of the hash of the tenant.yaml; so ultimately after trying to write tooling for all of this, we dropped the idea.

hopefully this writeout can be of help.