AAW-PROD Grafana oauth failed, users cannot login

[vexingly]

https://grafana.aaw.cloud.statcan.ca/

Sign-in with Microsoft (OAuth)

Error message: login.OAuthLogin(NewTransportWithCode)

Maybe the OAuth cert/creds are expired?

[Souheil-Yazji]

Oauth secret is indeed not expired, grafana oauth secret expires in 2026

[vexingly]

Oauth secret is indeed not expired, grafana oauth secret expires in 2026

The grafana log indicates the secret is incorrect:
error="oauth2: cannot fetch token: 401 Unauthorized\nResponse: {\"error\":\"invalid_client\",\"error_description\":\"AADSTS7000215: Invalid client secret provided.

I verified the secrete in the prometheus-system ns is being injected into the grafana pod correctly, and on review of the aaw-grafana-sp application in the azure portal, it appears that the secrets listed there are not the one configured in the ns secret...

Not sure what has happened but I think a new secret will be needed.