unhealthy status after upgrade from v0.8.6 to v0.9.0

[astayleraz]

Our tests started to fail when we upgraded from v0.8.6 to v0.9.0. When we brought this up in docker, the status would go from health: starting to unhealthy soon after, and the logs did not show anything useful about what was failing.

docker logs databricks-oidc-server-mock
[17:33:17 Information] Duende.IdentityServer.Startup
Starting Duende IdentityServer version 7.0.3+1bb29dd27f53edbf25f6d5c52df6c5547caffc0f (.NET 8.0.3)
[17:33:17 Warning] Duende.IdentityServer.License
You do not have a valid license key for the Duende software. This is allowed for development and testing scenarios. If you are running in production you are required to have a licensed version. Please start a conversation with us: https://duendesoftware.com/contact
[17:33:17 Warning] Duende.IdentityServer.License
You have automatic key management enabled, but you do not have a license. This feature requires the Business or Enterprise Edition tier of license. Alternatively you can disable automatic key management by setting the KeyManagement.Enabled property to false on the IdentityServerOptions.
[17:33:17 Information] Duende.IdentityServer.Startup
You are using the in-memory version of the persisted grant store. This will store consent decisions, authorization codes, refresh and reference tokens in memory only. If you are using any of those features in production, you want to switch to a different store implementation.
[17:33:17 Information] Duende.IdentityServer.Startup
Using the default authentication scheme idsrv for IdentityServer
[17:33:17 Debug] Duende.IdentityServer.Startup
Using idsrv as default ASP.NET Core scheme for authentication
[17:33:17 Debug] Duende.IdentityServer.Startup
Using idsrv as default ASP.NET Core scheme for sign-in
[17:33:17 Debug] Duende.IdentityServer.Startup
Using idsrv as default ASP.NET Core scheme for sign-out
[17:33:17 Debug] Duende.IdentityServer.Startup
Using idsrv as default ASP.NET Core scheme for challenge
[17:33:17 Debug] Duende.IdentityServer.Startup
Using idsrv as default ASP.NET Core scheme for forbid
[17:33:17 Warning] Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository
Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed. For more information go to https://aka.ms/aspnet/dataprotectionwarning
[17:33:17 Warning] Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager
No XML encryptor configured. Key {b5424d46-1024-4432-8c5b-eae41782a956} may be persisted to storage in unencrypted form.
[17:33:17 Information] Microsoft.Hosting.Lifetime
Now listening on: http://[::]:8080
[17:33:17 Information] Microsoft.Hosting.Lifetime
Application started. Press Ctrl+C to shut down.
[17:33:17 Information] Microsoft.Hosting.Lifetime
Hosting environment: Development
[17:33:17 Information] Microsoft.Hosting.Lifetime
Content root path: /OpenIdConnectServerMock

We are unsure if we need to change something in our yml file with the upgrade. When we pin the version to 0.8.6 our tests and the instances work again. Here is the yml file we are using but now pinning to 0.8.6:

version: '3'
services:
  oidc-server-mock:
    container_name: databricks-oidc-server-mock
    image: ghcr.io/soluto/oidc-server-mock:0.8.6
    ports:
      - "3000:80"
    environment:
      ASPNETCORE_ENVIRONMENT: Development
      API_SCOPES_INLINE: | 
        - Name: all-apis
      SERVER_OPTIONS_INLINE: |
        {
          "AccessTokenJwtType": "JWT",
          "Discovery": {
            "ShowKeySet": true
          },
          "Authentication": {
            "CookieSameSiteMode": "Lax",
            "CheckSessionCookieSameSiteMode": "Lax"
          }
        }
      ACCOUNT_OPTIONS_INLINE: |
        {
          "AutomaticRedirectAfterSignOut": true
        }
      USERS_CONFIGURATION_PATH: /tmp/common/openid/users-config.json
      CLIENTS_CONFIGURATION_PATH: /tmp/common/openid/clients-config.json
      ASPNET_SERVICES_OPTIONS_INLINE: |
        {
          "BasePath": "/oidc"
        }
    volumes:
      - .:/tmp/common/openid:ro

Let me know if there are more details that you might need or anything you want me to try.

[AleF83]

Hi, sorry for late reply.
I've fixed the image healthcheck. Try to use version 9.0.1.

[mahmoud-m-abadi]

No it does not work yet and show a unhealthy status

[astayleraz]

9.0.1 is not working for me either.

[mahmoud-m-abadi]

@astayleraz
Yes version 0.9.0 and 0.9.1 does not work because of healthy check
I installed the version the 0.8.6 and it work fines.

[dotansimha]

Same here (kamilkisiela/graphql-hive#4414)

[AleF83]

Sorry for delay...
I'm trying to find time to take a look on it.

[anmeeks-vt-edu]

When I inspect the unhealthy docker container, I see:
"Health": { "Status": "unhealthy", "FailingStreak": 365, "Log": [ { "Start": "2024-06-17T08:50:12.570369871-04:00", "End": "2024-06-17T08:50:12.592084642-04:00", "ExitCode": 1, "Output": " % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\ncurl: (7) Failed to connect to localhost port 443 after 0 ms: Couldn't connect to server\n" }, ... (several more of the same 443 failure)

Also, it's probably unrelated, but when I run this container using the docker compose in the readme, it does not work at all. I have to change the port mapping from 4011:80 to 4011:8080.

[pniederlag]

From my finding the image does start with a service listening on Port 8080...
.. while healthcheck tries https on port 443 and Dockerfile and Docs mention Port 80 und 443.

For me changing

healthcheck.test: curl --fail http://localhost:8080/health || exit 1
(not using https and port 8080) did help

[AleF83]

@pniederlag thank you for your comment!

I'm moving all the endpoints to work with HTTPS because Chrome started to block HTTP.
So I recommend to everybody to switch to HTTPS.
It requires some more configuration to ignore certificate validation errors.

I can remove healthcheck at all and everybody will add by themselves if desired.