/Setup error and/or better docker support

[DanceMore]

I logged into Github today and my feed said you cut a new release, so I raced to go try it out, including the new docker image....

well, it turns out, I have been using a slightly fudged docker image:

FROM ghcr.io/michielpost/michielpost/huelightdj:1.0.2

COPY files/appsettings.json /app/appsettings.json

Docker "volumes" basically hard-assume directory, not file. and while they "sometimes work" using a file as a volume mount, it can be error-prone and some toolchains will outright syntax error over the use of a non-directory volume mount. my quick workaround is simply publish my own site-local Docker image with my config "baked in", ie: hardcoded. this works, but is not an ideal setup because it's extra steps and not every Docker user has a full private registry and CICD stack to customize Docker images for site-local configs.

the quickest ""fix"" is probably to just put appsettings.json in some sort of site-local config directory and change whatever code reads the config to expect a directory basedir of some kind. the directory can be basically anywhere and named anything, it can even include a default copy of appsettings.json, it just must be a directory in a documented PATH so people like me can more easily override it.

"no bother, I'll try this new Setup Wizard button... oh." incidentally, Docker Users probably want the Setup Wizard to write the appsettings to a Volume Directory so it can persist between container runs, so I have little interest in drilling into this Setup error, but it is included for completeness:

TaskCanceledException: A task was canceled.
System.Net.Http.HttpClient.HandleFailure(Exception e, bool telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, bool disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
HueApi.BridgeLocator.HttpBridgeLocator.LocateBridgesAsync(CancellationToken cancellationToken)
HueApi.BridgeLocator.BridgeLocator.LocateBridgesAsync(TimeSpan timeout)
HueLightDJ.Services.Controllers.HomeController.Setup() in HomeController.cs
Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor+TaskOfIActionResultExecutor.Execute(ActionContext actionContext, IActionResultTypeMapper mapper, ObjectMethodExecutor executor, object controller, object[] arguments)
System.Threading.Tasks.ValueTask<TResult>.get_Result()
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask<IActionResult> actionResultValueTask)
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(ref State next, ref Scope scope, ref object state, ref bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(ref State next, ref Scope scope, ref object state, ref bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)

[michielpost]

Thanks for the feedback!

You're absolutely right about the whole docker setup. The configuration is currently inside the docker image. So if you want to add your own configuration, you will have to edit the files and build your own docker image.
There might be other solutions, like injecting the appsettings.json like you describe. But my Docker knowledge does not go that far.

The Setup Wizard probably crashed because your docker image could not connect to the internet to query for your bridge IP at a Philips Hue API endpoint. I've fixed that in the latest version.
Also bumped the GitHub docker tag to v2.0.0

The result of the Setup Wizard is json that you will have to manually add to your appsettings.json. So it's not as streamlined as I want it to be.

The good news is, I'm currently working on v3.
v3 will store the config outside the docker image. I'm currently looking at storing the config in the browser. But this has the disadvantage that when using multiple clients, each browser needs the config.
So if there is a way to store it to some persistant storage inside docker, that would be ideal. Not sure yet how to do that.

Suggestions are welcome! Thanks

[DanceMore]

does DotNet et al make it prohibitively difficult to just mkdir config/ and change all the paths to config/appsettings.json ?

it can even be inside the project folder, it just needs to be a sub-directory because docker generally assumes volume about-equal directory. but there is no need to modify the Dockerfile or complicate non-Docker configs by using like /config or /etc/huelightdj inside the Container File System. it can just be ./$PROJECTROOT/config. (which winds up as /app/config in your Docker builds)

here's an example of a docker-compose using a volume for a Complex Application with a config directory.

  hass:
    image: homeassistant/home-assistant
    restart: always
    network_mode: host
    volumes:
      - "/etc/localtime:/etc/localtime:ro"
      - "/etc/timezone:/etc/timezone:ro"
      - "/srv/docker/homeassistant/config:/config"
    environment:
      - TZ=America/Los_Angeles

here's an example of a desired docker-compose using a volume for HueDJ config

  huedj:
    image: ghcr.io/michielpost/michielpost/huelightdj:latest
    restart: always
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
    ports:
      - 8888:80
    volumes:
      - "/srv/docker/hugelightdj/config:/app/config"

by default, without the :ro, the volume will be writeable. sometimes sysadmins can have permissions errors but those are easy to diagnose and resolve and are not the application's fault. sometimes applications do have problems if they expect some default appsettings.json to exist and cannot handle switching to /Setup if it's completely missing. (since the first time docker-compose runs, it's just mkdiring stuff if it doesn't exist, it doesn't know to populate it with complex content.)

however, the good news is the Docker volume being writeable by default means once the permissions are ironed out (ie: one chown later), the pattern supports both BYO-config and "preserve across reboots a server-side generated config". (having to BYO-config or generate and copy in as a secondary step is an example of that "populate it with complex content" step that some applications in Dockerland require.)

[michielpost]

Today the first version of the v3 docker image is released. The frontend is rewritten in Blazor and the setup / configuration of the bridges now happens through the UI instead of config files. So no need to map a dicrectory to docker or anything like that.
Please try it out and share your feedback. Thanks.

[DanceMore]

finally looping back around to this... it's definitely the new UI, but the "Add bridge" fails.

fail: Grpc.AspNetCore.Server.ServerCallHandler[6]
      Error when executing service method 'Register'.
      System.UriFormatException: Invalid URI: The hostname could not be parsed.
         at System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind, UriCreationOptions& creationOptions)
         at System.Uri..ctor(String uriString)
         at HueApi.LocalHueApi.RegisterAsync(String ip, String applicationName, String deviceName, Boolean generateClientKey, Nullable`1 cancellationToken)
         at HueLightDJ.Services.HueSetupService.RegisterAsync(HueSetupRequest request, CallContext context) in /src/HueLightDJ.Services/HueSetupService.cs:line 46
         at Grpc.Shared.Server.UnaryServerMethodInvoker`3.AwaitInvoker(Task`1 invokerTask, GrpcActivatorHandle`1 serviceHandle)
         at Grpc.Shared.Server.UnaryServerMethodInvoker`3.AwaitInvoker(Task`1 invokerTask, GrpcActivatorHandle`1 serviceHandle)
         at Grpc.AspNetCore.Server.Internal.CallHandlers.UnaryServerCallHandler`3.HandleCallAsyncCore(HttpContext httpContext, HttpContextServerCallContext serverCallContext)
         at Grpc.AspNetCore.Server.Internal.CallHandlers.ServerCallHandlerBase`3.<HandleCallAsync>g__AwaitHandleCall|8_0(HttpContextServerCallContext serverCallContext, Method`2 method, Task handleCall)
fail: Grpc.AspNetCore.Server.ServerCallHandler[6]
      Error when executing service method 'Register'.
      System.UriFormatException: Invalid URI: The hostname could not be parsed.
         at System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind, UriCreationOptions& creationOptions)
         at System.Uri..ctor(String uriString)
         at HueApi.LocalHueApi.RegisterAsync(String ip, String applicationName, String deviceName, Boolean generateClientKey, Nullable`1 cancellationToken)
         at HueLightDJ.Services.HueSetupService.RegisterAsync(HueSetupRequest request, CallContext context) in /src/HueLightDJ.Services/HueSetupService.cs:line 46
         at Grpc.Shared.Server.UnaryServerMethodInvoker`3.AwaitInvoker(Task`1 invokerTask, GrpcActivatorHandle`1 serviceHandle)
         at Grpc.Shared.Server.UnaryServerMethodInvoker`3.AwaitInvoker(Task`1 invokerTask, GrpcActivatorHandle`1 serviceHandle)
         at Grpc.AspNetCore.Server.Internal.CallHandlers.UnaryServerCallHandler`3.HandleCallAsyncCore(HttpContext httpContext, HttpContextServerCallContext serverCallContext)
         at Grpc.AspNetCore.Server.Internal.CallHandlers.ServerCallHandlerBase`3.<HandleCallAsync>g__AwaitHandleCall|8_0(HttpContextServerCallContext serverCallContext, Method`2 method, Task handleCall)

I suspected the URL Parse was failing because my hostname is a fake TLD for internal network use, but the same error fires when I use IP.

I'm also not sure where this data eventually gets stored; but I probably want it in a docker volume so I don't have to rejoin via a physical button elsewhere every time the container is recreated. this warning implies another reason / location that might need volume'd:

warn: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[60]
      Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed.

is /root/.aspnet/ a candidate for volume ...?

[edit: my ancient container with a hard-coded appsettings.json still connects reliably to my Hue network, even after being deleted and recreated as part of these experiments]

[michielpost]

Make sure that the IP or hostname your input does not contain http / https or any spaces. Just the IP or hostname.
I just did a small commit that should auto remove the spaces.

With this new version, the data gets stored in your browser's local storage, so you won't have to re-register when the container is recreated.