Kamal 2 Setup Failing with Vanilla Rails 8.01

[justronin]

Struggled for 2 days with getting my app running in a "production" environment using Hetzner and Kamal 2. The error I'm getting is "First web container is unhealthy". The container with the app never seems to start.

Ubuntu 24.04.1 LTS (GNU/Linux 6.8.0-51-generic x86_64)
Docker version 27.3.1, build ce12230

I gave up using my app and tried just using a brand new Rails app and got the same error, as follows:

Rails Version
$ rails --version
Rails 8.0.1
$ rails new vanilla

I work on a Macbook and wanted to deploy on a Hetzner cpx21 in the US with Docker CE on AMD64. The only change I made was to add a remote builder to avoid building using my macbook's ARM64 architecture -- see the builder section of config/deploy.yml below (website name was redacted with EXAMPLE):

service: vanilla

image: justronin/vanilla

servers:
  web:
    - vanilla.EXAMPLE.com

proxy:
  ssl: true
  host: vanilla.EXAMPLE.com

registry:
  username: justronin

  password:
    - KAMAL_REGISTRY_PASSWORD

env:
  secret:
    - RAILS_MASTER_KEY
  clear:
    SOLID_QUEUE_IN_PUMA: true

volumes:
  - "vanilla_storage:/rails/storage"

asset_path: /rails/public/assets

builder:
  arch: amd64
  remote: ssh://[email protected]
  args:
    RUBY_VERSION: <%= File.read('.ruby-version').strip %>

deploy_timeout: 60

.kamal/secrets

KAMAL_REGISTRY_PASSWORD=$KAMAL_REGISTRY_PASSWORD
RAILS_MASTER_KEY=$(cat config/master.key)

I fails like this (^^^^ marks the error:

 INFO [c76c25e2] Running /usr/bin/env mkdir -p .kamal/apps/vanilla/env/roles on vanilla.EXAMPLE.com
 DEBUG [c76c25e2] Command: /usr/bin/env mkdir -p .kamal/apps/vanilla/env/roles
  INFO [c76c25e2] Finished in 0.117 seconds with exit status 0 (successful).
  
  
 ERROR Failed to boot web on vanilla.EXAMPLE.com
  INFO First web container is unhealthy on vanilla.EXAMPLE.com, not booting any other roles
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  INFO [08246c2f] Running docker container ls --all --filter name=^vanilla-web-901c1432146a9ec4c01a21a6cb84eec4b35adaef$ --quiet | xargs docker logs --timestamps 2>&1 on vanilla.EXAMPLE.com
 DEBUG [08246c2f] Command: docker container ls --all --filter name=^vanilla-web-901c1432146a9ec4c01a21a6cb84eec4b35adaef$ --quiet | xargs docker logs --timestamps 2>&1
 DEBUG [08246c2f] 	"docker logs" requires exactly 1 argument.
See 'docker logs --help'.

Usage:  docker logs [OPTIONS] CONTAINER

Fetch the logs of a container
^C  Finished all in 257.7 seconds
  Finished all in 259.1 seconds

The result is that the container with the app never starts. Below is what I can see on the CPX21.

Docker containers:

root@docker-ce-ubuntu-4gb-ash-1:~# docker container ls

CONTAINER ID   IMAGE                           COMMAND                  CREATED          STATUS             PORTS                                                                      NAMES

08873a80893f   basecamp/kamal-proxy:v0.8.4     "kamal-proxy run"        33 minutes ago   Up 33 minutes      0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp   kamal-proxy

c96b16fc53bc   moby/buildkit:buildx-stable-1   "buildkitd --allow-i…"   2 hours ago      Up About an hour                                                                              buildx_buildkit_kamal-remote-ssh---root-vanilla-EXAMPLE-com0

No errors noted on the kamal-proxy container:

root@docker-ce-ubuntu-4gb-ash-1:~# docker logs 08873a80893f

{"time":"2024-12-31T21:08:58.105772025Z","level":"INFO","msg":"No previous state to restore","path":"/home/kamal-proxy/.config/kamal-proxy/kamal-proxy.state"}

{"time":"2024-12-31T21:08:58.10668033Z","level":"INFO","msg":"Server started","http":80,"https":443}

[justronin]

A followup to the above:

The Kamal Proxy seems to be failing to start the application container. I am able to execute docker run with the image and without the proxy everything works fine.

[justronin]

On the off-chance that the following issue was causing the problem, I re-ran the test and the setup still fails:
basecamp/thruster#63