Kamal heath check

kjdelys kjdelys posted on 2025-04-23 08:26:23 UTC

Hello guys,
i'm building an rails 7.2 app with classic esbuild, tailswind stuff. I would like to deploy it with kamal but for some reason, the healthcheck returns 403. I already tested locally, it works perfectly. (  get "up" => "rails/health#show", as: :rails_health_check)

This is my full kamal setup
---
:roles:
- web
:hosts:
- IP
:primary_host: IP
:version: d2124e62dd6be84d5db204511602962a4961c146
:repository: xxxxxxxxxxxxxxxxxxxxxxx
:absolute_image: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
:service_with_version: xxxxxxxxxxxxxxxxxxxxxxxxxxxx
:volume_args: []
:ssh_options:
  :user: ubuntu
  :port: 22
  :keepalive: true
  :keepalive_interval: 30
  :log_level: :fatal
:sshkit: {}
:builder:
  arch: amd64
  args:
    RAILS_ENV: staging
    RAILS_MASTER_KEY: xxxxxxxxxxxxxxxxxxxxxxxxxxxx
    PORT: 3000
:accessories:
  db:
    image: postgres:16
    port: 5432
    env:
      clear:
        POSTGRES_USER: deployer
        POSTGRES_DB: db_staging
      secret:
      - POSTGRES_PASSWORD
    directories:
    - data:/var/lib/postgresql/data
    options:
      health-cmd: pg_isready -U deployer -d db_staging
      health-interval: 10s
      health-timeout: 5s
      health-retries: '3'
    host: IP
  pgadmin:
    image: dpage/pgadmin4
    env:
      clear:
        PGADMIN_DEFAULT_EMAIL: admin@website.com
      secret:
      - PGADMIN_DEFAULT_PASSWORD
    options:
      publish:
      - 5050:80
      health-cmd: wget --quiet --tries=1 --spider http://localhost/login || exit 1
      health-interval: 10s
      health-timeout: 5s
      health-retries: 3
    host: 91.134.41.115
  redis:
    image: redis:7.0
    port: 6379
    directories:
    - data:/data
    options:
      health-cmd: redis-cli ping
      health-interval: 10s
      health-timeout: 60s
      health-retries: '6'
    host: 91.134.41.115
  minio:
    image: minio/minio
    env:
      secret:
      - MINIO_ROOT_USER
      - MINIO_ROOT_PASSWORD
    directories:
    - data:/data
    cmd: server /data --console-address ":9001"
    options:
      publish:
      - 9000:9000
      - 9001:9001
      health-cmd: curl -f http://localhost:9000/minio/health/live || exit 1
      health-interval: 10s
      health-timeout: 60s
      health-retries: '6'
    host: 91.134.41.115
  minio-client:
    image: minio/mc:RELEASE.2025-04-16T18-13-26Z
    env:
      clear:
        RAILS_ENV: staging
      secret:
      - MINIO_ROOT_USER
      - MINIO_ROOT_PASSWORD
    cmd: |-
      mc alias set local http://minio:9000 $${MINIO_ROOT_USER} $${MINIO_ROOT_PASSWORD} && \
      mc mb local/mega-$${RAILS_ENV:-development} || true && \
      tail -f /dev/null
    host: 91.134.41.115
:logging:
- "--log-opt"
- max-size="10m


it keeps failing with
2025-04-23T07:57:05.101722808Z {"time":"2025-04-23T07:57:05.101692592Z","level":"INFO","msg":"Request","path":"/up","status":403,"dur":0,"method":"GET","req_content_length":0,"req_content_type":"","resp_content_length":0,"resp_content_type":"text/html; charset=UTF-8","remote_addr":"172.18.0.2:48356","user_agent":"kamal-proxy","cache":"miss","query":""}
2025-04-23T07:57:06.101610566Z {"time":"2025-04-23T07:57:06.10153792Z","level":"INFO","msg":"Request","path":"/up","status":403,"dur":0,"method":"GET","req_content_length":0,"req_content_type":"","resp_content_length":0,"resp_content_type":"text/html; charset=UTF-8","remote_addr":"172.18.0.2:48356","user_agent":"kamal-proxy","cache":"miss","query":""}
2025-04-23T07:57:07.101083463Z {"time":"2025-04-23T07:57:07.101046975Z","level":"INFO","msg":"Request","path":"/up","status":403,"dur":0,"method":"GET","req_content_length":0,"req_content_type":"","resp_content_length":0,"resp_content_type":"text/html; charset=UTF-8","remote_addr":"172.18.0.2:48356","user_agent":"kamal-proxy","cache":"miss","query":""}
2025-04-23T07:57:08.101504114Z {"time":"2025-04-23T07:57:08.101471623Z","level":"INFO","msg":"Request","path":"/up","status":403,"dur":0,"method":"GET","req_content_length":0,"req_content_type":"","resp_content_length":0,"resp_content_type":"text/html; charset=UTF-8","remote_addr":"172.18.0.2:48356","user_agent":"kamal-proxy","cache":"miss","query":""}
2025-04-23T07:57:09.100956320Z {"time":"2025-04-23T07:57:09.100921414Z","level":"INFO","msg":"Request","path":"/up","status":403,"dur":0,"method":"GET","req_content_length":0,"req_content_type":"","resp_content_length":0,"resp_content_type":"text/html; charset=UTF-8","remote_addr":"172.18.0.2:48356","user_agent":"kamal-proxy","cache":"miss","query":""}
2025-04-23T07:57:10.101359339Z {"time":"2025-04-23T07:57:10.101326657Z","level":"INFO","msg":"Request","path":"/up","status":403,"dur":0,"method":"GET","req_content_length":0,"req_content_type":"","resp_content_length":0,"resp_content_type":"text/html; charset=UTF-8","remote_addr":"172.18.0.2:48356","user_agent":"kamal-proxy","cache":"miss","query":""}
2025-04-23T07:57:11.101740577Z {"time":"2025-04-23T07:57:11.101706473Z","level":"INFO","msg":"Request","path":"/up","status":403,"dur":0,"method":"GET","req_content_length":0,"req_content_type":"","resp_content_length":0,"resp_content_type":"text/html; charset=UTF-8","remote_addr":"172.18.0.2:48356","user_agent":"kamal-proxy","cache":"miss","query":""}
  INFO [327a50cf] Running docker container ls --all --filter name=^axcio-web-staging-d2124e62dd6be84d5db204511602962a4961c146$ --quiet | xargs docker inspect --format '{{json .State.Health}}' on 91.134.41.115                              
  INFO [327a50cf] Finished in 0.081 seconds with exit status 0 (successful).
 ERROR null
  INFO [dbaee8a9] Running docker container ls --all --filter name=^axcio-web-staging-d2124e62dd6be84d5db204511602962a4961c146$ --quiet | xargs docker stop on 91.134.41.115
  INFO [dbaee8a9] Finished in 0.167 seconds with exit status 0 (successful).
Releasing the deploy lock...
  Finished all in 49.7 seconds
  ERROR (SSHKit::Command::Failed): Exception while executing on host IP: docker exit status: 1
docker stdout: Nothing written
docker stderr: Error: target failed to become healthy within configured timeout (30s)


Any idea?
Thanks in advance


David Kimura PRO said about 1 year ago :
If this is a new deployment, you may be able to shell into the server and run the container locally on the server. Shell into the container and try basic things like opening the rails console from within the container. Try touching a record to make sure the database connection is working. If all of that checks out, try running the rails server. You should have an entry point shell script or a CMD in the dockerfile which starts up the rails process. Typically, what I've found is that the deployment will go through, but there's something missing that keeps the rails server from running properly.

Login to Comment
Back to forums