ruby-on-railspostgresqldockerdeploymentkamal

Kamal deployment error on rails app: PostgreSQL connection issues


I am trying to deploying a rails app using Kamal but deployment fails with the following error

ERROR (SSHKit::Command::Failed): Exception while executing on host 139.162.238.85: docker exit status: 1
docker stdout: Nothing written
docker stderr: Error: target failed to become healthy

Cause of the Error

 2024-11-11T08:37:10.285904312Z bin/rails aborted!
    2024-11-11T08:37:10.286315763Z ActiveRecord::DatabaseConnectionError: There is an issue connecting with your hostname: 45.92.9.7. (ActiveRecord::DatabaseConnectionError)
    2024-11-11T08:37:10.286325094Z 
    2024-11-11T08:37:10.286327734Z Please check your database configuration and ensure there is a valid connection to your database.
    2024-11-11T08:37:10.287105015Z 
    2024-11-11T08:37:10.287207778Z 
    2024-11-11T08:37:10.287214998Z Caused by:
    2024-11-11T08:37:10.287428294Z PG::ConnectionBad: connection to server at "45.92.9.7", port 5432 failed: Connection refused (PG::ConnectionBad)
    2024-11-11T08:37:10.287438574Z  Is the server running on that host and accepting TCP/IP connections?
    2024-11-11T08:37:10.287698321Z 
    2024-11-11T08:37:10.287821165Z Tasks: TOP => db:prepare
    2024-11-11T08:37:10.287906737Z (See full trace by running task with --trace)

I attempted to test the database connection directly inside the container

root@b820408ba640:/# psql -h 45.92.9.7 -p 5432 -U postgres -d writehub_productiondoe
psql: error: connection to server at "45.92.9.7", port 5432 failed: Connection refused
        Is the server running on that host and accepting TCP/IP connections?
root@b820408ba640:/#

Configurations files database.yml

default: &default
  adapter: postgresql
  encoding: unicode
  # For details on connection pooling, see Rails configuration guide
  # https://guides.rubyonrails.org/configuring.html#database-pooling
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

production:
  <<: *default
  database: writehub_production
  username: postgres
  password: Verystrongsecurepassword
  host: 45.92.9.7

deploy.yml

image: ibunhabibu/contaborails

# Deploy to these servers.
servers:
  web:
    - 139.162.238.85

proxy: 
  ssl: true
  host: habibtech.online
  # Proxy connects to your container on port 80 by default.
  # app_port: 3000

# Credentials for your image host.
registry:
  username: ibunhabibu
  password:
    - KAMAL_REGISTRY_PASSWORD

# Configure builder setup.
builder:
  arch: amd64

env:
  clear:
    DB_HOST: 139.162.238.85
  secret:
    - RAILS_MASTER_KEY
    - POSTGRES_USER
    - POSTGRES_PASSWORD


ssh:
  user: deployer

accessories:
  db:
    image: postgres:15
    host: 139.162.238.85
    port: 127.0.0.1:5433:5433  # Ensure this is the correct port or change it to 5432 if needed
    env:
      clear:
        POSTGRES_USER: 'postgres'
        POSTGRES_DB: 'writehub_production'
      secret:
        - POSTGRES_USER
        - POSTGRES_PASSWORD
    files:
      - config/init.sql:/docker-entrypoint-initdb.d/setup.sql

Dockerfile

# syntax = docker/dockerfile:1

# This Dockerfile is designed for production, not development. Use with Kamal or build'n'run by hand:
# docker build -t my-app .
# docker run -d -p 80:80 -p 443:443 --name my-app -e RAILS_MASTER_KEY=<value from config/master.key> my-app

# Make sure RUBY_VERSION matches the Ruby version in .ruby-version
ARG RUBY_VERSION=3.3.3
FROM docker.io/library/ruby:$RUBY_VERSION-slim AS base

# Rails app lives here
WORKDIR /rails

# Install base packages
RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y curl libjemalloc2 libvips postgresql-client && \
    rm -rf /var/lib/apt/lists /var/cache/apt/archives

# Set production environment
ENV RAILS_ENV="production" \
    BUNDLE_DEPLOYMENT="1" \
    BUNDLE_PATH="/usr/local/bundle" \
    BUNDLE_WITHOUT="development"

# Throw-away build stage to reduce size of final image
FROM base AS build

# Install packages needed to build gems
RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y build-essential git libpq-dev pkg-config && \
    rm -rf /var/lib/apt/lists /var/cache/apt/archives

# Install application gems
COPY Gemfile Gemfile.lock ./
RUN bundle install && \
    rm -rf ~/.bundle/ "${BUNDLE_PATH}"/ruby/*/cache "${BUNDLE_PATH}"/ruby/*/bundler/gems/*/.git && \
    bundle exec bootsnap precompile --gemfile

# Copy application code
COPY . .


# Ensure bin/rails and docker-entrypoint are executable
RUN chmod +x ./bin/rails && chmod +x /rails/bin/docker-entrypoint

# Precompile bootsnap code for faster boot times
RUN bundle exec bootsnap precompile app/ lib/

# Precompiling assets for production without requiring secret RAILS_MASTER_KEY
RUN SECRET_KEY_BASE_DUMMY=1 ./bin/rails assets:precompile

# Final stage for app image
FROM base

# Copy built artifacts: gems, application
COPY --from=build "${BUNDLE_PATH}" "${BUNDLE_PATH}"
COPY --from=build /rails /rails

# Run and own only the runtime files as a non-root user for security
RUN groupadd --system --gid 1000 rails && \
    useradd rails --uid 1000 --gid 1000 --create-home --shell /bin/bash && \
    chown -R rails:rails db log storage tmp
USER 1000:1000

# Entrypoint prepares the database.
ENTRYPOINT ["/rails/bin/docker-entrypoint"]

# Start the server by default, this can be overwritten at runtime
EXPOSE 3000

CMD ["./bin/rails", "server", "-b", "0.0.0.0"]

I have ensured the .env file includes the necessary environment varibles but I still can't figure out why Kamal's deployment fails to connect to postgreSQL. Could someone help to identify what might be the issue?


Solution

  • This could happen if the kamal setup failed with an incorrect DB config. After that, kamal deploy will just add to the existing incorrect DB setup.

    So first, try resetting. (Assuming that you're just trying it out, Don't do this for PRODUCTION)

    1. kamal remove
    2. kamal setup

    If the above does not work, try making the following changes and do a reset using the above commands.

    # config/database.yml
    
    # ... default config
    
    production:
      <<: *default
      username: <%= ENV["POSTGRES_USER"] %>
      password: <%= ENV["POSTGRES_PASSWORD"] %>
      host: <%= ENV["DB_HOST"] %>
    

    The following changes are in the config.deploy.yml file, but I have separated them into blocks to make them easier to read.

    1. env
    # config.deploy.yml
    
    env:
      secret:
        - RAILS_MASTER_KEY # I am not sure why it's missing for you
        - POSTGRES_USER
        - POSTGRES_PASSWORD
      clear:
        DB_HOST: contaborails-postgres # Add this instead of putting the host directly
    
    1. accessories
    # config.deploy.yml
    
    accessories:
      postgres:
        image: postgres:16
        host: 139.162.238.85
        port: "127.0.0.1:5432:5432" # I have not tried this without quotes
        env:
          clear:
            POSTGRES_USER: "postgres"
            POSTGRES_DB: "writehub_production"
          secret:
            - POSTGRES_PASSWORD
            - POSTGRES_USER
        files:
          - config/init.sql:/docker-entrypoint-initdb.d/setup.sql
        directories: # This is missing for you
          - data:/var/lib/postgresql/data