postgresqldockeropennms

opennms cannot connect to postgresql database on initializing

I tried to install the OpenNMS docker version on a Debian 12 based on the steps in the official documentation at https://github.com/OpenNMS/opennms/blob/develop/docs/modules/deployment/pages/core/docker/ (core -> postgresql -> initialize). Everything went well on initializing until the opennms failed to valitdate database verion:

08:38:39.864 [Main] INFO  org.opennms.core.schema.Migrator - validating database version
java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.opennms.bootstrap.Bootstrap$4.run(Bootstrap.java:524)
        at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: org.opennms.core.schema.MigrationException: an error occurred getting the version from the database
        at org.opennms.core.schema.Migrator.getDatabaseVersion(Migrator.java:296)
        at org.opennms.core.schema.Migrator.validateDatabaseVersion(Migrator.java:324)
        at org.opennms.core.schema.Migrator.setupDatabase(Migrator.java:1008)
        at org.opennms.install.Installer.install(Installer.java:248)
        at org.opennms.install.Installer.main(Installer.java:1009)
        ... 6 more
Caused by: org.postgresql.util.PSQLException: Connection to 127.0.0.1:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:342)
        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:54)
        at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:253)
        at org.postgresql.Driver.makeConnection(Driver.java:434)
        at org.postgresql.Driver.connect(Driver.java:291)
        at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:681)
        at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:190)
        at org.opennms.core.db.install.SimpleDataSource.getConnection(SimpleDataSource.java:106)
        at org.opennms.core.schema.Migrator.getDatabaseVersion(Migrator.java:284)
        ... 10 more
Caused by: java.net.ConnectException: Connection refused
        at java.base/sun.nio.ch.Net.pollConnect(Native Method)
        at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
        at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:547)
        at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:602)
        at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
        at java.base/java.net.Socket.connect(Socket.java:633)
        at org.postgresql.core.PGStream.createSocket(PGStream.java:243)
        at org.postgresql.core.PGStream.<init>(PGStream.java:98)
        at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:132)
        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:258)
        ... 18 more
+ exit 127
ERROR: 127

I have checked iptables rules and everything seems fine:

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER-USER  0    --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 DOCKER-ISOLATION-STAGE-1  0    --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     0    --  *      br-3b1a06639d96  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     0    --  *      br-3b1a06639d96  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     0    --  br-3b1a06639d96 !br-3b1a06639d96  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     0    --  br-3b1a06639d96 br-3b1a06639d96  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     0    --  *      br-a71aec85e9c3  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     0    --  *      br-a71aec85e9c3  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     0    --  br-a71aec85e9c3 !br-a71aec85e9c3  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     0    --  br-a71aec85e9c3 br-a71aec85e9c3  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     0    --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     0    --  *      docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     0    --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     0    --  docker0 docker0  0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER (3 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     6    --  !br-3b1a06639d96 br-3b1a06639d96  0.0.0.0/0            172.20.0.2           tcp dpt:5432

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER-ISOLATION-STAGE-2  0    --  br-3b1a06639d96 !br-3b1a06639d96  0.0.0.0/0            0.0.0.0/0
    8   480 DOCKER-ISOLATION-STAGE-2  0    --  br-a71aec85e9c3 !br-a71aec85e9c3  0.0.0.0/0            0.0.0.0/0
    0     0 DOCKER-ISOLATION-STAGE-2  0    --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
   15   629 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-ISOLATION-STAGE-2 (3 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       0    --  *      br-3b1a06639d96  0.0.0.0/0            0.0.0.0/0
    0     0 DROP       0    --  *      br-a71aec85e9c3  0.0.0.0/0            0.0.0.0/0
    0     0 DROP       0    --  *      docker0  0.0.0.0/0            0.0.0.0/0
    7   292 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
   23  1109 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0

Postgresql username/password is also checked:

~/horizon/docker-compose.yml

version: '3'

volumes:
  data-opennms: {}
  data-config: {}

services:
  horizon:
    image: opennms/horizon:latest
    container_name: horizon
    environment:
      TZ: 'Asia/Shanghai'
      POSTGRES_HOST: '127.0.0.1'
      POSTGRES_PORT: 5432
      POSTGRES_USER: 'postgres'
      POSTGRES_PASSWORD: 'MY.POSTGRES.PASSWORD'
      OPENNMS_DBNAME: 'opennms-core-db'
      OPENNMS_DBUSER: 'opennms'
      OPENNMS_DBPASS: 'MY.OPENNMS-DB.PASSWORD'
    volumes:
      - data-opennms:/opennms-data
      - data-config:/opt/opennms/etc
    command: ["-s"]
    ports:
      - '8980:8980/tcp'
      - '8101:8101/tcp'
    healthcheck:
      test: [ 'CMD', 'curl', '-f', '-I', 'http://localhost:8980/opennms/login.jsp' ]
      interval: 1m
      timeout: 5s
      retries: 3

~/postgres/docker-compose.yml

version: '3'

volumes:
  data-postgres: {}

services:
  database:
    image: postgres:latest
    container_name: database
    environment:
      TZ: 'Asia/Shanghai'
      POSTGRES_USER: 'postgres'
      POSTGRES_PASSWORD: 'MY.POSTGRES.PASSWORD'
    volumes:
      - 'data-postgres:/var/lib/postgresql/data'
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready -U postgres" ]
      interval: 10s
      timeout: 3s
      retries: 3
    ports:
      - '5432:5432/tcp'
Solution

  • You can't use 127.0.0.1 as a PostgreSQL host in a container service. The container horizon runs isolated in a network namespace different from the database service. If you want to keep the services in separate docker services, you need to publish the PostgreSQL port and use the IP address from the host system instead.