postgresqlapache-kafkaapache-kafka-connectdebeziumchange-data-capture

org.postgresql.util.PSQLException: ERROR: syntax error


I am trying to add debezium-connector-postgres to my Kafka Connect.

First I validated my config by

PUT http://localhost:8083/connector-plugins/io.debezium.connector.postgresql.PostgresConnector/config/validate

{
    "name": "postgres-kafkaconnector",
    "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
    "tasks.max": "1",
    "database.hostname": "example.com",
    "database.port": "5432",
    "database.dbname": "my_db",
    "database.user": "xxx",
    "database.password": "xxx",
    "database.server.name": "my_db_server",
    "table.include.list": "public.products",
    "plugin.name": "pgoutput"
}

It returns this which shows no error:

{
    "name": "io.debezium.connector.postgresql.PostgresConnector",
    "error_count": 0,
    ...

Then I try add this connector by

POST http://localhost:8083/connectors

{
    "name": "postgres-kafkaconnector",
    "config": {
        "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
        "tasks.max": "1",
        "database.hostname": "example.com",
        "database.port": "5432",
        "database.dbname": "my_db",
        "database.user": "xxx",
        "database.password": "xxx",
        "database.server.name": "my_db_server",
        "table.include.list": "public.products",
        "plugin.name": "pgoutput"
    }
}

The connector got added successfully.

However, I got error when I run

GET http://localhost:8083/connectors/postgres-kafkaconnector/status

ERROR WorkerSourceTask{id=postgres-kafkaconnector-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:179)
io.debezium.jdbc.JdbcConnectionException: ERROR: syntax error
    at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.initPublication(PostgresReplicationConnection.java:180)
    at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.createReplicationSlot(PostgresReplicationConnection.java:351)
    at io.debezium.connector.postgresql.PostgresConnectorTask.start(PostgresConnectorTask.java:136)
    at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:130)
    at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:208)
    at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)
    at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: ERROR: syntax error
    at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2565)
    at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2297)
    at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:322)
    at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:481)
    at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:401)
    at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:322)
    at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:308)
    at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:284)
    at org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:236)
    at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.initPublication(PostgresReplicationConnection.java:137)
    ... 11 more

How can I see the full log especially that 11 more lines? I also checked my Kubernetes pod log, it shows same part without that 11 more lines.

The current error content is not very useful. Any help for further debugging would be appreciate!


UPDATE 1:

GET http://localhost:8083/connectors

returns

[
    "postgres-kafkaconnector"
]

Solution

  • Before we are using Postgres 9.6.12, after switching to Postgres 13.6.

    With same setup step, it works well this time.

    My best guess is maybe because the debezium-connector-postgres version 1.8.1.Final I am using does not work well with old Postgres 9.6.12.