I am using Debezium (a Kafka Connector) to capture Postgres database changes and I am getting an error from Debezium. Does anyone know what the error below means and perhaps offer a suggestion to fix it.
A bit more debugging info:
Postgres|dbserver1|snapshot After applying the include/exclude list filters, no changes will be captured. Please check your configuration! [io.debezium.relational.RelationalDatabaseSchema]
I have verified (in the logs) that Kafka (and schema registry etc) is running properly, and the Debezium connector seems to have started, and Postgres iw working properly and the database and tables are created.
Below is the Debezium connector configuration:
{
"name": "banking-postgres-connector",
"config": {
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"database.hostname": "postgres",
"database.port": "5432",
"database.user": "postgres",
"database.password": "postgres",
"database.dbname" : "banking",
"database.server.name": "dbserver1",
"database.include.list": "banking",
"tasks.max": "1",
"table.include.list": "public.x_account,public.x_party,public.x_product,public.x_transaction"
}
}
After many hours debugging (plus some advice from @OneCricketeer which pointed me in the right direction), I managed to get a configuration that works with debezium/connect:1.9. The solution was to use defaults by eliminating configuration items:
database.include.list
schema.include.list
This final working Debezium configuration is as follows:
{
"name": "banking-postgres-connector",
"config": {
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"database.hostname": "postgres",
"database.port": "5432",
"database.user": "postgres",
"database.password": "postgres",
"database.dbname" : "banking",
"database.server.name": "dbserver1",
"tasks.max": "1"
}
}
This does point to a minor gap in the Debezium documentation and code:
NOTE: I offer the above as humble suggestions because I find Debezium to be an OUTSTANDING product!