I am using debezium in distributed mode to push the cdc events of mssql db to kafka topics.
Command: %KAFKA_HOME%\bin\windows\connect-distributed.bat
Debezium starts fine, I deployed the connector and it synced tables and pushed records to kafka perfectly fine. But if I stop the %KAFKA_HOME%\bin\windows\connect-distributed.bat and restart, ideally it should resume from the last offset where I had stopped earlier but it gives an exception:
DebeziumException: The connector previously stopped while taking a snapshot, but now the connector is configured to never allow snapshots. Reconfigure the connector to use snapshots initially or when needed.
Connector configuration:
{
"name": "mssql-dbz-connector",
"config": {
"connector.class": "io.debezium.connector.sqlserver.SqlServerConnector",
"database.hostname": "***",
"database.port": "1433",
"database.user": "***",
"database.password": "***",
"database.names": "***",
"database.encrypt": "false",
"topic.prefix": "***",
"schema.history.internal.kafka.bootstrap.servers": "***",
"schema.history.internal.kafka.topic": "***",
"snapshot.mode": "schema_only"
}
}
Exception:
ERROR [mssql-dbz-connector|task-0] WorkerSourceTask{id=mssql-dbz-connector-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:212)
io.debezium.DebeziumException: The connector previously stopped while taking a snapshot, but now the connector is configured to never allow snapshots. Reconfigure the connector to use snapshots initially or when needed.
at io.debezium.connector.common.BaseSourceTask.validateAndLoadSchemaHistory(BaseSourceTask.java:97)
at io.debezium.connector.sqlserver.SqlServerConnectorTask.start(SqlServerConnectorTask.java:108)
at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:240)
at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.initializeAndStart(AbstractWorkerSourceTask.java:280)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:202)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:259)
at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.run(AbstractWorkerSourceTask.java:77)
at org.apache.kafka.connect.runtime.isolation.Plugins.lambda$withClassLoader$1(Plugins.java:236)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
[2024-05-19 23:48:29,610] INFO [mssql-dbz-connector|task-0] Stopping down connector (io.debezium.connector.common.BaseSourceTask:398)
I am using 2.6.1 version and this issue is a known issue and will be released in 2.6.2 and it is still not released.