kafka-connect is getting error as below when opening in distributor mode. I've already increased the some connect-distributed.propertieses parameters which as below. I have to open connect in distributed mode also.
connect-distributed.properties;
offset.flush.interval.ms=5000
offset.flush.timeout.ms=300000
producer.buffer.memory=300097152
status.storage.topic=connect-status
status.storage.replication.factor=1
config.storage.topic=connect-configs
config.storage.replication.factor=1
offset.storage.topic=connect-offsets
offset.storage.replication.factor=1
key.converter.schemas.enable=false
value.converter.schemas.enable=false
INFO [AdminClient clientId=adminclient-8] Disconnecting from node 0 due to request timeout.
(org.apache.kafka.clients.NetworkClient:797)
INFO [AdminClient clientId=adminclient-8] Cancelled in-flight METADATA request with
correlation id 530256 due to node 0 being disconnected (elapsed time since creation: 1ms,
elapsed time since send: 1ms, request timeout: 0ms)
(org.apache.kafka.clients.NetworkClient:341)
WARN Attempt 2 to list offsets for topic partitions resulted in RetriableException; retrying
automatically. Reason: Timed out while waiting to get end offsets for topic 'connect-offsets'
on brokers at *.*.*.*:9092 (org.apache.kafka.connect.util.RetryUtil:84)
org.apache.kafka.common.errors.TimeoutException: Timed out while waiting to get end offsets
for topic 'connect-offsets' on brokers at *.*.*.*:9092
Caused by: java.util.concurrent.ExecutionException:
org.apache.kafka.common.errors.TimeoutException: Call(callName=metadata,
deadlineMs=1680677446792, tries=1, nextAllowedTryMs=1680677446893) timed out at 1680677446793
after 1 attempt(s)
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:165)
at org.apache.kafka.connect.util.TopicAdmin.endOffsets(TopicAdmin.java:672)
at org.apache.kafka.connect.util.TopicAdmin.lambda$retryEndOffsets$6(TopicAdmin.java:725)
at org.apache.kafka.connect.util.RetryUtil.retryUntilTimeout(RetryUtil.java:82)
at org.apache.kafka.connect.util.TopicAdmin.retryEndOffsets(TopicAdmin.java:724)
at org.apache.kafka.connect.util.KafkaBasedLog.readEndOffsets(KafkaBasedLog.java:375)
at org.apache.kafka.connect.util.KafkaBasedLog.readToLogEnd(KafkaBasedLog.java:335)
at org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:201)
at org.apache.kafka.connect.storage.KafkaOffsetBackingStore.start(KafkaOffsetBackingStore.java:151)
at org.apache.kafka.connect.runtime.Worker.start(Worker.java:186)
at org.apache.kafka.connect.runtime.AbstractHerder.startServices(AbstractHerder.java:135)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:320)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=metadata,
deadlineMs=1680677446792, tries=1, nextAllowedTryMs=1680677446893) timed out at
1680677446793 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.DisconnectException: Cancelled metadata request
with correlation id 530256 due to node 0 being disconnected
I've also added the result of the connect-offsets detail.
kafka/bin/kafka-topics.sh --bootstrap-server localhost:9092 -describe --topic connect-offsets
Topic: connect-offsets TopicId: naW7oTihQwK PartitionCount: 25
ReplicationFactor: 1 Configs:
cleanup.policy=compact,segment.bytes=1073741824
Topic: connect-offsets Partition: 1 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 2 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 3 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 4 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 5 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 6 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 7 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 8 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 9 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 10 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 11 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 12 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 13 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 14 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 15 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 16 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 17 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 18 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 19 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 20 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 21 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 22 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 23 Leader: 0 Replicas: 0 Isr: 0
Topic: connect-offsets Partition: 24 Leader: 0 Replicas: 0 Isr: 0
That issue has been solved when I delete all of files which were kafka log.dirs and zookeeper data files.