I have an application that listens to a Kafka topic and reads 1 message from it in 1 thread. After reading a message, there is a certain logic, during which there are two options - everything is successful and a commit occurs call method ...acknowledge() to offset the offset and move on to the next message, or everything is unsuccessful and this message must be read again and the offset is not committed (...acknowledge() is not called)
If I do not call Acknowledge (), then shouldn't the consumer go on all reports in the topic and then go to the second circle, re-read the messages on which Acknowledge () was not called? Or how can I configure it?
I'm using spring-kafka and the following settings are specified:
spring.kafka.consumer.enable-auto-commit=false
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.consumer.max-poll-records=1
And I programmatically set the settings:
...setGroupId("test");
...setSyncCommits(true);
...setAckMode(ContainerProperties.AckMode.MANUAL_IMMEDIATE);
...setConcurrency(1);
No; Kafka maintains two pointers, the last committed offset and the current position. They are completely unrelated, except the current position is set to the last committed offset when a consumer first starts.
Not acknowledging (committing the offset) of a record does not change the current position.
You cannot continue processing other records after the failure because their offsets will override the offset of the failed record. There is only one committed offset per partition.
You can either throw an exception (and the DefaultErrorHandler will perform the necessary seeks so that the current record will be redelivered, until retries are exhausted).
Or, you can call nack() on the acknowledgment, which will do the same (but has no concept of retry exhaustion).
See https://docs.spring.io/spring-kafka/docs/current/reference/html/#annotation-error-handling
and https://docs.spring.io/spring-kafka/docs/current/reference/html/#committing-offsets