[SOLVED] Meaning of beginning_offset in Kafka topic GET /offsets API

Meaning of beginning_offset in Kafka topic GET /offsets API

This is a API from https://docs.confluent.io/5.5.11/kafka-rest/api.html#partitions Confluent documentation.

GET /topics/(string:topic_name)/partitions/(int:partition_id)/offsets

It returns this -

 HTTP/1.1 200 OK
Content-Type: application/vnd.kafka.v2+json

{
  "beginning_offset": 10,
  "end_offset": 50,
}

My question is, what is the meaning of beginning_offset? Will it not be zero for all cases, pushing messages to a topic starts from zero offset? It is the end_offset that will change as messages are pushed to the topic. So, if I push one more message to this topic partition and again call this API, the offset will be 51. In what cases, will the beginning_offset be changed? Kafka explorer here, help will be appreciated.

Solution

New topics will have a zero offset, yes.

However, Kafka retention policies will cause the beginning offset to be greater than zero as data is deleted or compacted