[SOLVED] Is there a way to purge the topic in Kafka?

Is there a way to purge the topic in Kafka?

I pushed a message that was too big into a kafka message topic on my local machine, now I'm getting an error:

kafka.common.InvalidMessageSizeException: invalid message size

Increasing the fetch.size is not ideal here, because I don't actually want to accept messages that big.

Solution

Temporarily update the retention time on the topic to one second:

kafka-topics.sh \
  --zookeeper <zkhost>:2181 \
  --alter \
  --topic <topic name> \
  --config retention.ms=1000

And in newer Kafka releases, you can also do it with kafka-configs --entity-type topics

kafka-configs.sh \
  --zookeeper <zkhost>:2181 \
  --entity-type topics \
  --alter \
  --entity-name <topic name> \
  --add-config retention.ms=1000

then wait for the purge to take effect (duration depends on size of the topic). Once purged, restore the previous retention.ms value.