capache-kafkapartitionconsumerlibrdkafka

How to consume messages from a specific partition of a topic in kafka


I want a kafka consumer to consume messages from a particular/specified partition of a topic

It works using kafka-console-consumer.sh with switch --partition partition_number

I am using kafka_complex_consumer_example.c code from librdkafka

Upon static code analysis, I feel it can serve my purpose but I am not able to find out the exact command line parameters to be passed to main(int argc & char **argv) function for the code to run and start consuming on a topic's particular partition.

Have a look at the code here - rdkafka_complex_consumer_example.c

Full github code of librdkafka here FYR

If this code doesn't serve the purpose, please specify some other code that can help


Solution

  • If you look at the usage string, putting a colon after the topic name designates which partition to consume from

    More specifically, topicpartition type holds information for a specific partition, and this line creates the list of those

    https://github.com/edenhill/librdkafka/blob/master/examples/rdkafka_complex_consumer_example.c#L528