cconfluent-platformlibrdkafka

How to get number of items on producer(using librdkafka) send queue


In producer using librdkafka, is it possible to know the number of items produced but not sent yet. I have turned ack off, only interested in sending the message out from the producer. I wish to avoid producing more messages if there are certain number of messages on the send queue(ie the messages produced but not yet sent).


Solution

  • If you set statistics.interval.ms in the producer configuration file, it will start sending the internal metrics in the callback you register. The callback you can set using the function rd_kafka_conf_set_stats_cb. The statistics are received in the callback in the form of json data. One the fields from the json data, msg_cnt, tell you the queued messages that you are looking for. More details are available at https://github.com/edenhill/librdkafka/blob/master/STATISTICS.md