node.jsnetwork-programmingapache-kafkatcpaiven

Get kafka message processing time


I have 2 services: producer and consumer.

As far as I understand, message.ts is the time the producer produced the message (not the time the kafka-broker received the message).

Questions

  1. When the consumer consume the message, how can I know how much time it was inside the kafka-broker (without the network latency: from producer to kafka-broker and from kafka-broker to consumer)?

  2. I did a ping from my consumer vm to the kafka broker. the ping result was 0.7ms (millisecond). Does the network latency from each side to the kafka broker is 0.3ms? I assume kafka transport is TCP so there is a "ACK" message for everything. And I assume that each side won't do nothing without "ACK" so I conclude that the network latency on each size is the same as the ping result: 0.7ms (millisecond). Am I correct?


Solution

  • I tested manually this by producing and consuming from the same vm to a kafka (which was inside my cluster). The result was 1.3-1.5 ms.

    It means that the procssing time took 0.1 ms on average.

    This is not the best solution, but it is suffient to my research.