javaspring-bootapache-kafkaspring-kafkaapache-kafka-streams

Kafka Streams Admin Client disconnections

We are using Kafka Streams to do some Streams processing and we are seeing some strange logs in our observability system.
Everything is operational and the processing works without any problems, but the INFO log is a little bit concerning.

Here is our stream definition

@Bean
public KTable < Long, String > kStream(StreamsBuilder streamsBuilder) {
 KStream < Long, String > stream1 = streamsBuilder.stream(topic1);
 KStream < Long, String > stream2 = streamsBuilder.stream(topic2);

 stream1.merge(stream2)
     .selectKey((userId, v) - > userId)
     .groupByKey()
     .aggregate(
         CustomObject::new,
         (aggKey, newValue, aggValue) - > aggValue.aggregate(newValue),
         Materialized
         . < Long, OutputObject, KeyValueStore < Bytes, byte[] >> as(storeName)
         .withKeySerde(Serdes.Long())
         .withValueSerde(outputCustomSerde)
         .withStoreType(Materialized.StoreType.IN_MEMORY)
         .withCachingDisabled()
     )
     .toStream()
     .to(topic3);
 return null;
}

Based on the stream definition - 2 kafka topics are created - one changelog and one repartition.

Here are the 3 info logs that we get:
[AdminClient clientId=test-kafka-streams-v2-ade9efd5-bf02-4f98-baf7-97db2bfcb2ef-admin] Cancelled in-flight DELETE_RECORDS request with correlation id 1780227 due to node 1 being disconnected (elapsed time since creation: 3ms, elapsed time since send: 3ms, request timeout: 2ms)

[AdminClient clientId=test-kafka-streams-v2-ade9efd5-bf02-4f98-baf7-97db2bfcb2ef-admin] Cancelled in-flight METADATA request with correlation id 1845715 due to node 1 being disconnected (elapsed time since creation: 19ms, elapsed time since send: 19ms, request timeout: 18ms)

[AminClient clientId=test-kafka-streams-v2-ade9efd5-bf02-4f98-baf7-97db2bfcb2ef-admin] Disconnecting from node 1 due to request timeout.

Our question being - should we be worried about those 3 logs? If yes - what we can do about that?
If not - should we supress it somehow or just ignore it?
Thanks a lot

Solution

  • should we be worried about those 3 logs

    No. The admin client would just re-connect and retry.

    should we supress it somehow or just ignore it?

    I would just ignore it. Of course, you could also change the log level.