After upgrading Kafka from 2.7 to 3.7, we benchmark a topic with Kafka's kafka-producer-perf-test.sh. the scale of perf test results change. for example, the latency moves from 5ms to 60ms.
bin/kafka-topics.sh --bootstrap-server X:9092 --create --topic perf-test compression.type=snappy --config min.insync.replicas=2 --config retention.ms=84000000 --partitions 9 --replication-factor 3 --command-config /Y
After investigating we found the result is not because of the change in the Kafka version itself but it's because of updating the client that runs the test,kafka-producer-perf-test.sh
So remember that in upgrading this bash file from version 2.8 to 3.0, there are some changes that don't let you compare test results. These changes are listed below.
ack=all, enable.idempotence=false