Apache Kafka installed on Mac (Intel). Single local producer and single local consumer. 1 topic with 3 partitions and 1 replication factor is created:
bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic animal --partitions 3 --replication-factor 1
Producer code:
bin/kafka-console-producer.sh --bootstrap-server localhost:9092 --topic animal
Producer Messages:
>alligator
>crocodile
>tiger
When producing messages (manually via producer-console), all go into the same partition. Shouldn't they get distributed across partitions?
I've tried with 3 records (as above), but they get sent to 1 partition only. Checked within tmp/kafka-logs/topic-0/00**00.log Other logs in topic- are empty.
I've tried with tens of records, but no luck.
I even increased the default partition configuration (num.partitions=3) within 'config/server.properties', but no luck.
I've also tried with different topics, but no luck.
Starting with kafka 2.4, the default partitioner was changed from round-robin to sticky, which will stick to the same partition (pun intended) for an entire batch.
With my kafka version, the kafka-console-producer uses a default batch size of 16384, so once you produce enough messages to fill that buffer, the partition will change.