I am trying to understand how redis streams does partitioning , if a specific message can be sent to a specific partition (similar to how you can do with Kafka).
I have checked the redis-cli api , and there is nothing similar to partitioning , also nothing about this when using the StackExchangeRedis redis library.
The only method is : IDatabase.StreamAdd(key,streamKey,streamValue,messageId)
Am i missing anything ? Is the partitioning done only in a fixed way ?
P.S If partitioning can be done , can the partitioning key be composed ?
You cannot achieve Kafka partition with a single Redis Stream. Redis Stream's consumer group looks like a load balancer, which distributes messages in the stream to different consumers without any partition. If one consumer is faster than others, it will consume more messages than others, regardless of the logical partition of messages.
If you want to do partition, you have to use multiple Redis Streams, i.e. multiple keys, and each key for a different partition. And your producers add messages in a logical partition to a dedicated Redis Stream. The different consumers xread messages from different key.