Is there a concept like kafka routing key in redis streams? We have a number of microservice instances (let say 10) and we intend to create a consumer group, each consumer is one instance. We want messages of instance n will be read/retrieved only when instance n read/subscribe to the stream (based on some criteria in inside the message itself). is it possible?
There is no way to reserve or route messages to a specific consumer in Redis: once a XREADGROUP command is issued, the returned messages are bound to the specified consumer - which must be specified upfront.
With that being said, Redis allows to change the ownership of a message through the XCLAIM command, once it is read. Thanks to that, you could somewhat mimic reservations / routings by having a specialized dispatcher consumer which continuously reads undelivered messages only (by way of the XREADGROUP command along with the special ID >) and assigns them through the XCLAIM command to the intended target consumer, based on their content. Each regular consumer reads messages delivered to it but still not acknowledged by way of the XREADGROUP command along with the starting ID 0, processes each message according to the business logic and finally XACK it to remove it from the PEL.
From the referenced documentation:
The special > ID, which means that the consumer want to receive only messages that were never delivered to any other consumer. It just means, give me new messages.
The target consumers, on the other side, would only read messages assigned to them (and not the undelivered ones) with the XREADGROUP command.