I would like to know can we do server side filtering somehow in Kafka? What are the alternatives that support server side filtering natively? I mean can Kafka provide specific messages to the client to avoid filtering in the client side?
can Kafka provide specific messages to the client
Kafka is not a push-based system; it does not "provide messages". Consumer clients must be assigned topics to poll from.
Producers, on the other hand, perform "filtering" based on topics (e.g. user registration events to one topic, and user-session expiration events to another), or other conditional statements around the data they are sending; thus client-side filtering.
There is no other filtering between a producer ultimately calling send()
and the consumer ultimately calling poll()
, e.g. "server-side" filtering. If the consumer is only interested in a subset of records, then that client must have a mutual agreement with some producer to get specifically those records, otherwise, other extraneous records must be ignored or handled otherwise.
In Kafka Streams API, you have a filter()
operation, and that also is client-side filtering.
What are the alternatives that support server side filtering natively?
None that I know of... That would be weird if the message broker hid messages from being read