I looked at the following documentation: https://github.com/akka/reactive-kafka and I saw the following code snippet:
implicit val actorSystem = ActorSystem("ReactiveKafka")
implicit val materializer = ActorMaterializer()
val kafka = new ReactiveKafka()
val publisher: Publisher[StringConsumerRecord] = kafka.consume(ConsumerProperties(
bootstrapServers = "localhost:9092",
topic = "lowercaseStrings",
groupId = "groupName",
valueDeserializer = new StringDeserializer()
))
I understand that 'publisher' is supposed to write messages to Kafka. However, Consumer in Kafka means exactly the opposite, meaning consumer reads messages from Kafka. If so, how does it make sense that 'publisher' is related to kafka.consume(ConsumerProperties...) ?
This is a really confusing case of terminology collision. Reactive-kafka uses Akka Streams, which is an implementation of Reactive Streams specification.
In this specification, Publisher
publishes to a stream and Subscriber
receives results from a stream. As you may see, when you define a stream for processing Kafka messages, Kafka consumer acts as a Publisher
because it is the source of messages (Source
in akka-streams terminology). Similarly, Kafka producer would be a Subscriber
, because it is at the end of a stream (Sink
in akka-streams).
So in your code you define a Publisher
to your stream which is a Kafka consumer.