apache-kafkaapache-kafka-streamsksqldbktable

Querying a Kafka Streams KTable using KSQL


I'm writing this Kafka streams application that takes the sensor readings that are being registered in a Kafka topic (as messages in JSON), and performs some aggregations on the value of those readings in a per-minute, per-hour and per-day basis. Then I materialize the KTables derived from those aggregates and store them using the default state store. I was wondering if it might be possible to query these tables using KSQL.


Solution

  • While Kafka Streams is the runtime for KSQL, KTables that you crate via a Kafka Streams application are not available in KSQL. If you want to have a TABLE in KSQL, you need to write a KSQL query that creates that TABLE.

    However note, that KSQL queries are continuous queries, and not "lookup" queries as in a relational database.

    In contrast, Kafka Streams support a feature called "interactive queries" (https://docs.confluent.io/current/streams/developer-guide/interactive-queries.html) that allows you to do key-based lookups into the state of a KTable.

    Last, there is current work in progress in KSQL, to expose "interactive queries", too.