pyflinkflink-statefun

Invoking Flink stateful function using REST API

I'm writing an Apache Flink Statefun application using python. I'm looking for help/pointers to invoke existing stateful function via a REST POST/GET call.

I referred to https://github.com/apache/flink-statefun-playground/blob/release-3.2/python/greeter/module.yaml Here it seems a new Type has been defined to accomplish this task.

However I'm looking for a solution which is available using for https://hub.docker.com/r/apache/flink-statefun.

Thanks

Solution

  • Since Stateful Functions is a Flink job internally, it inherits Flink's ability to rewind the progress. When Flink restarts job processing from a checkpoint or a savepoint, it resets a source position to reconsume the data and recreate the state. A job needs to have a resettable source unless at-most-once delivery semantics is enough for your application.

    Assuming you need at-least-once delivery semantics, I suggest implementing an API that produces data to a resettable source (Kafka, for instance). So, the whole process would be the following:

    1. Someone calls HTTP API
    2. HTTP handler produces a message to Kafka
    3. Flink consumes the message and routes it to your function