Apache Flink Stateful Functions forwarding the same message to N functions

I'm trying to send incoming messages to multiple stateful functions but I couldn't fully understand how to do. For the sake of understandability let's say one of my stateful function getting some integers and sending them to couple of remote functions. These functions adds this integers to their state values ​​and saves it as the new state.

When one of these 2 remote functions fails, the other should continue to work the same way. When the failed function recovered, it should process messages that it cannot process during failure.

I thought about sending them one after another as below, but I don't think it will work

context.send(RemoteFuncType1,someID,someInteger);
context.send(RemoteFuncType2,someID,someInteger);
...

• how can I do this in a fault tolerant way?
• if possible how it works in the background?

The way you are suggesting to do it is the correct way!

StateFun would deliver the messages to the remote functions in a consistent manner. If one of the functions is experiencing a short downtime, StateFun would retry sending the message until:

• It would successfully deliver it (with back off)
• A maximum timeout for retries would be reached. When a timeout is reached the whole StateFun job would be rewind to a previously consistent checkpoint.

Since StateFun is managing message delivery and the state of the functions (remote included) it would make sure that a consistent state and message would be delivered to each function. In your example: the second remote function would receive someInteger with whatever state it had before, once recovered.

To get a deeper understanding of how checkpointing works in Flink and how it enables exactly once processing I’d recommend the following:

https://ci.apache.org/projects/flink/flink-docs-stable/internals/stream_checkpointing.html