javastreamapache-flinkstream-processing

Apache Flink : filtering based on previous value

All filtering examples in apache flink documentation display simple cases of filtering according to a global threshold.

But what if filtering on an entry should take into account the previous entry?

Let's say we have a stream of sensor data. We need to discard the current sensor data entry if it's X% larger than then previous entry.

Is there a simple solution for this? Either in Apache Flink or in plain Java.

Thanks

Solution

  • In flink, this can be done with state.

    Your use case is very similar to the fraud detection example from flink doc.