streamapache-flinkbroadcast

flink how to combine stream and multiply maps


I have one stream (not keyed stream), and 3 maps (each map is result comes from different rest api).

These 3 maps are static, won't change after.

I want to map elements to new type in stream by using these 3 maps, how can I broadcast 3 maps to stream?

As I know join or connect is not sufficient to do this.. please help.


Solution

  • If the maps (as in java map) are static, you can just load them inside a RichMapFunction in open and apply them in map. To increase performance, you should initialize them once in a static variable (sync on the class or a static mutex). If the maps are small, you can also initialize them in your main and just pass them to a MapFunction as a parameter. As long as everything in the map is Serializable, it will just work.