I want to use hazelcast or coherence EntryProcessor to process some logic in parallel execution on different nodes where the keys are stored in a cache. I see i can use something like sendToEachKey(EntryProcessor process).
My problem comes when i need to send also with the logic a piece of data to process too that belongs to another system and i receive it(in a http request for example).
Sure i can do something like sendToEachKey(EntryProcessor(data) process). But if the data it's diferent to each key and i want to send to a specific key only his data to process, how can i do that? Why i want to do that is because the data is too big and I have network overload.
Sure that if I open a thread pool to send each data to each key is possible but it is inefficient because of the huge requests.
Thanks!
For Hazelcast you could retrieve all values and send each key it's own EntryProcessor, however this will create a lot of overhead.
The other option would be to use a combination of EntryProcessor and our distributed ExecutorService.
You send a Runnable to an ExecutorService. Inside the Runnable you retrieve the local keyset, retrieve all the external values (all that already local to the node) and than you emit one EntryProcessor per local key. Since you're already local to the node, there's no more traffic flying around (apart from the backups, obviously :)). That said you might want to implement a specific EntryProcessor that only transmits the changed value but not the full processor itself (to save even more traffic).