javahazelcastvert.xhazelcast-jet

Convert a single node to distributed java application with Hazelcast

We have a current java application built with Vert.x using simple local HashMaps. In order to distribute our application on several nodes, we would like to replace local HashMaps by a single Hazelcast IMap.

Basically we have our 2 HashMaps :

Map<String, Group> // <groupId - group>
Map<String, Set<String>  // <memberId - set of groupIds>

We have 2 HashMaps in order to query easily our groups by their ids and to know to which groups belong our members.

However, with IMap, according to documentation, it should be possible to create only one map :

IMap<String, Group> // <groupId - group>

class Group {
  String id;
  Set<Member> members;
}

class Member {
  String id;
  // some data
}

and we should be able to perform :

Collection<Group> groups = imap.values(Predicates.in("members", varargs of member ids))

So, I have 4 questions :

  1. Could you confirm this behavior is possible with IMap ?
  2. If yes, is it possible the collection of groups returned by imap.values contain duplicates (same groupId) ? Do I have to perform a kind of distinct operation ?
  3. We are not sure if we should use Vertx + Hazelcast, or move all the processing code to Hazelcast Jet. What would be the advantages to move to Hazelcast Jet only ?
  4. If we use Hazelcast Jet, is it possible to use IMap.values(Predicate) feature inside StreamStage like that :
StreamStage.flatMap(memberIds -> Traversers.traverseIterable(imap.values(Predicates.in("memberIds", memberIds.toArray(new String[0])))

Thanks a lot !

Solution

    1. yes

    2. it should not contain duplicates if your original map doesn't contain them and the IMap isn't concurrently updated

    3. depends on your use case, I can't tell in general

    4. This is not good as the imap.values call is blocking and the function in flatMap must not block. By blocking I mean it does blocking IO operations. Also you can't capture the imap instance from a local variable because it's not serializable and Jet can't send it to the member. But you can use mapUsingService and use ServiceFactories.imapService like this:

     .mapUsingService(
         ServiceFactories.iMapService("my_map").toNonCooperative(),
         (imap, memberIds) -> 
             imap.values(Predicates.in("memberIds", memberIds.toArray(new String[0]))))