jgroups

Can JGroups be configured to support work queues?


I'm looking for a distributed, reliable work queue. Where remote processes can take units of work from the queue when they are free.

I'm dealing with a distribution of quite heavy-weight tasks, where consumer microservices can only take one task at a time.

I've seen in examples that JGroups can provide reliable round robin message distribution. But I haven't managed to find any examples where the consumer can choose when to take a task.

Any advice would be greatly appreciated.


Solution

  • JGroups used to have a ReplicatedQueue, but I removed it some time ago. The reason was that I didn't like the semantics it provided: without total order (SEQUENCER), replicas could get out of sync. Also, network partitions were not handled in a meaningful way.

    Besides, JGroups is more about reliable messaging and cluster management, and I didn't want to focus on building blocks too much.

    However, it should be quite simple to create a replicated queue on top of jgroups-raft [1].

    [1] https://github.com/belaban/jgroups-raft