javamessage-queuezeromqmessagingjeromq

ZeroMQ brokerless network with queues (Java)


Is it possible to implement a brokerless network with queues using ZeroMQ (with JeroMQ Java porting)?

In my network all peers are both publishers and receivers (SUB/PUB pattern), so that when a peer sends a message all other peers get the message.

The problem is messages are not reliable and can get lost (for example for connectivity issues) and not recovered anymore.

I'd like to implement a queue where peers can retrieve messages they have not received.

I'm looking at this guide (even though it's for Python) and it seems I should implement the XREP/XREQ pattern:

XREP/XREQ layout

but it seems this is possible only implementing a queue server. Is it true?


Solution

  • Q: Is this possible only implementing a queue server?
    A: No.

    May be I did not get your point of view exactly, but having a few years spent inside ZeroMQ based , I can address a few misses in the concept.

    First:
    Yes, Zen-of-Zero does provide ZERO-Warranty for a respective message delivery. This may seem surprising, but there are many reasons for working this way and no other. There is a warranty of consistency - i.e. a message is either delivered as-is or none at all. This means, if the message has made it through the socket, the receiving side may be sure, that the sender was dispatching this very content and no error-checking need be put in place, as the ZeroMQ has already spent all its effort to deliver a 1:1 bit-by-bit copy of the original.

    Next:
    ZeroMQ is designed as a Broker-less asynchronous lightweight signalling / messaging tool. The word Broker-less means, there are zero-efforts spend for any sort of a tool-based persistence, so indeed there is no care about any Broker-side storing any (semi-)persistent replica(s) of the messages, be it those delivered or those not delivered due to whatever technical reason ( yet, those delivered are -- as expressed above -- guaranteed to be OK and an exact copy of the original ).

    Implication:
    this means, there will be zero effect from designing a zmq.device( zmq.Queue, f, b ) as this will have all the properties reported above, so it will principally live under the same set of paradigms.


    Solution?

    If one needs to have both the delivered content-warranty and also the all-messages-delivered warranty, the former is included in the ZeroMQ tools since inception, the latter is to be added on top of the standard tools, as an extended supra-pattern, re-using the delivery-agnostic standard tools.

    This way one can get what you have sketched above, yet not wasting a single CPU-clock in all other use-cases, where delivery-agnostic, just "best-effort" transports are okay.