architecturemessage-queuezeromqpublish-subscribemessaging

What's the right ZeroMQ architecture for my scenario?


I have a service A that every once in a while needs to send a message to a cluster of services B1, B2 ... BN. Then all of these services need to receive that message reliably, and send a confirmation back to A. After A receives the confirmation from all services from cluster B, it needs to send a message to another cluster of services C1, C2 ... CN.

What is the right ZeroMQ pattern for this, and what sockets should I pick?

It seems like a ROUTER-DEALER situation where A is a ROUTER and Bs and Cs are DEALER-s. I need the transmission of these messages to have delivery guarantee so not sure if I can do PUB-SUB here.


Solution

  • Q :
    "What is the right ZeroMQ pattern for this, and what sockets should I pick?"

    I dare reveal a view, collected during last 15(? and counting...) years with ZeroMQ, since v.2.0.

    There is no such "pick", one shall ever choose, when designing a real world application. ZeroMQ built-in Scalable Formal Communication Archetypes patterns are smart, fast, robust, yet -please- do not consider them "complete" to just pick one of them and be ready to go into field practices.

    Both Pieter Hintjens & Martin Sustrik explicitly stated, wherever possible, what is WARRANTED ( and rest is not ).

    So, we have a WARRANTY the ZeroMQ will deliver a bit-exact copy of the original ( or delivered nothing at all & nothing in between these two principal corner states ).

    So, we have to design any and all logic we need "above" the given, built-in primitives, because that way we :

    (a) design exactly what we indeed need to have, and

    (b) do not overdesign anything we do not need to have

    Typical system then consists of messaging ( transport ) layer, where ZeroMQ archetype primitives are operated so as to deliver ( or not ) messages with application related content, and signalling layer, in which we use separately constructed ZeroMQ links, used for signalling app-level heart-beats, for our app-level ACKs / NACKs of delivered or missing transport-messages, for our app-level self-diagnostic and performance-evaluating needs.

    So our effort result in a concept of rather new ecosystem, built ad-hoc from ZeroMQ primitive archetypes, that together, as seen from the application level, compose a smart messaging/signalling meta-plane.

    Such meta-plane can intentionally use many paired PUSH-PULL-s for simple, universal node-interconnects ( there and back ) Transport-layer and use, besides these, several, both universal and service-specific PUB-SUB-s altogether with whatever other patterns, as needed, for Signalling-layer, split across several Context()-instances, so as to fine tune performance and isolate both priorities and resource capacities.

    There is never a "one-size-fits-all" ZeroMQ pattern & we enjoy the smart, super-fast built-in primitive archetypes right for designing our just-enough complex Messaging & Signalling meta-plane to best fit all our needs in our applications.