activemq-classicjms-topicmessage-forwarding

Maintain order of messages while forwarding messages between two ActiveMQ brokers


I have an ActiveMQ setup where a source broker living in one data center forwards all messages arriving on certain topics to a destination broker in another data center. The consumer application consumes messages only from the destination broker. (This setup is mainly to ensure fast and efficient forwarding of messages between the two data centers.)

The configuration for forwarding looks something like this:

<networkConnectors>
  <networkConnector name="Q:DontForwardQueueMessages"
    uri="static:(tcp://destination-broker.example.com:61616)"
    duplex="false" decreaseNetworkConsumerPriority="true" networkTTL="2"
    dynamicOnly="true">
    <excludedDestinations>
      <queue physicalName=">" />
    </excludedDestinations>
  </networkConnector>
  <networkConnector name="T:ForwardSampleMessages"
    uri="static:(tcp://destination-broker.example.com:61616)"
    duplex="false" decreaseNetworkConsumerPriority="true" networkTTL="2"
    dynamicOnly="true">
    <excludedDestinations>
      <topic physicalName=">" />
    </excludedDestinations>
    <staticallyIncludedDestinations>
      <topic physicalName="SampleTopic1" />
      <topic physicalName="SampleTopic2" />
      <topic physicalName="SampleTopic3" />
      <topic physicalName="SampleTopic4" />
    </staticallyIncludedDestinations>
  </networkConnector>
</networkConnectors>

Our application needs message order to be maintained. However, we are losing messages when the destination broker goes down. Messages arriving at the source broker pile up in the topic, but do not get forwarded when the connection with the destination broker is re-established. However, messages arriving after re-connection are forwarded as usual.

I'm looking for a way I can configure the setup so that:


Solution

  • It looks like it was a poor design choice to have messages forwarded from a Topic. As per the ActiveMQ documentation:

    Only subscribers who had an active subscription at the time the broker receives the message will get a copy of the message.

    The destination broker acts like a subscriber to the source topic from which messages are being forwarded. So, when messages arrive in the source topic in the absence of a subscriber (destination disconnected), they are not available to anyone.

    As a solution, I changed the design:

    Now since messages at the source are in a queue, they will be consumed in the order in which they were received, and no messages are lost, even if the brokers are disconnected from each other.