rabbitmqvirtual-topic

Rabbit MQ virtual topics?


In Rabbit MQ is there an equivalent to Active MQ Virtual Topics? Or any other mechanism achieve queue semantics. Some topic subscribers need persistent messages delivered when they come back online, and load balancing.

EDIT

There are multiple consumers, few of them have leewway to lose messages since they only act on real time basis; few don't, since they need historical data.

As a trivial example, from the main application a message is published whenever a user fails login. One of the listeners sends an email for every login failure, while another one needs to send an email in case of 5 failures in a day.


Solution

  • I'm not super familiar with ActiveMQ, so this is a bit of a guess based on documentation examples from AMQ.

    Looking at this page (http://activemq.apache.org/what-is-the-difference-between-a-virtual-topic-and-a-composite-destination.html):

    The main difference between a Virtual Topic and a Composite Desetination is that with a Composite Destination the list of consumer destinations is static and hard wired. Whereas with a Virtual Topic at run time a new consumer or queue can be created dynamically and addded to the subscription without having to reconfigure the broker.

    and this page (http://activemq.apache.org/virtual-destinations.html):

    However if the topic is virtual, consumer can consume from a physical queue for a logical topic subscription, allowing many consumers to be running on many machines & threads to load balance the load.

    From what I am reading here, a virtual topic in ActiveMQ is the way that RabbitMQ routing keys and queue bindings work, already.

    You can have a message consumer define a queue with a binding to an exchange, and subscribe to that queue at any time. You could create a single queue that has multiple consumers, as well.

    One of the phrases above, stands out for ActiveMQ: "without having to reconfigure the broker"

    With RabbitMQ, you always have the ability to reconfigure the broker at runtime. There is no separate design-time for the broker. The entire layout of the RabbitMQ topology is defined through the same protocol that produces and consumes messages. This gives you a lot of flexibility and the ability to dynamically create bindings and queues, and consume them as needed. When your consumer is done, it can destroy the queue it had created. No need to redesign or restart RabbitMQ.

    If you need persistence with RabbitMQ, you have a few things to consider. Persistence in the message? (as-in, persisted to disk) or persistence in the queue staying alive and holding messages?

    message persistence (save to disk) allows a message to survive when RabbitMQ dies and comes back online... that is, when the message broker itself goes down and comes back up. This is an incredibly expensive thing to do, compared to holding messages in memory. Sometimes it is important, though, and worth the cost.

    It sounds like you are talking about queue persistence, though... where a queue will stay alive and continue to receive messages even if there are zero consumers attached to the queue. Then, when a consumer attaches to the queue again, it will being receiving messages from the queue. This is a combination of "durable" and "auto-delete" for the queue configuration.

    A durable queue will survive when rabbitmq (the broker) goes down and comes back up. This is probably what you want.

    An "autodelete" queue will delete itself when there are no more consumers connected to the queue. If you need the queue to survive when there are no consumers, be sure to set autodelete to false (this is the default, I think. but it's good to be explicit and set it to false).

    Hope that helps!

    P.S. The idea of "consume from a physical queue for a logical topic subscription" with virtual topics sounds a lot like the "selective consumer" pattern... which is an anti-pattern in RabbitMQ.