rabbitmqazureservicebusrabbitmq-shovel

Proxying internal traffic to Azure Service Bus

What I'm trying to achieve is interoperability between RabbitMQ clients in an internal network, and Azure Service Bus consumers running in Azure.

The RabbitMQ clients need to publish and subscribe, as do the Azure Service Bus consumers - so I need some kind of 'bi-directional proxy'. A diagram of what I'm trying to achieve:

                                      +
                    Internal network  |  Azure
                                      |
                                      |
+--------+                            |                               +----------+
| Client +---+                        |                           +---+ Consumer |
+--------+   |                        |                           |   +----------+
             |                        |                           |
             | +-----------------+    |    +-------------------+  |
             +-+ RabbitMQ Broker +---------+ Azure Service Bus +--+
             | +-----------------+    |    +-------------------+  |
             |                        |                           |
+--------+   |                        |                           |   +----------+
| Client +---+                        |                           +---+ Consumer |
+--------+                            |                               +----------+
                                      |
                                      |
                                      |
                                      +

AFAIK, both the RabbitMQ broker and Azure Service Bus can do AMQP 1.0. I've looked at the rabbit shovel plugin, but I think this would only handle publishing of messages from the internal clients to Azure, and wouldn't allow the clients to subscribe to messages published by the Azure consumers? Or have I got this wrong, and shovel will work for this?

If shovel won't work, is there any other way to achieve this

Solution

  • I got a response from a couple of Pivotal engineers, over on the rabbitmq-users group.

    I was incorrect - shovel is actually bi-directional, acting as both publisher and consumer, so you can both send and receive messages from a remote broker.

    I should then be able to fulfill my requirements by installing and configuring the shovel plugin in the internal RabbitMQ broker.

    There are some gotchas when connecting to an Azure Service Bus broker though:

    1. If you need to use a queue with sessions, you should use the GroupId property for the session ID (amqpMessage.Properties.GroupId = mySessionId)
    2. If you need to use a queue with partitions, you should use the x-opt-partition-key annotation for the partition ID (amqpMessage.MessageAnnotations[new Symbol("x-opt-partition-key")] = myPartitionKey)