integrationapache-stormspark-streamingsolacecustom-receiver

How to Integrate Apache Storm with Solace Message queue to receive messages from Solace .. just like spark streaming integration

I want to receive messages from Solace queues and want to store in any data stores like Hbase/HDFS via Apache Storm.

Solace has published integration guide to setup a custom receiver which will store the messages in Spark in-memory. Other integration guides were also available, except integration with Apache Storm.

I am not sure if this is possible or not. if this is, Can anyone give some pointers on the same(if not solace some other messaging provider) if they have already done this?

Solution

  • Looking at the Apache Storm's documentation, there's two ways to go about this.

    1. Integrate via MQTT - https://storm.apache.org/releases/1.0.0/storm-mqtt.html

    2. Integrate via JMS - https://github.com/ptgoetz/storm-jms

      The storm-jms project provides a SpringJmsProvider where you can make use of the Spring framework (which Solace has a integration guide for) to lookup a ConnectionFactory and Destination.

      Alternatively, you can implement your own org.apache.storm.jms.JmsProvider to provide a ConnectionFactory and Destination without the Spring Framework.