azure-stream-analyticsazure-queuesmultipleoutputsstream-analytics

Azure Stream Analytics How to handle more than 5 query?


I have created one ASA job and also created one input(eventhub) and 6 output(2 cosmos and 4 service bus queue) and Queries are like below. SA allows to write more than 5 query but giving error in activity logs.Because of which I am getting watermark delay also.

1: Select
             *
          INTO
             CosmosOutput
          FROM
             eventhubinput ;
2:       Select
             id,long,lat,timestamp
          INTO
             CosmosOutput1
          FROM
             eventhubinput ;
3:       Select
             *
          INTO
             SB1
          FROM
             eventhubinput 
           Where <condition>;

4:          Select
             *
          INTO
             SB2
          FROM
             eventhubinput 
           Where <condition>;
5:          Select
             *
          INTO
             SB3
          FROM
             eventhubinput 
           Where <condition1>;
6:          Select
             *
          INTO
             SB4
          FROM
             eventhubinput 
           Where <condition1>;

Question:

How do I write more than 5 query in efficient way? Thanks in advance!


Solution

  • Since you have multiple queries, you could try to allocate the Stream Units Settings.

    Streaming Units (SUs) represents the computing resources that are allocated to execute a Stream Analytics job. The higher the number of SUs, the more CPU and memory resources are allocated for your job. Choosing the number of required SUs for a particular job depends on the partition configuration for the inputs and the query that's defined within the job.

    enter image description here

    Definitely, SUs produces more cost.Another workaround,you could set azure function output to replace some queries.For example, I notice you need to push data into different Service Bus Output with totally same conditions. You could sum them into one query and push same data into Azure function as parameters. Inside Azure function, configure multiple output bindings of service bus.